An unsupervised learning wavefront restoration method for overcoming background aberration

Abstract

The present invention discloses an unsupervised learning wavefront restoration method for overcoming background aberration. For the first time, the idea of unsupervised learning is introduced into the wavefront restoration method independent of the target. The model is trained using the unsupervised learning mode without making labels. The trained perfect network can perform fast phase inversion for any imaging target. First, a refined feature is proposed, which depends only on the wavefront aberration and is independent of the imaging target. Then, an unsupervised learning model is composed by combining a lightweight neural network and an optical feature system. The neural network is promoted to achieve non-linear fitting between the feature and the wavefront aberration by inversely outputting the fine feature. This method does not require making labels and can effectively overcome the problem of background aberration existing in the actual optical path, realizing high-precision wavefront restoration in any extended target scenario, and has application potential and practical application significance in the field of image-based wavefront restoration.

Description

Technical Field

[0001] The present invention relates to the technical field of object - independent wavefront phase retrieval methods and unsupervised learning models, and particularly relates to an unsupervised learning wavefront restoration method for overcoming background aberration. Background Art

[0002] The phase retrieval method can reconstruct the incident wavefront by iterating multiple times according to a single - frame focal - plane intensity image. The optical system has a simple structure, high light - energy utilization rate, the algorithm is easy to implement, and it can be applied to both point targets and extended targets. When using two intensity images for phase recovery, usually one image is the intensity image on the focal plane and the other is the defocused - plane intensity image, and this method is called phase - difference retrieval (PD). However, the PD algorithm based on numerical iteration has problems such as low efficiency, long time - consumption, and being prone to falling into local optima, so it is limited in some application scenarios that require high real - time and high - precision performance. After J.R.P. Angel et al. from the University of Arizona in the United States first used deep - learning technology to achieve wavefront retrieval, neural networks have gradually been introduced into the field of wavefront measurement by scientists, and a large number of literatures on wavefront - free sensing and other aspects have emerged in recent years. Ma Huimin et al. from Anhui Agricultural University modified AlexNet, simulated and generated focal - plane and defocused - plane images under different atmospheric turbulence parameters, used these images as inputs, and trained to output the first 35 - order Zernike coefficients. Similarly, Wu Yu, Guo Youming et al. from the Institute of Optoelectronics, Chinese Academy of Sciences also input focal - plane and defocused - plane images into the neural network and output the 13 - order Zernike coefficients. In recent years, Qi Xin et al. and Li Dequan et al. used time - domain or sharpness information in focal and defocused images to eliminate image dependence, providing new ideas for the research of extended - object wavefront sensing.

[0003] The above - mentioned wavefront restoration methods based on neural networks often require far - field images and their corresponding wavefront aberration labels. To obtain the wavefront aberration labels corresponding to far - field images, a deformable mirror or a spatial light modulator needs to be introduced into the optical path to generate aberrations, and then the far - field images corresponding to each frame of aberration are collected in turn. However, due to the inevitable background aberration in the actual optical path, there are often certain deviations between the far - field images obtained by the above method and the collected labels, that is, the actual wavefront aberration corresponding to the far - field image is not the label we made. In addition, a large number of labeled data samples used for training greatly increase the application cost and difficulty.

[0004] Therefore, the present invention proposes an unsupervised learning wavefront restoration method for overcoming background aberration. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: to solve the problem of insufficient restoration accuracy caused by the inability of existing wavefront restoration deep learning networks to overcome the background aberration of optical systems. The present invention proposes an unsupervised learning wavefront restoration method to overcome the background aberration, which can achieve high-precision target-independent wavefront restoration while overcoming the background aberration existing in the actual optical path.

[0006] The technical solution adopted by the present invention to solve the above problems is: an unsupervised learning wavefront restoration method to overcome the background aberration, and the specific implementation steps are as follows:

[0007] Step 1: Design an optical imaging system based on far-field images, including the design of the central wavelength of the incident light, the focal length of the lens, the entrance pupil radius, and the defocus parameter.

[0008] Step 2: Introduce random aberrations that conform to the atmospheric transmission model into the ideal parallel light, and then introduce a random aberration to simulate the background aberration existing in the optical system.

[0009] Step 3: After obtaining the in-focus and defocused far-field pictures, perform fine feature extraction, eliminate the target information, and retain the aberration information.

[0010] Step 4: Record the feature image and its corresponding near-field wavefront data, and use the feature image as a sample to make a wavefront restoration data set based on an unsupervised learning model.

[0011] Step 5: Select the first 80% of the samples in the data set as the training set to enable the network to learn the non-linear mapping relationship between the feature image and the near-field wavefront; the remaining 20% of the data set is used as the validation set and the test set in a 1:1 manner to measure the accuracy and real-time performance of the method.

[0012] Step 6: Configure the deep learning environment and build a neural network.

[0013] Step 7: Establish a target-independent optical feature system according to the optical system parameters in Step 1, and inversely calculate the fine features.

[0014] Step 8: Compare the inversely calculated fine features with the input fine features, and calculate the loss value to promote the network to update the parameters in an unsupervised learning mode.

[0015] Among them, the parameters such as the central wavelength of the incident light, the focal length of the lens, the entrance pupil radius, and the defocus amount in Step 1 should correspond strictly to the parameters in the optical feature system in Step 7.

[0016] Among them, the introduced background small aberration in Step 2 can be static or dynamically changing.

[0017] Among them, the division method of the training set, validation set and test set in Step 5 can be appropriately changed according to actual needs.

[0018] Among them, in the said step 6, it is preferred to build a lightweight network to obtain shorter inference time and higher wavefront restoration efficiency.

[0019] Among them, in the said step 7, the implementation process of the optical feature system for inversely calculating the fine features is as follows. First, convert the Zernike coefficients output by the neural network into wavefront phases, then calculate the point spread functions on the focal plane and defocus plane according to the wavefront phases, and finally use the two point spread functions to inversely calculate the corresponding unique feature map.

[0020] Among them, in the said step 8, the loss value is calculated by comparing the fine features input to the neural network and the fine features inversely calculated by the optical feature system, and the L1 loss function is used as the loss function.

[0021] The principle of the present invention is as follows: First, extract a kind of feature related to wavefront aberration and independent of the target from the acquired in-focus and defocus images of the target and send it into the neural network. Then establish the combination of the optical feature system and the neural network for training in an unsupervised learning mode. The process of the unsupervised learning mode is: (1) After receiving the feature picture, the neural network outputs Zernike coefficients; (2) Use the output Zernike coefficients as intermediate variables and send them into the optical feature system to inversely calculate the feature picture; (3) Compare and calculate the loss value between the input feature map of the neural network and the output feature map of the optical feature system, and then promote the neural network to update its parameters, and finally realize the unsupervised wavefront restoration independent of the target.

[0022] The advantages of the present invention compared with the prior art are as follows:

[0023] (1) The feature extraction method proposed by the present invention can extract fine feature images that are independent of the extended target and correspond one-to-one with the aberration for the extended target, thereby realizing wavefront restoration in any application scenario.

[0024] (2) The present invention combines the neural network and the feature optical system to form an unsupervised learning model, which can realize network training and parameter update without label values.

[0025] (3) The present invention uses the lightweight AM-EffNet, which has high wavefront restoration accuracy and strong real-time performance. After testing, the time for inferring the first 20-order Zernike coefficients is about 3 ms.

[0026] (4) The wavefront restoration method proposed by the present invention can effectively overcome the background aberration problem existing in the optical system, making the wavefront restoration result closer to the true value. Description of the Drawings

[0027] Figure 1Schematic diagram of an unsupervised learning wavefront restoration method for overcoming background aberration proposed by the present invention;

[0028] Figure 2 Fine feature extraction result diagram for different imaging targets under the same turbulence condition;

[0029] Figure 3 Schematic diagram of the neural network architecture adopted by the present invention;

[0030] Figure 4 Wavefront restoration example diagram of the method proposed by the present invention. Specific implementation manners

[0031] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with specific implementation cases and with reference to the accompanying drawings.

[0032] Figure 1 It is a schematic diagram of the working principle of an unsupervised learning wavefront restoration method for overcoming background aberration, Figure 2 It is a fine feature extraction result diagram for different imaging targets under the same turbulence condition, Figure 3 It is a neural network architecture diagram adopted by the present invention, Figure 4 It is a wavefront restoration result example diagram of the method proposed by the present invention. The specific implementation process is as follows:

[0033] Step 1: Design an optical imaging system based on far-field images, including the design of parameters such as the central wavelength of the incident light, the focal length of the lens, the entrance pupil radius, and the defocus amount. The specific design parameters are as follows: the central wavelength of the incident light is 520 nm, the focal length of the lens is 200 mm, the entrance pupil radius is 6 mm, and the defocus amount is 520 nm;

[0034] Step 2: Introduce 15,000 groups of random aberrations that conform to the atmospheric transmission model with an average incident wavefront root mean square (RMS) of 0.5λ to the ideal parallel light, and then introduce a defocus aberration with a peak-to-valley value (PV) of 0.5λ. Assume that this aberration is the background aberration existing in the optical system;

[0035] Step 3: After obtaining the far-field light intensity distributions in focus and defocus, perform fine feature extraction, eliminate the target information, and retain the aberration information;

[0036] Step 4: Record the feature image and its corresponding near-field wavefront data, and use the feature image as a sample to make a wavefront restoration data set based on an unsupervised learning model;

[0037] Step 5: Select the first 80% of the samples in the data set as the training set to allow the network to learn the non-linear mapping relationship between the feature image and the near-field wavefront; the remaining 20% of the data set is used as the validation set and the test set in a 1:1 manner to measure the accuracy and real-time performance of the method;

[0038] Step 6: Configure the deep learning environment and build a neural network. The specific network is an efficient network based on the attention mechanism (AM-EffNet), and the network architecture is as Figure 3 shown. The adopted neural network consists of three EffNet-blocks, three attention mechanism layers, and one fully connected layer. Among them, the EffNet-block uses depthwise separable convolutional layers to replace traditional convolutional layers, greatly reducing the computational complexity. The attention mechanism layer uses channel attention combined with spatial attention mechanism. By adopting the attention mechanism in two different dimensions, image features with attention mechanism weights are generated, thereby improving the feature extraction ability of the network;

[0039] Step 7: Establish an object-independent optical feature system according to the optical system parameters in Step 1, and inversely calculate the fine features;

[0040] Step 8: Compare the inversely calculated fine features with the input fine features, and calculate the loss value to promote the network to update parameters in an unsupervised learning mode.

[0041] After the training makes the network converge, only by inputting a single-frame fine feature image to the network, the network can output the near-field wavefront information corresponding to the fine feature. Compared with the second-level restoration speed of the traditional phase difference method, the calculation speed of this method is greatly improved, and this method can be applied to wavefront restoration of imaging targets in any scenario. The simulation results show that the time required for the present invention to complete one wavefront restoration is about 3 milliseconds.

[0042] In Step 3, far-field images on the in-focus and out-of-focus planes of the mobile phone are required. The images collected by the CCD camera can be obtained by the convolution of the imaging target o and the point spread function h of the optical system on the observation plane:

[0043] i = o(x, y) * h(x, y),

[0044] Under the near-field approximation, the point spread function h(x, y) corresponding to the in-focus image is:

[0045]

[0046] where is the Fourier transform operator, P(x, y) represents the generalized pupil function of the system, and A(x, y) represents the binary aperture function, which is 1 inside the pupil and 0 outside. In the present invention, a set of Zernike polynomials is used to represent the distorted wavefront:

[0047]

[0048] The point spread function corresponding to the out-of-focus plane is:

[0049]

[0050] Defocus aberration Corresponding to the fourth term of the Zernike polynomial:

[0051]

[0052] where A4 is the fourth Zernike coefficient and Z4(x, y) is the fourth Zernike polynomial. Then the defocus amount δ caused by the defocus aberration satisfies the following relationship:

[0053]

[0054] The spectral information of the image can be obtained by performing a Fourier transform on i = o(x, y) * h(x, y):

[0055] O(u, v) = O(u, v) · OTF(u, v)

[0056] where O(u, v) is the spectral information of the image collected by the detector, O(u, v) is the spectral information of the detection target, and OTF(u, v) is the optical transfer function of the optical system.

[0057] Amplitude feature matrix M power and sharpness feature matrix M sharpness are calculated from the spectra of the in-focus and defocus images:

[0058]

[0059]

[0060] where I * (u, v) and are the complex conjugates of the spectra of the in-focus and defocus images. In the present invention, the fine feature is defined as:

[0061]

[0062] It can be seen from the above formula that this feature extraction method eliminates the imaging target information and retains the aberration information. The wavefront aberration on the focal plane can be represented by a set of Zernike coefficients: α = (α4, α5,..., α 20 ), and the wavefront aberration on the defocus plane can be expressed as: α d = (A4 + α4, α5,..., α 20 ), where A4 represents the fourth-order Zernike coefficient. Using the feature image M fine as the input, the unique near-field wavefront can be inversely calculated using a neural network.

[0063] The unsupervised learning model consists of a neural network and an optical feature system independent of labels. As Figure 3 (a) shows, in order to obtain efficient wavefront sensing capabilities, a lightweight network based on the attention mechanism (AM-EffNet) is proposed, which consists of 3 EffNet blocks, 3 CBAMs, and a fully connected layer. In addition, a Dropout operation is added to the fully connected layer to avoid overfitting during training. Figure 3 (b) shows the structure of the EffNet-block. The EffNet-block uses depthwise separable convolutions to reduce computational complexity and make the network more real-time. In the present invention, the convolutional attention mechanism (CBAM) is used: an integrator of channel attention and spatial attention, which can generate image features with attention mechanism weights from two dimensions, thereby improving the feature extraction ability of the network. The schematic diagram of this structure is shown in Figure 3 (c).

[0064] The Zernike coefficients α′ = (α4′, α5′,..., α 20 ′) output by AM-EffNet represent the wavefront aberration on the focal plane deduced by the network. Then the wavefront aberration on the defocus plane is α d ′ = (A4 + α4′, α5′,..., α 20 ′). Then the point spread functions on the in-focus and defocus planes predicted by the network can be calculated in sequence:

[0065]

[0066]

[0067] Finally, we can inversely calculate the refined features deduced by the unsupervised model:

[0068]

[0069] Using the L1 loss function to calculate the true refined feature M fine and the l fine value of the refined feature M oss ′ deduced by AM-EffNet can promote the effective training of the neural network. After the network is trained and mature, fast wavefront restoration for any imaging target can be achieved. Figure 4 This is a set of wavefront restoration example diagrams of the method proposed in the present invention. For extended targets in two different imaging scenarios, it is proved from two indicators: structural similarity (SSIM) and root mean square error of the residual wavefront (RMSE) that the unsupervised learning wavefront restoration method proposed in the present invention can effectively overcome the influence of the background aberration existing in the optical system, making the restored wavefront closer to the true wavefront and achieving wavefront detection for multiple imaging targets with high precision.

[0070] As described above, it is only the specific implementation manner in the present invention, but the protection scope of the present invention is not limited thereto. Any transformation or replacement that can be understood and conceived by those familiar with the technology within the technical scope disclosed by the present invention should be covered within the scope of the present invention.

Claims

1. An unsupervised learning wavefront restoration method for overcoming background aberration, characterized in that, It is achieved through the following steps: Step 1: Design an optical imaging system based on far-field images, including the design of the central wavelength of the incident light, the focal length of the lens, the entrance pupil radius, and the defocus parameter; Step 2: Introduce random aberrations that conform to the atmospheric transmission model into the ideal parallel light, and then introduce another random aberration to simulate the background aberration existing in the optical system; Step 3: After obtaining the in-focus and defocus far-field images, perform fine feature extraction, eliminate the target information, and retain the aberration information; Step 4: Record the feature images and their corresponding near-field wavefront data, and use the feature images as samples to produce a wavefront restoration data set based on an unsupervised learning model; Step 5: Select the first 80% of the samples in the data set as the training set for the network to learn the non-linear mapping relationship between the feature images and the near-field wavefront; the remaining 20% of the data set is used as the validation set and the test set in a 1:1 manner to measure the accuracy and real-time performance of the method; Step 6: Configure the deep learning environment and build a neural network; Step 7: Establish an object-independent optical feature system according to the optical system parameters in Step 1, and inversely calculate the fine features; Step 8: Compare the inversely calculated fine features with the input fine features, and calculate the loss value to promote the network to update parameters in an unsupervised learning mode.

2. The unsupervised learning wavefront restoration method for overcoming the background aberration according to claim 1, characterized in that: In Step 1, the central wavelength of the incident light, the focal length of the lens, the entrance pupil radius, and the defocus parameter should strictly correspond to the parameters in the optical feature system in Step 7.

3. The unsupervised learning wavefront restoration method for overcoming the background aberration according to claim 1, wherein: The background aberration introduced in Step 2 can be static or dynamically changing.

4. An unsupervised learning wavefront restoration method for overcoming background aberration according to claim 1, characterized in that: The division method of the training set, validation set, and test set in Step 5 can be appropriately changed according to actual needs.

5. An unsupervised learning wavefront restoration method for overcoming background aberration according to claim 1, characterized in that: In Step 6, it is preferred to build a lightweight network to obtain a shorter inference time and higher wavefront restoration efficiency.

6. A method for unsupervised learning wavefront restoration to overcome the background aberration according to claim 1, characterized in that: In Step 7, the implementation process of inversely calculating the fine features by the optical feature system is as follows. First, convert the Zernike coefficients output by the neural network into wavefront phases, then calculate the point spread functions on the focal plane and the defocus plane according to the wavefront phases, and finally inversely calculate the corresponding unique feature map using the two point spread functions.

7. The unsupervised learning wavefront restoration method for overcoming the background aberration according to claim 1, characterized in that: In Step 8, the loss value is calculated by comparing the fine features input to the neural network with the fine features inversely calculated by the optical feature system, and the L1 loss function is used as the loss function.

Images