Atmospheric turbulence wavefront reconstruction and image restoration method based on self-supervised learning
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2026-05-14
- Publication Date
- 2026-08-07
AI Technical Summary
本发明旨在克服现有监督学习在湍流波前探测中数据量大、物理解释性差和传统自适应光学系统成本过高，环境鲁棒性差等方面的不足，在减少数据输入的同时，实现了高精度的波前与图像的重建
[0015]本发明的有益效果在于，对于目前利用传统自适应光学系统进行大气湍流波前重建的成本高、系统结构复杂和利用监督学习进行大气湍流波前重建所遇到的数据集量大，物理解释性差以及湍流畸变图像恢复过程复杂的问题，提出了一种基于自监督学习的大气湍流波前重建与图像恢复方法。该方法通过监督学习预训练的自监督学习模式，降低了数据集的量，显著提高了训练效率；采用图像与PSF卷积的形式，提高了模型的物理解释性和重建精度；同时该方法可以重建图像，显著降低了模型的复杂性。
Smart Images
Figure CN122530003A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of atmospheric turbulence wavefront detection technology, and specifically to a method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning in atmospheric turbulence wavefront detection. Background Technology
[0002] Due to the randomness and spatiotemporal variation of atmospheric turbulence, images acquired through turbulence suffer from blurring, geometric distortion, and intensity fluctuations, severely impacting image quality. Wavefront reconstruction and correction for atmospheric turbulence has always been an important direction in the field of wavefront sensing. However, currently, both traditional adaptive optics systems and existing deep learning methods have limitations in wavefront reconstruction and turbulence distortion image restoration for atmospheric turbulence.
[0003] Traditional adaptive optics systems are complex, expensive, and have poor environmental robustness. For example, the measurement accuracy of traditional Shack-Hartmann wavefront sensors decreases significantly when the environment changes (such as temperature and vibration). Supervised learning methods for wavefront detection require large datasets to train the network, which is difficult to obtain in reality. Due to the complex characteristics of turbulence, deep learning network models often involve complex layers, leading to high training difficulty, slow convergence, and high memory requirements, making them unsuitable for limited experimental platforms. Common supervised learning methods may only learn some input-output mappings through loss functions, without modeling the physical properties of the wavefront, resulting in reconstructions that may not conform to conventional physical laws. Image restoration from turbulence-affected distortions often requires wavefront phase detection and wavefront corrector compensation, a relatively complex process.
[0004] To address the aforementioned problems, researchers have proposed various improvement methods. For example, researchers have combined deep learning and adaptive optics systems for wavefront reconstruction of atmospheric turbulence. Using sub-aperture dot matrix images obtained from Shack-Hartmann sensors as input, they can directly output the corresponding Zernike polynomial coefficients or the reconstructed wavefront phase, enhancing physical interpretability and reducing dataset complexity. Other researchers have utilized reinforcement learning for wavefront reconstruction of atmospheric turbulence, improving the model's generalization ability. For the restoration of turbulent distortion images, various end-to-end deep learning network models, such as CNNs and UNet, directly achieve deblurring at the image level. However, these methods only address specific aspects of the problem and lack a comprehensive and effective solution.
[0005] Therefore, this invention aims to avoid increasing the training difficulty by using a large dataset through self-supervised learning, generate initial estimated aberrations by using supervised learning pre-training, complete image feature extraction and reconstruction using a simple and effective MLP-based coordinate neural network, and use point spread function (PSF) convolution as a physical constraint to achieve atmospheric turbulence wavefront phase estimation and clear image reconstruction. Summary of the Invention
[0006] This invention proposes a self-supervised learning algorithm for turbulent wavefront estimation and image reconstruction based on a coordinate neural network architecture. A supervised learning pre-training process and physical constraints are added to improve reconstruction accuracy. Supervised learning pre-training is performed using clear ground-truth images and turbulent images simulated using the Zernike polynomial method. The corresponding Zernike aberration coefficients are obtained as constraints, improving wavefront reconstruction accuracy. Image reconstruction is performed using a coordinate neural network, and a simulated turbulent image is obtained through convolution. The loss is calculated between this simulated turbulent image and the input turbulent image, achieving self-supervised learning. This invention aims to overcome the shortcomings of existing supervised learning methods in turbulent wavefront detection, such as large data volume, poor physical interpretability, high cost of traditional adaptive optics systems, and poor environmental robustness. It achieves high-precision wavefront and image reconstruction while reducing data input. The specific technical solution of this invention is as follows: A method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning, characterized by the following steps: S1. Construct the dataset, select one image as the ground truth, use the Zernike polynomial method to simulate and generate the corresponding turbulence distortion image, use two images as input, and label the corresponding Zernike polynomial coefficients; S2. Establish a neural network model. In the pre-training stage, a neural network model based on CNN + fully connected network is used to initially generate pre-trained aberrations. In the self-supervised learning stage, a model based on coordinate neural network is used to achieve image-level reconstruction. S3. Perform pre-training: Pre-train the network model using the constructed dataset and the network model from the pre-training stage. Use RmsLoss as the loss function and Adam optimizer to initially generate the pre-trained aberrations. S4. In the self-supervised learning stage, a turbulence distortion image from the dataset is input, and the network model in the self-supervised learning stage is used to realize the image output. At the same time, the Zernike polynomial coefficients generated in the pre-training stage are used as the initial values of the aberration iteration. The reconstructed aberration is updated by the optimizer. The PSF can be calculated from the reconstructed aberration through Fourier transform and pupil function. Then, the reconstructed image is convolved with the PSF, and the result is used to calculate the loss with the input. The network parameters and aberration coefficients are updated. S5. Calculate the SSIM and PSNR of the reconstructed image and the input sharp image, and compare them with the calculated SSIM and PSNR of the turbulent image and the sharp image to quantify the image reconstruction effect. Then compare the difference between the reconstructed aberrations and the true aberrations, including the comparison of the values of each term of the Zernike polynomial and the error, to quantify the aberration reconstruction effect.
[0007] The ground-truth image in step S1 can be a real picture taken in the experiment or a simulated picture. Its main function is to serve as a benchmark for simulating the generation of turbulence images and verifying the image indicators of the reconstructed images.
[0008] The pre-training stage network in step S2 is a neural network model based on CNN+fully connected network, but it is not limited to this structure. For example, it can be extended to a neural network based on the Transformer structure.
[0009] The coordinate neural network encoding method mentioned in step S2 mainly uses... Figure 3 The `input_coord_2d` layer normalizes the coordinate grid (range -1 to 1) of all pixels in an image and flattens it into a list of coordinate points for use by subsequent networks. The set of all coordinate points is as follows: in The width of the input image. The height of the input image. Indexed in the width direction. This is an index for the height direction.
[0010] The loss function, optimizer type, and training parameters in step S3 can be modified depending on the dataset, and are not limited to RmsLoss, Adam, etc.
[0011] The reconstructed aberrations in step S4 are Zernike polynomial coefficients. The corresponding wavefront phase can be generated according to the Zernike polynomial method, and the corresponding PSF can be generated through wavefront diffraction calculation. The specific method is as follows: in, Indicates wavefront phase, This indicates the number of terms in the Zernike coefficients. Indicates the first Term coefficient, For the first The coefficient value of the zernike term. For the first The basis functions of the Zernike coefficients.
[0012] in Indicates Fourier transform, This is the generalized pupil function, taking the value 1 within the pupil and 0 elsewhere. Indicates beam.
[0013] The aberration constraint and loss in step S4 can be expressed by the following formula: in Represents structural similarity loss. Represents the total variation loss. This represents the relative standard deviation loss. These are the weighting coefficients for the aberration term. The aberrations are obtained from pre-training.
[0014] The index calculation in step S5 is performed using a MATLAB simulation platform, but is not limited to this method.
[0015] The beneficial effects of this invention are that it addresses the problems of high cost and complex system structure in current atmospheric turbulence wavefront reconstruction using traditional adaptive optics systems, and the large dataset size, poor physical interpretability, and complex turbulence distortion image restoration process encountered in supervised learning-based atmospheric turbulence wavefront reconstruction. This invention proposes a self-supervised learning-based method for atmospheric turbulence wavefront reconstruction and image restoration. This method reduces the dataset size and significantly improves training efficiency through a self-supervised learning model with pre-trained supervised learning; it improves the physical interpretability and reconstruction accuracy of the model by employing image-PSF convolution; and it can reconstruct images, significantly reducing the complexity of the model. Attached Figure Description
[0016] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings: Figure 1 The following is a flowchart of a method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning, provided by an embodiment of the present invention. Figure 2The pre-training process of the CNN+ fully connected neural network provided in this embodiment of the invention is illustrated in the diagram. Figure 3 The network structure diagram based on coordinate neural network in the self-supervised learning process provided in this embodiment of the invention; Figure 4 The method provided in this embodiment of the invention provides a comparison result of the reconstructed image and aberrations with the true values during the testing phase. Figure 2 In the input image, there is a pair of clear images and a simulated turbulent blurred image. The output label is a set of 52 Zernike coefficients with the first 3 terms removed. Figure 3 In the middle: The function of Input_coord_2d is to generate a two-dimensional coordinate grid and flatten it into a format suitable for MLP input; Figure 4 In the study, comparisons were made between real and simulated turbulence images, SSIM and PSNR of reconstructed images, and the Zernike coefficients and errors of reconstructed and real aberrations were compared. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.
[0018] A method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning, characterized by the following steps: S1. Construct the dataset, select one image as the ground truth, use the Zernike polynomial method to simulate and generate the corresponding turbulence distortion image, use two images as input, and label the corresponding Zernike polynomial coefficients; S2. Establish a neural network model. In the pre-training stage, a neural network model based on CNN + fully connected network is used to initially generate pre-trained aberrations. In the self-supervised learning stage, a model based on coordinate neural network is used to achieve image-level reconstruction. S3. Perform pre-training: Pre-train the network model using the constructed dataset and the network model from the pre-training stage. Use RmsLoss as the loss function and Adam optimizer to initially generate the pre-trained aberrations. S4. In the self-supervised learning stage, a turbulence distortion image from the dataset is input, and the network model in the self-supervised learning stage is used to realize the image output. At the same time, the Zernike polynomial coefficients generated in the pre-training stage are used as the initial values of the aberration iteration. The reconstructed aberration is updated by the optimizer. The PSF can be calculated from the reconstructed aberration through Fourier transform and pupil function. Then, the reconstructed image is convolved with the PSF, and the result is used to calculate the loss with the input. The network parameters and aberration coefficients are updated. S5. Calculate the SSIM and PSNR of the reconstructed image and the input sharp image, and compare them with the calculated SSIM and PSNR of the turbulent image and the sharp image to quantify the image reconstruction effect. Then compare the difference between the reconstructed aberrations and the true aberrations, including the comparison of the values of each term of the Zernike polynomial and the error, to quantify the aberration reconstruction effect.
[0019] The ground-truth image in step S1 can be a real picture taken in the experiment or a simulated picture. Its main function is to serve as a benchmark for simulating the generation of turbulence images and verifying the image indicators of the reconstructed images.
[0020] The pre-training stage network in step S2 is a neural network model based on CNN+fully connected network, but it is not limited to this structure. For example, it can be extended to a neural network based on the Transformer structure.
[0021] The coordinate neural network encoding method mentioned in step S2 mainly uses... Figure 3 The `input_coord_2d` layer normalizes the coordinate grid (range -1 to 1) of all pixels in an image and flattens it into a list of coordinate points for use by subsequent networks. The set of all coordinate points is as follows: Where W is the width of the input image, H is the height of the input image, i is the index in the width direction, and j is the index in the height direction.
[0022] The loss function, optimizer type, and training parameters in step S3 can be modified depending on the dataset, and are not limited to RmsLoss, Adam, etc.
[0023] The reconstructed aberrations in step S4 are Zernike polynomial coefficients. The corresponding wavefront phase can be generated according to the Zernike polynomial method, and the corresponding PSF can be generated through wavefront diffraction calculation. The specific method is as follows: in, Indicates wavefront phase, This indicates the number of terms in the Zernike coefficients. Indicates the first Term coefficient, For the first The coefficient value of the zernike term. For the first The basis functions of the Zernike coefficients.
[0024] in Indicates Fourier transform, This is the generalized pupil function, taking the value 1 within the pupil and 0 elsewhere. Indicates beam.
[0025] The aberration constraint and loss in step S4 can be expressed by the following formula: in Represents structural similarity loss. Represents the total variation loss. This represents the relative standard deviation loss. These are the weighting coefficients for the aberration term. The aberrations are obtained from pre-training.
[0026] The index calculation in step S5 is performed using a MATLAB simulation platform, but is not limited to this method.
[0027] Example 1: The workflow of a self-supervised learning-based method for atmospheric turbulence wavefront reconstruction and image restoration is as follows: As described in step S1, the dataset is first established. This invention selects a checkerboard image taken in the laboratory as the ground truth, cropping it to 256*256 pixels to save training resources. The corresponding turbulence distortion image is generated using the Zernike polynomial method. The Zernike polynomial has 52 coefficients (excluding the first three terms). The ground truth image and the turbulence distortion image are used as inputs, and the corresponding Zernike polynomial coefficients are used as labels to construct the dataset.
[0028] As described in step S2, a neural network model is built. The model used in the pre-training stage is a network model based on CNN + fully connected neural network, such as... Figure 2 As shown, this network takes a pair of turbulent and clear images as input and trains to obtain the corresponding Zernike aberration coefficients, which can then be used as initial values for subsequent self-supervised learning iterations. The self-supervised learning stage employs a coordinate neural network-based model, such as... Figure 3 As shown, by inputting a turbulence image, a clear image can be directly reconstructed based on subsequent loss calculations, without the need for labels.
[0029] As described in step S3, perform pre-training, based on... Figure 2 The network model is then developed, and relevant parameters are adjusted, such as the learning rate, loss function, and optimizer. The model is then trained using the dataset constructed in step S1. After 50 epochs of training, a relatively accurate mapping from input to label can be obtained.
[0030] As described in step S4, self-supervised learning is performed. The process of this method is shown in Figure 1. By inputting a turbulence distortion image, and utilizing... Figure 3 The coordinate neural network shown performs image reconstruction, and then convolves the reconstructed image with the PSF calculated from the reconstructed aberrations. The reconstructed aberrations are constrained by the aberration loss obtained from the pre-training in step S3. The result after convolution is compared with the input for loss calculation, and the network parameters and aberration coefficients are continuously updated. After setting the relevant parameters, the input image is first normalized to facilitate pixel mapping in the subsequent coordinate neural network. This experiment trained for 800 epochs to obtain the corresponding reconstructed image and reconstructed aberrations.
[0031] As described in step S5, the indicators were verified, and Matlab was used to calculate the indicators. The results are as follows: Figure 4 As shown, the method includes calculating the SSIM and PSNR of the reconstructed image and the sharp image, and comparing them with the SSIM and PSNR of the turbulent image and the sharp image to verify the image-level restoration effect; calculating the difference between the reconstructed aberration and the true aberration, comparing the values of each term of the Zernike polynomial and the error, and quantifying the aberration reconstruction effect.
[0032] The hardware and software equipment used in the method of this invention are: Windows 10 operating system, GeForce 1650Ti graphics card, Python 3.8 programming language, PyTorch 1.9.1 deep learning framework, PyCharm compilation environment, and MATLAB 2024a simulation platform.
[0033] In summary, this invention relates to a self-supervised learning-based method for atmospheric turbulence wavefront reconstruction and image restoration. It aims to address the problems of high cost and complex system structure associated with traditional adaptive optics systems for turbulence wavefront reconstruction, and the large dataset size, poor physical interpretability, and complex image restoration processes encountered with supervised learning for atmospheric turbulence wavefront reconstruction. This technique achieves fast and efficient wavefront reconstruction and image restoration of turbulence-distorted images through a pre-trained self-supervised learning method. Compared to conventional supervised learning methods, this technique does not require a large dataset, increases physical interpretability through image-PSF convolution, and the network can directly reconstruct clear images, simplifying model complexity.
[0034] In practical applications, this method demonstrates good wavefront reconstruction accuracy and image restoration performance. Tests show that the reconstructed images and ground-truth images have high SSIM and PSNR values. Furthermore, the SSIM and PSNR values of the reconstructed images and ground-truth images are significantly higher. The error between the reconstructed Zernike coefficient and the true Zernike coefficient is also very small. This method is suitable for wavefront detection in some scenarios without wavefront atmospheric turbulence.
Claims
1. A method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning, characterized in that... The method includes the following steps: S1. Construct the dataset, select one image as the ground truth, use the Zernike polynomial method to simulate and generate the corresponding turbulence distortion image, use two images as input, and label the corresponding Zernike polynomial coefficients; S2. Establish a neural network model. In the pre-training stage, a neural network model based on CNN + fully connected network is used to initially generate pre-trained aberrations. In the self-supervised learning stage, a model based on coordinate neural network is used to achieve image-level reconstruction. S3. Perform pre-training: Pre-train the network model using the constructed dataset and the network model from the pre-training stage. Use RmsLoss as the loss function and Adam optimizer to initially generate the pre-trained aberrations. S4. In the self-supervised learning stage, a turbulent distortion image from the dataset is input, and the network model in the self-supervised learning stage is used to realize the image output. At the same time, the Zernike polynomial coefficients generated in the pre-training stage are used as the initial values of the aberration iteration. The reconstructed aberration is updated by the optimizer. The PSF can be calculated from the reconstructed aberration through Fourier transform and pupil function. Then, the reconstructed image is convolved with the PSF, and the result is used to calculate the loss with the input to update the network parameters and aberration coefficients. S5. Calculate the SSIM and PSNR of the reconstructed image and the input sharp image, and compare them with the calculated SSIM and PSNR of the turbulent image and the sharp image to quantify the image reconstruction effect. Then compare the difference between the reconstructed aberrations and the true aberrations, including the comparison of the values of each term of the Zernike polynomial and the error, to quantify the aberration reconstruction effect.
2. The method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning according to claim 1, characterized in that, The dataset images in step S1 can be resized to reduce the amount of training resources required and achieve faster network convergence.
3. The method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning according to claim 1, characterized in that, The network model in the pre-training stage of step S2 is a CNN+fully connected network, which can be extended to, for example, a transformer-based neural network.
4. The method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning according to claim 1, characterized in that, The input to the pre-trained network model in step S3 is a pair of clear images and turbulence images, labeled with the Zernike coefficients corresponding to the simulated turbulence.
5. The method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning according to claim 1, characterized in that, The self-supervised learning stage in step S4 uses a simple network structure with fast training speed. It can achieve wavefront reconstruction and image restoration without additional datasets or manual operation, and can also train the network for systems with low computing power.
6. The method for atmospheric turbulence wavefront reconstruction and image restoration based on self-supervised learning according to claim 1, characterized in that, The clear image and true aberrations in step S5 are the ground-truth used when simulating the turbulence dataset, which are used to quantify the accuracy of reconstructed aberrations and reconstructed images.