An extended target super-resolution wavefront restoration method

Abstract

The present invention provides a super-resolution wavefront restoration method for extended targets, applied to a Shack-Hartmann wavefront sensor. First, a feature image independent of the extended target shape is extracted from the image captured by the wavefront sensor's focal plane camera. This feature image is then calculated using a trained artificial neural network to directly obtain the mode coefficients of the incident light wavefront of the Hartmann wavefront sensor. Compared to the correlation wavefront restoration method commonly used for extended target wavefront detection, the method provided by the present invention is not only unaffected by the extended target shape but also capable of extracting more information from the wavefront sensor image for wavefront restoration. Under the same focal plane camera measurement noise intensity, this method can restore higher-order mode coefficients with higher accuracy than the correlation method, resulting in a smaller residual error in the restored wavefront.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical information measurement, and specifically relates to a method for measuring optical wavefront, and more particularly to a method for restoring an extended target super-resolution wavefront. Background Art

[0002] The existing modal wavefront restoration method of the Hartmann wavefront sensor usually reconstructs the coefficients of the aberration mode based on the average slope of each sub-aperture (Guang-ming Dai, "Modal wave-front reconstruction with Zernike polynomials and Karhunen-Love functions," J.Opt.Soc.Am.A13, 1218-1225, 1996). The acquisition of the average slope depends on the calculation of the centroid of the sub-aperture image, and each sub-aperture can usually only obtain two slopes in the x and y directions. This is true when the observation target of the Hartmann wavefront sensor is a point source, that is, when the incident light wavefront distortion completely determines the focal spot deformation of each sub-aperture image on the Hartmann focal plane camera. However, when the observation target is an extended target, the focal spot deformation of the sub-aperture image is caused by the combined effect of the extended target shape and the wavefront aberration. At this time, the traditional wavefront restoration method has a large error in the wavefront measurement, and the measurement is inaccurate.

[0003] Furthermore, the centroid calculation method is valid when the wavefront aberrations of the incident light from the subaperture only include tilt, without defocus or higher-order aberrations. However, when the spatial frequency of the wavefront distortion of the incident light from the Hartmann wavefront sensor is significantly higher than the sampling rate of the Hartmann subaperture, traditional wavefront restoration methods will suffer from significant pattern aliasing errors. This is because the Hartmann wavefront sensor reconstructs changes in wavefront slope caused by higher-order aberrations as lower-order pattern aberrations during pattern reconstruction.

[0004] The root of this problem lies in the fact that, in wavefront measurement of extended targets, the average slope calculated in traditional wavefront restoration methods is a dual mapping relationship between the extended target shape and the wavefront aberrations. Therefore, it is impossible to directly calculate the average slope to invert the wavefront aberrations. If a feature affected only by wavefront distortion could be extracted from the subaperture image, then the measurement accuracy of the wavefront aberrations would be significantly improved by calculating this feature.

[0005] Furthermore, the root cause of pattern aliasing errors lies in the use of a slope vector composed of the average slopes of each subaperture to represent the pattern. The dimensionality of this slope vector can only be twice the number of effective subapertures in the Hartmann wavefront sensor. Furthermore, because the slope vectors of certain order aberrations are too close, significant mode coupling can occur under the influence of detection noise. If wavefront aberrations could be represented with features from more dimensions, the discrimination between different modes would be significantly improved, and inter-mode aliasing and coupling could be better suppressed, thereby improving the order of pattern recovery and the phase recovery accuracy of the Hartmann wavefront sensor.

[0006] Invention patent 201910312442.2 proposed using deep learning to directly establish a mapping relationship between pattern coefficients and images captured by a Hartmann wavefront sensor, thereby improving the pattern recovery order and phase recovery accuracy of the Hartmann wavefront sensor. However, this method is limited to point source wavefront measurement scenarios and suffers from large errors in extended target detection scenarios.

[0007] Invention Patent 202010113444.1 uses the correlation between the subaperture map of the detection area and the reference subaperture map in the Hartmann wavefront sensor to calculate the tilt and reconstruct the wavefront. This solution can be used in the field of extended target wavefront detection. However, this method has limited spatial resolution in extended target wavefront detection scenarios. Summary of the Invention

[0008] The technical problem to be solved by the present invention is: how to extract effective features independent of the shape of the extended target, so as to achieve high-precision and high-spatial-resolution wavefront restoration in the extended target scene.

[0009] The technical solution adopted by the present invention to solve the technical problem is: a super-resolution wavefront restoration method for extended targets based on a Shack-Hartmann wavefront sensor, which calculates and extracts a feature image from the extended target sub-aperture image collected by a Hartmann focal plane camera. A trained artificial neural network is then used to directly calculate the feature image to restore the mode coefficients of the incident light wavefront. The specific steps are as follows:

[0010] Step (1): Calculate the random pattern coefficient matrix :Use computer to randomly generate pattern coefficient matrix ;in The number of rows in the matrix is , indicating that the mode order is Step, The number of columns of the matrix is , indicating co-generation Group mode coefficient; mode function is expressed as ,in k is the mode order, x0, y0 are the two-dimensional coordinates in the domain;

[0011] Step (2): Calculate the phase function sequence : According to the mode function and the mode coefficient matrix The j-th column vector of Generate a sequence of phase functions: ;

[0012] Step (3): Calculate the entrance pupil complex amplitude sequence :The phase function Substitute the calculation formula of the entrance pupil complex amplitude of the Hartmann wavefront sensor one by one get ,in is an amplitude function, which can be flexibly set according to the actual application scenario of the Hartmann wavefront sensor; where e is a natural constant, which is the base of the natural logarithm, and i is the imaginary unit;

[0013] Step (4): Create the first relational function :According to the wavelength of incident light of Hartmann wavefront sensor , entrance pupil function , microlens array transmittance function And the focal plane camera two-dimensional sampling function Create focal plane camera image The complex amplitude of the entrance pupil of the Hartmann sensor The first relationship function between ; Where m, n are the two-dimensional coordinates of the pixel point on the focal plane camera image, is the two-dimensional coordinate of the focal plane camera image before sampling, and U is the complex amplitude sequence;

[0014] Step (5): Calculate the focal hole diameter image of the point source :The complex amplitude sequence of the entrance pupil of L Hartmann wavefront sensors is Substitute into the first relation function respectively Calculate the corresponding point source focal hole diameter image ;

[0015] Step (6): Calculate the ideal focal hole diameter image of the point source : Ideal case phase function , substitute the phase function into the calculation formula of the complex amplitude of the entrance pupil of the Hartmann wavefront sensor Get the ideal entrance pupil complex amplitude , the ideal entrance pupil complex amplitude Substitute into the first relation function Calculate the ideal focal hole diameter image of a point source ;

[0016] Step (7): Create the second relational function :According to the ideal focal hole diameter image of point source Building feature images with Hartmann subaperture images The second relationship function between ;

[0017] Step (8): Calculate the feature image :The ideal focal hole diameter image of the point source , point source focal plane subaperture image Substitute the second relation function Calculate point source characteristic image And serve as the input sample set for subsequent artificial neural network training { }, record vector The collection of { } is the label sample set for artificial neural network training;

[0018] Step (9): Establish an artificial neural network whose input is a two-dimensional matrix of the feature image or a one-dimensional vector with the number of units equal to the total number of pixels in the feature image;

[0019] Step (10): Train the network: Use the input sample set { } and label sample set { } to train the artificial neural network established in step (9), and save the artificial neural network after training;

[0020] Step (11): When performing extended target wavefront detection in actual applications, first select a reference sub-aperture image from the far-field image captured by the focal plane camera based on the evaluation index C, and then use the second relationship function The calculated feature image is used as the input of the artificial neural network trained in step (10), and its output is used as the pattern coefficient of wavefront restoration.

[0021] Furthermore, the mode function in step (1) may be a Zernike mode function, A pattern function or any other type of two-dimensional function.

[0022] Furthermore, the amplitude function in step (3) It can follow uniform distribution, Gaussian distribution or other arbitrary distribution.

[0023] Furthermore, in step (4) The expression of is calculated using the Fraunhofer diffraction formula:

[0024] ,

[0025] ,

[0026] Where λ is the wavelength, f is the focal length of the microlens, and the transmittance function of the microlens array is It is expressed by the following formula:

[0027] ,

[0028] in, The function is the unit impulse function, is the entrance pupil function of the lens, which can be a square, rectangle, hexagon or other arbitrary shapes;

[0029] Among them, the sampling function of the far-field camera is Expressed as:

[0030] ,

[0031] ;

[0032] ;

[0033] Where p is the pixel size of the far-field camera, The far-field camera x f ,y f The number of pixels in the direction.

[0034] Furthermore, the evaluation index C in step (11) is contrast, which is specifically expressed as:

[0035]

[0036]

[0037] The image The total number of pixels is S;

[0038] Furthermore, in step (7) The expression is:

[0039]

[0040] in, The symbol represents the Fast Fourier Transform, represents the inverse fast Fourier transform, Represents the focal plane subaperture image of a point source The i-th sub-aperture corresponds to the regional image, N represents the number of Hartmann sub-apertures, and L represents the dataset { }Number of samples.

[0041] Furthermore, the artificial neural network in step (9) can be a perceptron, a multi-layer perceptron, a deep neural network, a convolutional neural network, a recurrent neural network, or any other type of neural network that satisfies the input and output dimensions.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] The present invention calculates a characteristic image independent of the shape of the extended target from a Hartmann wavefront sensor image of the extended target, and uses an artificial neural network to establish a relationship between the pattern coefficients of the incident light wavefront aberration and the image captured by the Hartmann wavefront sensor. The method provided by the present invention simplifies the multiple mappings between the extended target wavefront sensor image and the pattern coefficients into a single mapping relationship between the characteristic image and the pattern coefficients, thereby enabling the restoration of the incident light wavefront aberration from the extended target wavefront sensor image. The method provided by the present invention can process more characteristic information from the characteristic image, help better establish the mapping relationship between the characteristic image and the pattern coefficients, reduce pattern confusion and pattern coupling errors, and restore higher-order pattern coefficients with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of an extended target super-resolution wavefront restoration method based on a Shack-Hartmann wavefront sensor.

[0045] Figure 2 Schematic diagram of the sub-aperture arrangement of a 100-unit Hartmann wavefront sensor (the microlens entrance pupil function is a square, and the outer circle is the Hartmann entrance pupil function).

[0046] Figure 3 are the Hartmann aperture diagram with aberration and the reference image generated by simulation and the characteristic image calculated, where Figure 3 (a) is the simulated point source Hartmann aperture diagram, (b) is the reference image, and (c) is the point source feature image calculated using the Hartmann aperture diagram and the reference image.

[0047] Figure 4 This is a structural diagram of a specific implementation of an artificial neural network used in the present invention, namely a convolutional neural network.

[0048] Figure 5 are grayscale images of two extended targets and Hartmann aperture maps, reference images, and characteristic images simulated and generated for the two extended targets respectively; Figure 5(a) is the grayscale image of rice grain 1, (b) is the grayscale image of rice grain 2, (c) is the out-of-focus pore diameter image of rice grain 1, (d) is the out-of-focus pore diameter image of rice grain 2, (e) is the sub-aperture image with the largest contrast among the 100 sub-apertures in the out-of-focus pore diameter image of rice grain 1, (f) is the sub-aperture image with the largest contrast among the 100 sub-apertures in the out-of-focus pore diameter image of rice grain 2, and (g) is the sub-aperture image with the largest contrast among the 100 sub-apertures in the out-of-focus pore diameter image of rice grain 2. Figure 5 The characteristic images of rice grain 1 calculated in (c) and (e) are as follows. Figure 5 (h) in the formula is through Figure 5 (d) and (f) are the calculated characteristic images of rice grain 2.

[0049] Figure 6 The reconstructed wavefront phase and residual wavefront phase of the Hartmann far-field image of the same extended target under the same aberration are compared using the method provided by the present invention and the traditional correlation method. Figure 6 (a) is the true aberration phase diagram, (b) is the wavefront phase diagram estimated by the correlation method, (c) is the residual wavefront phase diagram of the wavefront phase estimated by the correlation method relative to the true aberration phase, (d) is the wavefront phase diagram estimated by the method provided by the present invention, and (e) is the residual wavefront phase diagram of the wavefront phase estimated by the method provided by the present invention relative to the true aberration phase.

[0050] Figure 7 The purpose is to compare the Hartmann far-field image wavefront reconstruction coefficients of the same extended target under the same aberration using the method provided by the present invention and three traditional correlation methods using restoration matrices of different orders. Figure 7 (a) is the comparison of the Zernike coefficients from the 3rd to the 50th order, (b) is the comparison of the Zernike coefficients from the 51st to the 100th order, (c) is the comparison of the Zernike coefficients from the 101st to the 150th order, (d) is the comparison of the Zernike coefficients from the 151st to the 200th order, (e) is the comparison of the Zernike coefficients from the 201st to the 250th order, and (f) is the comparison of the Zernike coefficients from the 251st to the 299th order. DETAILED DESCRIPTION

[0051] An embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0052] Figure 1 This is a principle flow chart of the extended target super-resolution wavefront restoration method based on the Shack-Hartmann wavefront sensor described in the present invention, which mainly includes the computer simulation process of generating a training data set and the actual measurement process. Figure 2 The sub-aperture arrangement of a 100-unit Hartmann wavefront sensor is shown in Figure 1. The microlens entrance pupil function is a square, and the outer circle is the Hartmann entrance pupil function. Figure 2 As shown, this embodiment uses a Hartmann wavefront sensor comprising 19 sub-apertures. Assume that the wavelength of the light source used in this embodiment is The entrance pupil of the Hartmann wavefront sensor is a circular function with a diameter of 1.8 mm. The microlens spacing a in the x-direction is 0.15 mm, and the spacing b in the y-direction is approximately 0.15 mm. The microlens is a regular polygon. The focal length f of the microlens is 1.66 cm. The pixel size p of the focal plane camera is 24 The size is 240x240 and the data depth is 12 bits. The following steps are used to achieve high-precision restoration of the incident light wavefront of the extended target.

[0053] Step 1: Use a computer to randomly generate a pattern coefficient matrix ;in Number of rows in the matrix Equal to 299, indicating that the mode order is 299 in total. Number of columns in the matrix is 100000, indicating that a total of 100000 sets of pattern coefficients are generated; the pattern function selects the Zernike pattern, which is expressed as , k = 1,2,…,299;

[0054] Step 2: According to the mode function and the mode coefficient matrix The j-th column vector of Generate a sequence of phase functions: ;

[0055] Step 3: Set the incident light intensity to be uniformly distributed , the phase function Substitute the calculation formula of the entrance pupil complex amplitude of the Hartmann wavefront sensor one by one:

[0056] Get the complex amplitude sequence ;

[0057] Step 4: According to the wavelength of incident light of Hartmann wavefront sensor , entrance pupil function , microlens array transmittance function And the focal plane camera two-dimensional sampling function Create focal plane camera image The complex amplitude of the entrance pupil of the Hartmann sensor The first relationship function between :

[0058]

[0059] Step 5: 100,000 Hartmann wavefront sensor entrance pupil complex amplitude sequences Substitute into the relational function respectively Calculate the corresponding focal plane camera image ;

[0060] Step 6: Calculate the ideal focal hole diameter image of the point source Ideally, there is no aberration, which can be regarded as a phase function. , substitute the phase function into the calculation formula of the complex amplitude of the entrance pupil of the Hartmann wavefront sensor Get the ideal entrance pupil complex amplitude , the ideal entrance pupil complex amplitude Substitute into the relational function Calculate the ideal focal hole diameter image of a point source ;

[0061] Step 7: Reference image based on sub-aperture , focal plane subaperture image Building feature images Image with Hartmann aperture The second relationship function between :

[0062]

[0063] Step 8: Place the Point Source Subaperture Reference Image like Figure 3 (a) Subaperture image of the point source focal plane (like Figure 3 Substitute (b) in the relational function Calculate point source characteristic image (like Figure 3 (c)) and serves as the input sample set for subsequent artificial neural network training { }, record vector { } is the label set for artificial neural network training;

[0064] Step 9: Create Figure 4 The artificial neural network shown in the figure consists of convolutional layers, pooling layers, and fully connected layers. The number of hidden units in each layer or the configuration of the convolution kernel are detailed in the figure. Its input is a two-dimensional image of dimension 240x240, and its output is a one-dimensional vector containing 299 elements. It has three hidden layers in the middle.

[0065] Step 10: Using the input sample set { } and label sample set { The artificial neural network established in step 9 is trained with the sample set composed of}, and the artificial neural network is saved after the training is completed;

[0066] Step 11: Use computer to randomly simulate and generate two kinds of rice grain images using Hartmann wavefront sensor images, 10,000 images each, where the order of pattern difference is 299. The evaluation index P is selected as contrast Where m=n=240, L=4; and 20,000 focal plane far field images Each image in (like Figure 5 The contrast values of the 100 sub-aperture images in (c) and (d) are calculated in sequence, and the sub-aperture image corresponding to the maximum contrast is selected. As the reference image of the focal plane image ,like Figure 5 (e) and (f) in . Then according to the relationship function Calculate the extended target feature image (like Figure 5 Finally, the artificial neural network trained in step 10 is used to calculate the corresponding output as the mode coefficient for wavefront restoration.

[0067] According to statistics, the average wavefront phase restoration error root mean square of 50 feature images tested under two different extended targets using the wavefront restoration method provided by the present invention is The average wavefront phase recovery errors tested by the correlation method are ,This method significantly improves the accuracy of extended target wavefront restoration.

[0068] A test image of a rice grain 1 is randomly selected, and the root mean square error of the wavefront phase restoration obtained by the wavefront restoration method provided by the present invention is , the root mean square error of the wavefront phase restoration obtained by the correlation method is The comparison of the restored wavefront obtained by this method and the correlation method is as follows: Figure 6 In (b) and (d), we can intuitively find that the wavefront image restored by this method is Figure 6 (d) is the wavefront image restored using the correlation method, that is, Figure 6 (b) has higher resolution.

[0069] In addition, for the Hartmann far-field image of the same extended target under the same aberration, the root mean square error of the 299th-order Zernike coefficients reconstructed by this method and five correlation methods using different restoration matrices is compared with the true aberration, as shown in Table 1. The reconstruction error of the correlation method first decreases and then increases with the order of the restoration matrix. When the reconstructed Zernike coefficient is 105, the 299th-order reconstruction error of the correlation method is as low as 56nm, while the 299th-order coefficient reconstruction error of this method is only 28nm.

[0070]

[0071] Table 1

[0072] The Zernike coefficients restored by this method and the three related methods using "20th order restoration matrix", "105th order restoration matrix" and "120th order restoration matrix" are compared with the Zernike coefficients of the real aberration. Figure 7 shown.

[0073] Parts of the present invention that are not described in detail belong to the well-known knowledge in the art.

Claims

1. An extended target super-resolution wavefront restoration method, applied to a Shack-Hartmann wavefront sensor, characterized by: The method extracts a characteristic image from the extended target subaperture image acquired by the Hartmann focal plane camera, and then directly calculates the characteristic image using a trained artificial neural network to restore the mode coefficients of the incident light wavefront. The method specifically includes the following steps: Step (1): Calculate the random pattern coefficient matrix :Use computer to randomly generate pattern coefficient matrix ;in The number of rows in the matrix is , indicating that the mode order is Step, The number of columns of the matrix is , indicating co-generation Group mode coefficient; mode function is expressed as ,in is the mode order, x0, y0 are the two-dimensional coordinates in the domain; Step (2): Calculate the phase function sequence : According to the mode function and the mode coefficient matrix The j-th column vector of Generate a sequence of phase functions: ; Step (3): Calculate the entrance pupil complex amplitude sequence :The phase function Substitute the calculation formula of the entrance pupil complex amplitude of the Hartmann wavefront sensor one by one get ,in It is an amplitude function, which can be flexibly set according to the actual application scenario of the Hartmann wavefront sensor; Step (4): Create the first relational function :According to the wavelength of incident light of Hartmann wavefront sensor , entrance pupil function , microlens array transmittance function And the focal plane camera two-dimensional sampling function Create focal plane camera image The complex amplitude of the entrance pupil of the Hartmann sensor The first relationship function between ; Where m, n are the two-dimensional coordinates of the pixel point on the focal plane camera image, is the two-dimensional coordinate of the focal plane camera image before sampling, and U is the complex amplitude sequence; Step (5): Calculate the focal hole diameter image of the point source :The complex amplitude sequence of the entrance pupil of L Hartmann wavefront sensors is Substitute into the first relation function respectively Calculate the corresponding point source focal hole diameter image ; Step (6): Calculate the ideal focal hole diameter image of the point source : Ideal case phase function , substitute the phase function into the calculation formula of the complex amplitude of the entrance pupil of the Hartmann wavefront sensor Get the ideal entrance pupil complex amplitude , the ideal entrance pupil complex amplitude Substitute into the first relation function Calculate the ideal focal hole diameter image of a point source ; Step (7): Create the second relational function :According to the ideal focal hole diameter image of point source Building feature images with Hartmann subaperture images The second relationship function between ; Step (8): Calculate the feature image :The ideal focal hole diameter image of the point source , point source focal plane subaperture image Substitute the second relation function Calculate point source characteristic image And serve as the input sample set for subsequent artificial neural network training { }, record vector The collection of { } is the label sample set for artificial neural network training; Step (9): Establish an artificial neural network whose input is a two-dimensional matrix of the feature image or a one-dimensional vector with the number of units equal to the total number of pixels in the feature image; Step (10): Train the network: Use the input sample set { } and label sample set { The artificial neural network established in step (9) is trained with the sample set composed of}, and the artificial neural network is saved after the training is completed; Step (11): When performing extended target wavefront detection in actual applications, first select a reference sub-aperture image from the far-field image captured by the focal plane camera based on the evaluation index C, and then use the second relationship function The calculated feature image is used as the input of the artificial neural network trained in step (10), and its output is used as the pattern coefficient of wavefront restoration.

2. The extended target super-resolution wavefront restoration method according to claim 1, characterized in that: The mode function in step (1) is the Zernike mode function, A pattern function or any other type of two-dimensional function.

3. The extended target super-resolution wavefront restoration method according to claim 1, characterized in that: The amplitude function in step (3) Obey uniform distribution, Gaussian distribution or other arbitrary distribution.

4. The extended target super-resolution wavefront restoration method according to claim 1, characterized in that: In step (4) The expression of is calculated using the Fraunhofer diffraction formula: ， ， Where λ is the wavelength, f is the focal length of the microlens, and the transmittance function of the microlens array is It is expressed by the following formula: ， in, The function is the unit impulse function, is the entrance pupil function of the lens, which is a square, rectangle, hexagon or other arbitrary shape; Among them, the sampling function of the far-field camera is Expressed as: ， ； ； Where p is the pixel size of the far-field camera, The far-field camera The number of pixels in the direction.

5. The extended target super-resolution wavefront restoration method according to claim 1, characterized in that: The evaluation index C in step (11) is contrast, which is specifically expressed as: The image The total number of pixels is S.

6. The extended target super-resolution wavefront restoration method according to claim 1, characterized in that: In step (7) The expression is: in, The symbol represents the fast Fourier transform, represents the inverse fast Fourier transform, Represents the focal plane subaperture image of a point source The i-th sub-aperture corresponds to the regional image, N represents the number of Hartmann sub-apertures, and L represents the dataset { }Number of samples.

7. The extended target super-resolution wavefront restoration method according to claim 1, characterized in that: The artificial neural network in step (9) is a perceptron, a multi-layer perceptron, a deep neural network, a convolutional neural network, a recurrent neural network, or any other type of neural network that satisfies the input and output dimensions.

Images