Wavefront image processing method based on dark light source
By using the lightweight neural network SPWFS-Net to process wavefront images under dark and weak light sources, the problem of degradation of detection accuracy in dark and weak light sources is solved in the prior art, real-time automatic adjustment and efficient wavefront detection are achieved.
Patent Information
- Application Number
- CN202510210506.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-25
- Publication Date
- 2025-06-24
AI Technical Summary
The existing wavefront image processing methods reduce detection accuracy under dark and weak light sources, making it difficult to adjust automatically in real time, resulting in limited correction capabilities of the adaptive optical system.
The lightweight neural network SPWFS-Net is used to simulate the wavefront sensor image under dark and weak light sources, and train the network to predict the missing wavefront slope signal to achieve real-time automatic adjustment.
The accuracy and speed of wavefront detection are significantly improved under dark and weak light sources, and can adapt to the observation environment in real time and make automatic adjustments, avoiding the failure of traditional methods under dark and weak targets and natural guided stars.
Smart Images
Figure CN120198769A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a wavefront image processing method based on a faint light source, which can improve the detection accuracy and speed of a wavefront sensor. Background Art
[0002] With the in-depth exploration of the universe, direct imaging of targets such as exoplanets outside the solar system is an important goal in the field of astronomy. However, it is very difficult to observe high-value targets such as exoplanets at night, and the light signals emitted by most observation targets are relatively faint. Moreover, when the light signal passes through the Earth's atmosphere, it is affected by adverse factors such as air flow disturbance and temperature change in the Earth's atmosphere, and will be distorted, resulting in the observed target being blurred (as shown in (a) of Figure 1 ), and the original physical characteristics cannot be analyzed. Therefore, an adaptive optical system came into being to correct the distortion of the light signal and enable the telescope to obtain imaging with diffraction-limited capabilities.
[0003] The basic composition of the adaptive optical system is as shown in (b) of Figure 1 , and it consists of a deformable mirror, a wavefront sensor, a controller, and an optical imaging component. The wavefront sensor detects the wavefront slope, thereby calculating the distortion information of the wavefront, and then sends the distortion information to the controller. The controller calculates the control signal according to the distortion information and sends it to the deformable mirror actuator to compensate for the wavefront distortion caused by factors such as atmospheric turbulence by changing the mirror surface shape of the deformable mirror. In order to improve the correction ability of the entire adaptive system, the detection accuracy of the wavefront sensor is crucial. However, the detection accuracy of the wavefront sensor will drop significantly when the signal intensity of the observation target is insufficient. At the same time, the observation conditions in the actual observation environment, such as the temperature, humidity, and background noise of the observation point, will also reduce the detection accuracy. Although some neural networks can enhance the accuracy by predicting the centroid of some lost spot signals through the wavefront image, their network structure is complex and the calculation speed is insufficient, making it difficult to keep up with the operating speed of the system. Currently, most wavefront sensors use the threshold method to preprocess the wavefront image obtained by the wavefront sensor, that is, use a pre-calibrated unified threshold or regional threshold to perform background subtraction on the wavefront image to reduce the influence of background noise. Although such methods are simple, when the target signal intensity is insufficient, it is extremely easy to lose signals and even cause wavefront detection failure.
[0004] In summary, the existing wavefront image processing methods mainly have the following problems:
[0005] (1) The existing threshold method will cause the loss of images in some areas when the light source signal is insufficient.
[0006] When the signal intensity of the observed target or natural guide star is insufficient in the traditional centroid wavefront detection method, due to the lack of spot signals in a large number of sub-apertures, insufficient wavefront slope information cannot be calculated, which will lead to the failure of wavefront detection, and thus the entire adaptive optical system will lose the correction ability.
[0007] In the traditional threshold method and gray centroid method, the calculated wavefront slopes are pre-calibrated values. Once the light intensity of the on-site observed target and the observation environment change, it is difficult to adjust automatically in real time.
[0008] The existing wavefront neural network method is mainly used to predict the centroid of the missing spot signal from the wavefront image and then calculate the wavefront slope. The network structure is too complex and the calculation amount is too large. It will not only slow down the overall operation speed of the system, but also has high requirements for the performance of the control system. High-cost custom hardware such as GPUs and DSPs is required, and it is difficult to directly integrate the algorithm into the existing hardware system. Summary of the Invention
[0009] In view of the above problems existing in the prior art, the present invention provides a wavefront image processing method based on a dim light source.
[0010] To achieve the above object, the present invention provides the following technical solutions:
[0011] A wavefront image processing method based on a dim light source includes the following steps:
[0012] Step 1: Construct a lightweight neural network SPWFS-Net;
[0013] Step 2: Simulate the wavefront sensor image under a dim light source;
[0014] Step 3: Partially cover the image used as the training set to simulate the signal loss under a dim light source;
[0015] Step 4: Calculate the incomplete slope matrix of the image used as the training set;
[0016] Step 5: Use the incomplete slope matrix as the input of the neural network SPWFS-Net for training, and end the training when the loss function value is less than the preset minimum value to obtain a complete slope matrix;
[0017] Step 6: Connect the trained neural network SPWFS-Net to the actual control loop, receive the actual image, generate a complete wavefront slope matrix, and thus automatically predict the missing wavefront slope signal in the actual image in real time.
[0018] Further, in step 1, the neural network SPWFS-Net includes the following seven stages:
[0019] The first stage: After the convolutional layer, a non-linear processing layer is connected. After repeating the convolutional layer and the non-linear processing layer, it enters the second stage;
[0020] The second stage: After the max pooling layer, a convolutional layer and a non-linear processing layer are connected. After repeating the convolutional layer and the non-linear processing layer, it enters the third stage;
[0021] The third stage: After the max pooling layer, a convolutional layer and a non-linear processing layer are connected. After repeating the convolutional layer and the non-linear processing layer, it enters the fourth stage;
[0022] The fourth stage: After the max pooling layer, a convolutional layer and a non-linear processing layer are connected. After repeating the convolutional layer and the non-linear processing layer, an upsampling layer is connected. After the upsampling layer, a non-linear processing layer is connected, and then it enters the fifth stage;
[0023] The fifth stage: After the convolutional layer, a non-linear processing layer is connected. After repeating the convolutional layer and the non-linear processing layer, an upsampling layer is connected. After the upsampling layer, a non-linear processing layer is connected, and then it enters the sixth stage;
[0024] The sixth stage: After the convolutional layer, a non-linear processing layer is connected. After repeating the convolutional layer and the non-linear processing layer, an upsampling layer is connected. After the upsampling layer, a non-linear processing layer is connected, and then it enters the seventh stage;
[0025] The seventh stage: After the convolutional layer, a non-linear processing layer is connected. After repeating the convolutional layer and the non-linear processing layer, a non-linear processing layer is connected, and finally it enters the convolutional regression layer.
[0026] Furthermore, the convolution kernel of the convolutional layer is 6×6, and the stride is 1; the ReLu function is used in the non-linear processing layer; 2×2 max pooling is adopted in the max pooling layer; the convolution kernel size of the convolutional regression layer is 1×1.
[0027] Furthermore, between each stage, skip channels are used to directly connect the bottom stage and the high stage, and the information of the bottom stage is directly transmitted to the high stage.
[0028] Furthermore, in step 2, the calculation formula for simulating the wavefront sensor image under a dim light source is as follows:
[0029] where f represents the focal length of the wavefront sensor lens, λ represents the wavelength of the wavefront sensor lens, T(x0, y0) represents the transmittance of the wavefront sensor lens, S(m, n, x f , y f ) represents the sampling method of the wavefront sensor, U i (x0, y0) represents the light intensity signal of the simulated input pupil, P(x0, y0) represents the simulated input pupil, (x0, y0) represents the entrance pupil coordinates of each point of the pupil, (x f, y f ) represents the pupil coordinates, (m, n) represent the horizontal and vertical coordinates of each point on the wavefront image, and I i (m, n) represents the pixel value of each point on the wavefront image.
[0030] Furthermore, random background noise is added during the generation process of the wavefront sensor image under a weak light source.
[0031] Furthermore, in step 4, the following formula is used to calculate the wavefront slopes in the X-axis and Y-axis directions within each sub-aperture on the wavefront sensor image, forming an incomplete slope matrix as the label for image training;
[0032]
[0033]
[0034] where f represents the focal length of the wavefront sensor lens, and (x ir , y ir ) represents the centroid coordinates of the reference spot, and (x i , y i ) represents the centroid coordinates of the actually calculated spot signal.
[0035] Furthermore, in step 5, during training, the following formula is used as the loss function:
[0036]
[0037] where (P ix , P iy ) represents the predicted slope information output by the neural network SPWFS-Net, (R ix , R iy ) represents the actually calculated slope information, and s represents the number of sub-apertures.
[0038] Furthermore, during network training, the calculation process of the spot centroid is not included.
[0039] Furthermore, the trained neural network SPWFS-Net is connected to the control loop of the adaptive optics system, receives the image sent by the wavefront sensor, generates a complete wavefront slope matrix, and sends the information of the complete wavefront slope matrix to the controller of the adaptive optics system to achieve real-time adaptive control of the controlled object.
[0040] Compared with the prior art, the beneficial effects of the present invention are:
[0041] (1) The wavefront image processing method based on dim light sources of the present invention adopts a neural network SPWFS-Net with a simple structure, which can run directly on a commercial computer without the need for professional devices such as GPUs and DSPs.
[0042] (2) The wavefront image processing method based on dim light sources of the present invention can predict complete wavefront slope information in real time under dim light source conditions (dim observation targets and dim natural guide stars) and different observation environments, and can achieve real-time automatic adjustment. Compared with the traditional threshold method and centroid prediction method, it can greatly improve the accuracy and speed of wavefront detection.
[0043] (3) It can be simply incorporated into the existing adaptive optical system. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a schematic diagram of the existing wavefront image processing method. Among them: (a) is the schematic diagram of the principle of imaging blur caused by the atmosphere; (b) is the schematic diagram of the composition of the adaptive optical system.
[0045] Figure 2 It is a schematic diagram of the SPWFS-Net structure.
[0046] Figure 3 It is the generation, training and testing of the SPWFS-Net dataset.
[0047] Figure 4 It is the overall control flow of the system.
[0048] Figure 5 It is the effect before and after the system runs. DETAILED DESCRIPTION OF THE INVENTION
[0049] The present invention will be further described in detail below with reference to the drawings.
[0050] The wavefront image processing method based on dim light sources of the present invention can predict complete wavefront slope information based on dim light source signals and incomplete Shack-Hartmann wavefront sensor images, improving the detection accuracy and speed of the wavefront. The method includes the following steps:
[0051] Step 1: Construct a lightweight neural network SPWFS-Net;
[0052] Step 2: Simulate the wavefront sensor image under dim light sources;
[0053] Step 3: Partially cover the images used as the training set to simulate signal loss under dim light sources;
[0054] Step 4: Calculate the incomplete slope matrix of the images used as the training set;
[0055] Step 5: Use the incomplete slope matrix as the input of the neural network SPWFS-Net for training. End the training when the value of the loss function is less than the preset minimum value to obtain a complete slope matrix;
[0056] Step 6: Connect the trained neural network SPWFS-Net to the actual control loop, receive the actual image, generate a complete wavefront slope matrix, and thus automatically predict the missing wavefront slope signals in the actual image in real time.
[0057] Figure 2 It is a schematic diagram of the specific structure of the neural network SPWFS-Net in this embodiment. This neural network has 38 layers and is divided into 7 stages, including:
[0058] The first stage: First, pass through a convolutional layer with a convolutional kernel of 6×6 and a stride of 1, followed by a ReLu non-linear activation unit, and then repeat the above convolution and non-linear processing to enter the next stage;
[0059] The second stage: First, perform a 2×2 max pooling operation, and then perform convolution and non-linear processing twice. The convolutional kernel is 6×6 and the stride is 1. The non-linear processing uses the ReLu function, and then enter the third stage.
[0060] The third stage: First, perform a 2×2 max pooling operation, and then perform convolution and non-linear processing twice. The convolutional kernel is 6×6 and the stride is 1. The non-linear processing uses the ReLu function, and then enter the fourth stage. (The network structures of the second and third stages are the same)
[0061] The fourth stage: First, perform a 2×2 max pooling operation, and then perform convolution and non-linear processing twice. The convolutional kernel is 6×6 and the stride is 1. The non-linear processing uses the ReLu function, and then perform an upsampling operation. After the upsampling is completed, perform non-linear processing once using the ReLu function, and then enter the fifth stage.
[0062] The fifth stage: First, perform convolution and non-linear processing twice. The convolutional kernel is 6×6 and the stride is 1. The non-linear processing uses the ReLu function, and then perform upsampling, and then perform non-linear processing once using the ReLu function, and then enter the sixth stage.
[0063] The sixth stage: First, perform convolution and non-linear processing twice. The convolutional kernel is 6×6 and the stride is 1. The non-linear processing uses the ReLu function, and then perform upsampling, and then perform non-linear processing once using the ReLu function, and then enter the sixth stage. (The network structures of the fifth and sixth stages are the same)
[0064] The seventh stage: After two convolutions and non-linear processing, the convolution kernel is 6×6, the stride is 1, and the ReLu function is used for non-linear processing. Finally, it enters the output convolution layer, and the convolution kernel size is 1×1.
[0065] Between each stage, skip channels are used to directly connect the bottom stage and the high stage, directly transmitting the information of the bottom stage to the high stage, avoiding the loss of some detailed information and at the same time improving a certain gradient propagation, reducing the training difficulty of this network.
[0066] During the training stage of SPWFS-Net, 10,000 simulated wavefront sensor (WFS) images are used as the dataset, 7,000 as the training set. Part of the wavefront images in the training set are covered with white squares to simulate the signal loss under faint light sources, 2,000 as the test set, and 1,000 as the validation set. The generation of training images, the training and testing processes are as Figure 3 shown.
[0067] During the process of simulating WFS images, various situations encountered in the actual observation process are considered. First, according to the inherent parameters of the wavefront sensor in the system: input pupil, wavelength, transmittance, sampling equation, data such as the size and resolution of the wavefront image are determined. The calculation formula for the random phase is:
[0068]
[0069] where C i is the vector coefficient, D k (x0, y0) is the state equation of the Zernike polynomial, and k represents the kth order. From this, the light intensity signal of the simulated input pupil can be obtained:
[0070]
[0071] where A(x0, y0) is the ideal strongest light signal that follows a Gaussian distribution or any on-site conditions
[0072] According to the Fourier optics principle, the calculation formula for the finally simulated wavefront image is as follows:
[0073] where f is the focal length of the micro-lens array of the wavefront sensor, λ is the wavelength of the micro-lens array, T(x0, y0) is the transmittance of the lens, and S (m, n, x f , y f ) represents the sampling method of the wavefront sensor. To simulate the actual observation environment, random background noise is also added during the generation.
[0074] Subsequently, the wavefront slopes in the X-axis and Y-axis directions within each sub-aperture on the WFS image are calculated using the following formula as the labels for image training.
[0075]
[0076]
[0077] f is the focal length of the microlens array of the wavefront sensor, (x ir , y ir ) is the centroid coordinate of the reference spot, and (x i , y i ) is the centroid coordinate of the actually calculated spot signal. In training, the following formula is used as the loss function:
[0078]
[0079] where (P ix , P iy ) represents the output predicted slope information of the SPWFS-Net, and (R ix , R iy ) is the actually calculated slope information. When the loss function reaches a minimum value of about 0.015, the optimal network training result is achieved.
[0080] After the training is completed, the SPWFS-Net can be incorporated into the control loop of the adaptive optics system, that is, it receives the image sent by the wavefront sensor, automatically predicts the wavefront slope in real time, generates a complete wavefront slope matrix, and then sends it to the controller to calculate the control command. Compared with the existing neural network algorithms and threshold methods, the method of the present invention only calculates the centroid during the training stage for the network to learn. After arriving at the observation site and starting to run, it can directly predict the missing slope information, no longer requiring on-site calibration calculations, and is more suitable for wavefront sensors with a large number of sub-apertures equipped on large-aperture telescopes when observing faint targets. Most of the existing neural networks for wavefront sensors are used to calculate the missing centroid information. Although they can make up for part of the signal loss, they have too high requirements for hardware equipment and too long running time.
[0081] The operation control process of the entire system is as Figure 4 shown. Using the image received by the wavefront sensor, first calculate an incomplete wavefront slope matrix as input data and send it to the SPWFS-Net network proposed by the present invention to predict a complete wavefront slope matrix in real time, and then send it to a commercial computer (controller) to calculate the control command, and then send it to the deformable mirror (DM). The operation control result of the system is as Figure 5 shown. It can be clearly seen from the figure that after the system runs, it can effectively concentrate the energy and achieve direct imaging of night targets.
[0082] In summary, in order to maintain the detection accuracy of the wavefront sensor under the condition of faint observation targets (faint natural guide stars), the present invention proposes a wavefront image processing method based on faint light sources. This method uses a lightweight neural network algorithm (SPWFS-Net) to automatically predict in real time the missing wavefront slope signals in the wavefront sensor images. The method of the present invention directly skips the calculation stage of the spot centroid, improves the detection accuracy and speed of the wavefront sensor, can adapt to the observation environment in real time, and automatically makes adjustments. The method of the present invention solves the defects that the traditional threshold method and centroid method cannot observe faint targets and cannot use faint natural guide stars. At the same time, the SPWFS-Net network structure is simple and can run smoothly even on a commercial PC, and can be incorporated into the existing adaptive optical system at low cost.
[0083] The foregoing are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A wavefront image processing method based on a dim light source, characterized in that: The steps include: Step 1: Build a lightweight neural network SPWFS-Net; Step 2: Simulate the wavefront sensor image under a dim light source; Step 3: Partially cover the images used as training sets to simulate signal loss under dim light sources; Step 4: Calculate the incomplete slope matrix of the images used as training set; Step 5: Use the incomplete slope matrix as the input of the neural network SPWFS-Net for training, and terminate the training when the loss function value is less than the preset minimum value to obtain a complete slope matrix; Step 6: Connect the trained neural network SPWFS-Net to the actual control loop, receive the actual image, and generate a complete wavefront slope matrix, so as to automatically predict the missing wavefront slope signal in the actual image in real time.
2. The wavefront image processing method based on a dim light source according to claim 1, characterized in that: In step 1, the neural network SPWFS-Net includes the following seven stages: The first stage: the convolutional layer is connected to the nonlinear processing layer, and the convolutional layer and the nonlinear processing layer are repeated before entering the second stage; The second stage: After the maximum pooling layer, the convolution layer and the nonlinear processing layer are connected, and the convolution layer and the nonlinear processing layer are repeated before entering the third stage; The third stage: After the maximum pooling layer, the convolution layer and the nonlinear processing layer are connected, and the convolution layer and the nonlinear processing layer are repeated before entering the fourth stage; The fourth stage: the convolution layer and the nonlinear processing layer are connected after the maximum pooling layer, the upsampling layer is connected after the repeated convolution layer and the nonlinear processing layer, and the nonlinear processing layer is connected after the upsampling layer, and then the fifth stage is entered; The fifth stage: the convolution layer is connected to the nonlinear processing layer, the repeated convolution layer and the nonlinear processing layer are connected to the upsampling layer, the upsampling layer is connected to the nonlinear processing layer, and then enters the sixth stage; Stage 6: The convolutional layer is connected to a nonlinear processing layer, the convolutional layer and the nonlinear processing layer are repeated, and then the upsampling layer is connected. The upsampling layer is connected to a nonlinear processing layer, and then enters the seventh stage; The seventh stage: the convolution layer is connected to the nonlinear processing layer, the convolution layer and the nonlinear processing layer are repeated, and then the nonlinear processing layer is connected, and finally the convolution regression layer is entered.
3. The wavefront image processing method based on a dim light source according to claim 2, characterized in that: The convolution kernel of the convolution layer is 6×6, and the step size is 1; the nonlinear processing layer uses the ReLu function; the maximum pooling layer uses 2×2 maximum pooling processing; the convolution kernel size of the convolution regression layer is 1×1.
4. The wavefront image processing method based on a dim light source according to claim 2, characterized in that: Between each stage, a jump channel is used to directly connect the bottom stage with the higher stage, and transmit the information of the bottom stage directly to the higher stage.
5. The wavefront image processing method based on a dim light source according to claim 1, characterized in that: In step 2, the calculation formula for simulating the wavefront sensor image under a dim light source is as follows: ； Where f is the focal length of the wavefront sensor lens, λ is the wavelength of the wavefront sensor lens, T(x0, y0) is the transmittance of the wavefront sensor lens, and S(m, n, x f , y f ) represents the sampling mode of the wavefront sensor, U i (x0, y0) represents the light intensity signal of the simulated input pupil, P(x0, y0) represents the input pupil, (x0, y0) represents the entrance coordinates of each point of the pupil, (x f , y f ) represents the vertical coordinate, (m, n) represents the horizontal and vertical coordinates of each point on the wavefront image, and Ii(m, n) represents the pixel value of each point on the wavefront image.
6. The wavefront image processing method based on a dim light source according to claim 5, characterized in that: Random background noise is added during the generation of the wavefront sensor image under dim light sources.
7. The wavefront image processing method based on a dim light source according to claim 1, characterized in that: In step 4, the wavefront slopes in the X-axis and Y-axis directions in each sub-aperture on the wavefront sensor image are calculated using the following formula to form an incomplete slope matrix as a label for image training; ； ； Where f is the focal length of the wavefront sensor lens, (x ir , y ir ) represents the coordinates of the reference spot centroid, (x i , y i ) represents the centroid coordinates of the spot signal actually calculated.
8. The wavefront image processing method based on a dim light source according to claim 1, characterized in that: In step 5, during training, the following formula is used as the loss function: ； Among them, (P ix , P iy ) represents the predicted slope information output by the neural network SPWFS-Net, (R ix , R iy ) represents the actually calculated slope information, and s represents the number of sub-apertures.
9. The wavefront image processing method based on a dim light source according to claim 1, characterized in that: The calculation process of the spot centroid is not included during network training.
10. The wavefront image processing method based on a dim light source according to claim 1, characterized in that: The trained neural network SPWFS-Net is connected to the control loop of the adaptive optical system to receive images from the wavefront sensor, generate a complete wavefront slope matrix, and send the complete wavefront slope matrix information to the controller of the adaptive optical system to achieve real-time adaptive control of the controlled object.