A Deep Learning-Based Method and System for Quality Restoration of Solar Extreme Ultraviolet Images
By constructing paired datasets and utilizing conditional generative adversarial networks based on deep learning, the problem of low quality in solar extreme ultraviolet (EUV) images was solved, achieving high-fidelity, automated image quality restoration applicable to various solar EUV imagers.
Patent Information
- Application Number
- CN202610084902.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-22
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-01-22
AI Technical Summary
The solar extreme ultraviolet imager suffers from poor image quality due to hardware defects, the influence of the space environment, and optical design problems, which affects the availability and accuracy of the data.
By acquiring high-quality reference images and constructing paired datasets, we utilize conditional generative adversarial networks (GANs) from deep learning to learn image degradation restoration relationships. By combining pixel consistency, temporal self-supervision, and generative adversarial loss functions to optimize the network, we achieve adaptive restoration of image quality.
It effectively restores the original structure and texture of the sun's surface, improves image quality, and has automation, generalization and accuracy, making it suitable for a variety of solar extreme ultraviolet imagers.
Smart Images

Figure CN122134573B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of solar extreme ultraviolet image processing, specifically to a method and system for quality restoration of solar extreme ultraviolet images based on deep learning. Background Technology
[0002] Intense solar activities such as solar flares and coronal mass ejections are the main drivers of disturbances in the Sun-Earth space environment, which may have a cascading effect on the Earth's magnetosphere and ionosphere, thereby threatening the safe operation of critical infrastructure such as navigation, communication, and power. The Solar Extreme Ultraviolet Imager enables long-term, continuous, and high-resolution remote sensing observations of the solar disk and corona, providing valuable data support for monitoring solar activity and forecasting space weather.
[0003] However, the Solar Extreme Ultraviolet Imager suffers from relatively low image quality due to multiple factors, including hardware defects and the complex space environment. First, the optical sensors in the extreme ultraviolet band exhibit significant radiation response degradation, causing the image signal-to-noise ratio to decrease rapidly with increasing on-orbit time. Second, some internal contaminants (such as water vapor and adhesives) may evaporate due to thermal effects, contaminating the imaging system. Third, design flaws in the optical system also affect image quality; for example, focusing anomalies between the grazing-incident and normal-incident telescopes can lead to bright spots, bright bands, and localized stray light, while the telescope support grid can project grid-like artifacts onto the imager's image. These overlapping problems significantly compromise the quality of the Solar Extreme Ultraviolet images, severely impairing the usability and accuracy of the data. Summary of the Invention
[0004] The purpose of this invention is to provide a deep learning-based method for restoring the quality of solar extreme ultraviolet (EUV) images, in order to solve the complex quality problems existing in current solar EUV images and improve the usability and stability of solar EUV data.
[0005] The technical method adopted in this invention is: a deep learning-based method for restoring the quality of solar extreme ultraviolet images, comprising the following steps: Step 1: Obtain high-quality reference solar extreme ultraviolet images of the solar image to be repaired and other sensors within the same time period, and perform quality screening on the solar extreme ultraviolet reference images. Step 2: Based on the solar extreme ultraviolet image to be repaired and the solar extreme ultraviolet reference image after quality screening in Step 1, perform time matching, radiometric normalization and geometric registration to construct a solar extreme ultraviolet paired dataset. Step 3: Using the solar extreme ultraviolet pairing dataset constructed in Step 2, establish and train a conditional generative adversarial network to learn degradation repair relationships. Optimize the joint loss function of the network using the stochastic gradient descent algorithm, and adjust the weights of the network layer by layer using the backpropagation algorithm. Once the loss function stabilizes, the network training is complete. Step 4: After radiometric normalization, the test image to be repaired is input into the trained network to obtain preliminary repair results. Then, radiometric inverse normalization is performed to obtain the solar extreme ultraviolet results after quality repair.
[0006] Furthermore, the solar extreme ultraviolet image to be repaired and the reference solar extreme ultraviolet image obtained in step 1 are in the same time period, and the time span completely covers the imager's self-cleaning cycle, the Earth's revolution cycle and the Sun's rotation cycle. Quality screening of solar extreme ultraviolet reference images includes: designing a rejection function based on a Poisson distribution prior to clean the solar extreme ultraviolet reference images, and calculating the global mean of each reference image. and variance The image will only be retained if the following conditions are met: ; in It is a predefined tolerance coefficient.
[0007] Furthermore, the specific implementation method of step 2 is as follows: First, temporal matching is performed. Using the solar extreme ultraviolet (EUV) image to be repaired as a baseline, a reference EUV image with an absolute time close to it is searched and matched, with a time error tolerance of ±k seconds, resulting in a temporally matched paired image. Next, all images are normalized to their maximum and minimum values based on adaptive constraints of brightness distribution. To avoid the influence of outliers, the radiometric range of different sensors is unified to 0 to 1. Then, the reference EUV image is resampled to match the size of the EUV image to be repaired. A geometric registration algorithm based on SIFT feature detection is used to register the reference image pixel-by-pixel onto the image to be repaired by estimating the homography transformation matrix, thus completing the geometric matching and obtaining a spatiotemporally aligned paired dataset.
[0008] Furthermore, the maximum and minimum value normalization based on the adaptive constraint of brightness distribution includes: By statistically analyzing the distribution characteristics of image pixel radiance, and constructing a cumulative distribution function for effective pixels, low-brightness boundary values are adaptively determined based on the cumulative distribution function. High brightness boundary value Subsequently, using the determined low-brightness boundary value High brightness boundary value As the upper and lower limits of normalization, the image pixel radiance values are cropped and subjected to maximum and minimum value normalization, and the expression is as follows: ; in Indicates the image at the pixel position The radiance value, the result after normalization The radiation range is strictly limited to This suppresses interference from pixels with abnormally extreme brightness.
[0009] Furthermore, the geometric registration algorithm based on SIFT feature detection utilizes scale-invariant feature transformation and matching SIFT algorithm to extract feature points in the image, and calculates the feature descriptor of each feature point to characterize its local texture features. Subsequently, the feature descriptors of the solar extreme ultraviolet image to be restored and the reference solar extreme ultraviolet image are initially matched using the fast approximate nearest neighbor search library FLANN, and fuzzy or erroneous matching relationships are eliminated. After obtaining a sufficient number of reliable feature correspondence points, the random sampling consensus algorithm RANSAC is used to suppress abnormal matching points, and the homography transformation matrix H is estimated to realize the spatial mapping from the coordinate system of the reference image to the coordinate system of the image to be restored. The transformation relationship is as follows:
[0010] Among them, symbols This represents a similarity equivalence relation in the sense of homogeneous coordinates; and The homography transformation matrix represents the spatial position of pixels in the reference solar extreme ultraviolet image and the solar extreme ultraviolet image to be restored in a two-dimensional coordinate system, respectively. It is a 3×3 projection transformation matrix used to describe the two-dimensional spatial mapping relationship between the reference solar extreme ultraviolet image coordinate system and the solar extreme ultraviolet image coordinate system to be repaired.
[0011] Furthermore, the conditional generative adversarial network described in step 3 includes a generator G and a discriminator D. The generator G employs a deep network based on a residual structure and introduces a channel attention mechanism into the residual structure to adaptively weight different feature channels. The generator takes the solar extreme ultraviolet image to be repaired as input and outputs the corresponding repair result. The discriminator D adopts a PatchGAN structure, and its input is a combination of the solar extreme ultraviolet image to be repaired and the repair result, or a combination of the solar extreme ultraviolet image to be repaired and a reference solar extreme ultraviolet image. It is used to judge the consistency between the repair result and the real solar extreme ultraviolet image in terms of radiation distribution and structural details at the local block scale. The discriminator D only participates in the overall model optimization during the network training phase through generative adversarial training.
[0012] Furthermore, the generator is composed of a downsampling stage, several residual modules with embedded channel attention, and an upsampling stage connected in series. The residual modules with embedded channel attention are used to perform deep modeling of the features output by the downsampling stage. Each residual module includes a residual branch and a skip connection branch. Let the input features be... The residual branch consists of two convolutional layers. The first convolutional layer is followed by a batch normalization layer and a ReLU activation function, and the second convolutional layer is followed by a batch normalization layer to obtain the residual feature representation. ( To achieve adaptive feature enhancement along the channel dimension, the residual module embeds a channel attention mechanism at the output position of the residual branch, which uses residual features... ( Taking a function as input, a channel-dimensional feature description vector is obtained through global average pooling (GAP). One-dimensional convolution is then used to model the local correlations between channels. Subsequently, a channel attention weight vector is generated using a sigmoid activation function. Based on this weight vector, the residual features are... ( Channel-by-channel weighting is performed, and skip connections are made between the branch and the input features. Element-by-element addition is performed to obtain the output of the residual module.
[0013] Furthermore, the joint loss function described in step 3 is defined as follows: ; ; ; ; in, This represents a high-quality reference solar extreme ultraviolet image. This represents the repair result. This represents a solar extreme ultraviolet image awaiting restoration. Indicates pixel consistency loss. The temporal self-supervised loss is defined as the first... t Time-based repair results With the adjacent first t -1 and t+ Two solar extreme ultraviolet images to be restored at time 1 , The root mean square error between them For least-squares-based generative adversarial loss, For the total loss, , and These are the weight coefficients for pixel consistency loss, temporal self-supervised loss, and generative adversarial loss, respectively.
[0014] Furthermore, the specific implementation process of step 4 is as follows: After radiometric normalization of any image to be repaired, it is input into the generator of the trained conditional generative adversarial network. G In the process, the initial restoration results are output; then, radiometric inverse normalization is performed to adjust the radiometric values to the same range as the original image to be restored, and finally, the restoration results of the solar extreme ultraviolet image with the correct range and clear structure are obtained. The calculation process for radiation anti-normalization is as follows:
[0015] in This represents the repair result output by the generative adversarial network, and its value range is... ; This represents the restoration result after radiometric inverse normalization; L and U represent the low-brightness boundary value and high-brightness boundary value, respectively, determined adaptively based on the image brightness distribution during the radiometric normalization stage.
[0016] The present invention also provides a solar extreme ultraviolet image quality restoration system based on deep learning, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the solar extreme ultraviolet image quality restoration method based on deep learning as described in the above technical solution.
[0017] The present invention has the following technical advantages and beneficial effects: ①This invention proposes a method for constructing solar extreme ultraviolet image datasets, which provides a possibility for the application of subsequent deep learning methods in the field of solar extreme ultraviolet image quality restoration; ② A deep learning-based method for restoring solar extreme ultraviolet images was developed, which achieved adaptive blind restoration of complex quality problems without the need for explicit modeling of multiple degradations, effectively restoring the original structure and texture of the solar surface; ③ The joint loss function proposed in this invention includes pixel reconstruction loss, temporal self-supervised loss and generative adversarial loss, which can guide model training and optimization from multiple dimensions such as pixels, time and features, and achieve high-fidelity image quality restoration while maintaining the effective structure of the image itself. ④ This invention does not require complex parameter adjustments or manual intervention, and is not limited to processing images from a single solar extreme ultraviolet imager. It has strong automation, generalizability, and accuracy, and is easy to put into practical use. Attached Figure Description
[0018] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.
[0019] Figure 1 A flowchart of a deep learning-based solar extreme ultraviolet image quality restoration method provided in an embodiment of the present invention.
[0020] Figure 2 This is a schematic diagram of the solar extreme ultraviolet image to be repaired and the reference solar extreme ultraviolet image before processing, and after radiometric normalization and geometric registration, provided in an embodiment of the present invention.
[0021] Figure 3 The image shows a comparison of the solar extreme ultraviolet image quality before and after restoration, as provided in an embodiment of the present invention.
[0022] Figure 4 Error map for solar extreme ultraviolet image quality restoration provided in an embodiment of the present invention. Detailed Implementation
[0023] To facilitate understanding and implementation of the present invention by those skilled in the art, the technical solutions in the embodiments of the present application and the accompanying drawings are described in detail below. It should be noted that the described embodiments are only for illustration and explanation of the present invention and are not limited to the present invention.
[0024] To address the response degradation and multi-factor quality deterioration issues present in solar extreme ultraviolet imagers during on-orbit operation, this application provides a deep learning-based method for restoring the quality of solar extreme ultraviolet images. By acquiring high-quality extreme ultraviolet reference images with approximate source wavelengths, a paired dataset is constructed, and deep learning methods are used to learn complex mapping relationships to achieve quality restoration of solar extreme ultraviolet images, supporting subsequent downstream applications.
[0025] Please see Figure 1 , Figure 1 A flowchart of a deep learning-based solar extreme ultraviolet image quality restoration method provided for embodiments of the present invention includes the following steps: Step 1: Acquire high-quality solar extreme ultraviolet (EUV) images from the Fengyun-3 X-ray Extreme Ultraviolet Imager (FY-3E / X-EUVI) with a center wavelength of 195 Å and from the Solar Dynamics Observatory Atmospheric Imaging Module (SDO / AIA) with a center wavelength of 193 Å within the same time period, and discard low-quality solar EUV reference images. This process includes the following sub-steps: Step 1.1: Acquire a large number of FY-3E / X-EUVI 195Å solar extreme ultraviolet images and SDO / AIA 193Å solar extreme ultraviolet reference images that cover the imager's self-cleaning cycle, Earth's revolution cycle, and Sun's rotation cycle within the same time period, ensuring that the selected images fully reflect the representativeness and periodic changes in instrument status, observation geometry, and solar activity conditions.
[0026] Step 1.2: To ensure the accuracy of the SDO / AIA 193Å reference image, a low-quality image removal function based on a Poisson distribution prior was designed to calculate the global mean of each reference image. and variance The image will only be retained if the following conditions are met: (1) in This is a predefined tolerance coefficient. If the deviation between the variance and the mean exceeds this threshold, the image is considered a low-quality SDO / AIA 193Å reference image and is discarded to ensure the correctness of the deep learning relationships.
[0027] Step 2: Using the FY-3E / X-EUVI 195Å solar extreme ultraviolet image to be repaired from Step 1 and the quality-screened SDO / AIA 193Å solar extreme ultraviolet reference image, perform temporal matching, radiometric normalization, and geometric registration to construct a paired solar extreme ultraviolet dataset. The specific implementation includes the following sub-steps: Step 2.1: Since different sensors have different time resolutions, time matching is required first. Using the absolute time of the FY-3E / X-EUVI 195Å solar extreme ultraviolet image to be repaired as the reference, search and match the SDO / AIA 193Å reference image with a similar absolute time. The time error is allowed to be within ±2 seconds to obtain the time-matched paired image.
[0028] Step 2.2: First, select effective pixels from the image for statistical analysis (e.g., remove missing pixels or non-finite value pixels), and statistically analyze the distribution characteristics of the effective pixel radiance to construct its cumulative distribution function. Then, based on the cumulative distribution function and according to the proportion of abnormally low-brightness and abnormally high-brightness pixels in the image brightness distribution, adaptively determine the low-brightness boundary value for normalization. High brightness boundary value This minimizes the proportion of pixels located outside the brightness boundary in the image, thereby preserving the main radiative information as much as possible while suppressing the influence of abnormally extreme brightness pixels on the normalization result; in determining the boundary value and After that, with and As the upper and lower limits of normalization, the maximum and minimum values are used to normalize the radiance values of image pixels, and their definitions are as follows: (2) in Indicates the image at the pixel position The radiance value, the result after normalization The radiation range is strictly limited to This approach avoids interference from abnormal extreme values on the normalization results while eliminating the impact of differences in the radiance range of different sensors on subsequent deep learning model training and inference.
[0029] Step 2.3: Resample the SDO / AIA 193Å reference image to match the pixel size of the FY-3E / X-EUVI 195Å image to be repaired. Then, use a geometric registration algorithm based on SIFT feature detection to extract feature points from the image to be repaired and the reference image using the Scale Invariant Feature Transform and Matching (SIFT) algorithm, and calculate the feature descriptor for each feature point to characterize its local texture features. Use the Fast Approximate Nearest Neighbor Search (FLANN) library to perform preliminary matching of the feature descriptors of the image to be repaired and the reference image, and eliminate fuzzy or erroneous matches. After obtaining a sufficient number of reliable feature correspondence points, use the Random Sample Consensus (RANSAC) algorithm to estimate the homography transformation matrix to suppress the influence of abnormal matching points. The transformation relationship is as follows: (3) Among them, symbols This represents a similarity equivalence relation in the sense of homogeneous coordinates; and The homography transformation matrix represents the spatial position of pixels in the reference image and the image to be repaired in a two-dimensional coordinate system, respectively. A 3×3 projection transformation matrix is used to describe the two-dimensional spatial mapping relationship between the reference image coordinate system and the coordinate system of the image to be repaired. This homography transformation matrix... The SDO / AIA 193Å reference image can be registered pixel by pixel onto the FY-3E / X-EUV I195Å image to be restored, thereby completing geometric matching and obtaining a spatiotemporally aligned paired dataset.
[0030] Step 3: Using the solar extreme ultraviolet pairing dataset constructed in Step 2, establish and train a conditional generative adversarial network to learn degradation repair relationships. Optimize the joint loss function of the network using the stochastic gradient descent algorithm, and adjust the network weights layer by layer using the backpropagation algorithm. After the loss function stabilizes, complete the network training. The specific implementation includes the following sub-steps: Step 3.1: Construct a conditional generative adversarial network for image quality restoration. This conditional generative adversarial network consists of a generator G and a discriminator D.
[0031] The generator G employs a deep convolutional network based on residual structures. In this embodiment, the generator consists of a downsampling stage, several residual modules with embedded channel attention, and an upsampling stage connected in series. Specifically, the generator first performs feature mapping on the input image to be repaired using a 7×7 convolution with a stride of 1. Then, it uses two 3×3 convolutions with a stride of 2 to achieve progressive downsampling, thereby expanding the receptive field while extracting multi-scale feature representations. Next, six residual modules with embedded channel attention are connected in series for deep feature learning. Finally, progressive upsampling is achieved through two deconvolutions, and the repair result is output via a 7×7 convolution. The generator takes the solar extreme ultraviolet image to be repaired as input and outputs the corresponding repaired image.
[0032] The embedded channel attention residual module is used for deep modeling of the features output from the downsampling stage. Each residual module includes a residual branch and a skip connection branch. Let the input features be... The residual branch consists of two 3×3 convolutional layers. The first convolutional layer is followed by a batch normalization layer and a ReLU activation function, and the second convolutional layer is followed by a batch normalization layer to obtain the residual feature representation. ( To achieve adaptive feature enhancement along the channel dimension, the residual module embeds a channel attention mechanism at the output position of the residual branch, preferably using Efficient Channel Attention (ECA). ECA uses residual features... ( Taking as input, a channel-dimensional feature description vector is obtained through global average pooling (GAP), and then a one-dimensional convolution is used. Modeling the local correlation between channels, followed by a Sigmoid activation function Generate channel attention weight vector Based on the weight vector For residual characteristics ( Channel-by-channel weighting is performed, and skip connections are made between the branch and the input features. By performing element-wise summation, the output y of the residual module is obtained. The calculation process can be expressed as follows: (5) Where ⊙ represents a channel-wise multiplication operation, and the attention weight vector The generation process can be represented as: (6) Through the aforementioned residual module design with embedded channel attention, the generator can adaptively enhance key radiation feature channels while maintaining the consistency of overall brightness and structural information in solar extreme ultraviolet images, thereby improving the stability and fidelity of image quality restoration.
[0033] The discriminator D adopts a PatchGAN structure and participates in generative adversarial training in a conditional discriminative manner. Its input is a combination of data of the image to be repaired and the repair result, or a combination of data of the image to be repaired and the reference image. It is used to judge the consistency between the repair result and the real solar extreme ultraviolet image in terms of radiation distribution and structural details at the local block scale, thereby suppressing over-smoothing, structural distortion or artifacts that may occur during the repair process. The discriminator D only participates in the overall model optimization in a generative adversarial training manner during the network training stage.
[0034] Step 3.2: Based on the defined joint loss function, train the constructed conditional generative adversarial network using the solar extreme ultraviolet pairing dataset built in Step 2 until the joint loss function converges. The joint loss function is defined as follows: (7) (8) (9) (10) in, Represents a high-quality SDO / AIA 193Å reference image. This represents the repair result. This represents the FY-3E / X-EUVI 195Å image to be restored. Indicates pixel consistency loss. The temporal self-supervised loss is defined as the first... t Time-based repair results With the adjacent first t -1 and t+ Two solar extreme ultraviolet images to be restored at time 1 , The root mean square error between the two is used to eliminate the effects of random noise or transient degradation, allowing the network to maintain the original solar structure of the image to be repaired. For least-squares-based generative adversarial loss, The total loss is the weighted sum of the aforementioned sub-losses. , and These are the weighting coefficients for each sub-loss.
[0035] Step 4: After performing radiometric normalization on any FY-3E / X-EUVI 195Å image to be repaired, input it into the generator of the trained conditional generative adversarial network. G In the first step, preliminary restoration results are output. Then, radiometric inverse normalization is performed to adjust the radiometric values to the same range as the original FY-3E / X-EUVI 195Å image to be restored. The calculation process is as follows: (11) in This represents the FY-3E / X-EUVI 195Å restoration result output by the network, with a value range of [value range missing]. ; The result is the repair after radiation denormalization; L and These represent the low-brightness boundary value and the high-brightness boundary value, respectively, determined adaptively based on the image brightness distribution during the radiation normalization stage; ultimately, the FY-3E / X-EUVI 195Å solar extreme ultraviolet image restoration result with correct value range and clear structure is obtained.
[0036] Figure 2 This application provides an embodiment of key steps in creating a dataset using FY-3E / X-EUVI 195Å as the image to be repaired and SDO / AIA 193Å as the reference image. The implementation effect diagrams are shown below. Figure 2 As can be seen from the present invention, the maximum and minimum value normalization based on the adaptive constraint of brightness distribution can effectively eliminate outliers, unify and align the radiation brightness distribution. After processing, the geometric registration method proposed in this invention can register the SDO / AIA 193 Å reference image pixel by pixel to the FY-3E / X-EUVI 195 Å image to be repaired, ensuring the consistency of the spatial coordinate system. Figure 3 The comparison images of solar extreme ultraviolet image quality restoration before and after, using FY-3E / X-EUVI 195Å as an example, provided for embodiments of this application, are from... Figure 3 It is evident that the method proposed in this invention has significant advantages in restoring the overall quality of the solar structure and processing the minute details of the solar surface, and effectively addresses degradation and contamination issues such as mixed noise, grid-like artifacts, bright spots, and stray light.
[0037] Figure 4 The solar extreme ultraviolet image quality restoration error map provided in this application embodiment, using FY-3E / X-EUVI 195Å as an example, is used to verify the role of key components. Figure 4 It can be seen that the overall repair error of the quality repair model proposed in this invention is relatively small, especially the generator based on the channel attention mechanism and the temporal self-supervised loss, which play a crucial role in it; for example Figure 4As shown in the white box, in the absence of the channel attention component, the model's ability to repair high-value regions such as flare bursts is significantly reduced, while... Figure 4 As shown in the yellow box, temporal self-supervised loss plays a crucial role in correcting degradation such as motion blur and transient ghosting.
[0038] On the other hand, embodiments of the present invention also provide a solar extreme ultraviolet image quality restoration system based on deep learning, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the solar extreme ultraviolet image quality restoration method based on deep learning as described in the above technical solution.
[0039] The above provides a detailed description of a deep learning-based method for restoring the quality of solar extreme ultraviolet images provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, based on the ideas of this application, several improvements and modifications can be made without departing from the principles of this invention. These improvements and modifications should also be considered within the scope of protection of this invention. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for repairing a solar EUV image quality based on deep learning, characterized in that, Includes the following steps: Step 1: Obtain high-quality reference solar extreme ultraviolet images of the solar image to be repaired and other sensors within the same time period, and perform quality screening on the solar extreme ultraviolet reference images. Step 2: Based on the solar extreme ultraviolet image to be repaired and the solar extreme ultraviolet reference image after quality screening in Step 1, perform time matching, radiometric normalization and geometric registration to construct a solar extreme ultraviolet paired dataset. Step 3: Using the solar extreme ultraviolet pairing dataset constructed in Step 2, establish and train a conditional generative adversarial network to learn degradation repair relationships. Optimize the joint loss function of the network using the stochastic gradient descent algorithm, and adjust the weights of the network layer by layer using the backpropagation algorithm. Once the loss function stabilizes, the network training is complete. The conditional generative adversarial network described in step 3 includes a generator G and a discriminator D. The generator G employs a deep network based on a residual structure, and introduces a channel attention mechanism into the residual structure to adaptively weight different feature channels. The generator takes the solar extreme ultraviolet image to be repaired as input and outputs the corresponding repair result. The discriminator D adopts a PatchGAN structure, and its input is a combination of the solar extreme ultraviolet image to be repaired and the repair result, or a combination of the solar extreme ultraviolet image to be repaired and a reference solar extreme ultraviolet image. It is used to judge the consistency between the repair result and the real solar extreme ultraviolet image in terms of radiation distribution and structural details at the local block scale. The discriminator D only participates in the overall model optimization during the network training phase through generative adversarial training. The joint loss function described in step 3 is defined as follows: ; ; ; ; in, This represents a high-quality reference solar extreme ultraviolet image. This represents the repair result. This represents a solar extreme ultraviolet image awaiting restoration. Indicates pixel consistency loss. The temporal self-supervised loss is defined as the first... t Time-based repair results With the adjacent first t -1 and t+ Two solar extreme ultraviolet images to be restored at time 1 , The root mean square error between them For least-squares-based generative adversarial loss, For the total loss, , and These are the weight coefficients for pixel consistency loss, temporal self-supervised loss, and generative adversarial loss, respectively. Step 4: After radiometric normalization, the test image to be repaired is input into the trained network to obtain preliminary repair results. Then, radiometric inverse normalization is performed to obtain the solar extreme ultraviolet results after quality repair.
2. The method for quality restoration of solar extreme ultraviolet images based on deep learning according to claim 1, characterized in that: The solar extreme ultraviolet image to be repaired and the reference solar extreme ultraviolet image obtained in step 1 are in the same time period, and the time span completely covers the imager's self-cleaning cycle, the Earth's revolution cycle and the Sun's rotation cycle. Quality screening of solar extreme ultraviolet reference images includes: designing a rejection function based on a Poisson distribution prior to clean the solar extreme ultraviolet reference images, and calculating the global mean of each reference image. and variance The image will only be retained if the following conditions are met: ; in It is a predefined tolerance coefficient.
3. The method for quality restoration of solar extreme ultraviolet images based on deep learning according to claim 1, characterized in that: The specific implementation method of step 2 is as follows: First, time matching is performed. Using the solar extreme ultraviolet image to be repaired as a reference, a reference solar extreme ultraviolet image with an absolute time close to it is searched and matched. The time error is allowed to be within ±k seconds, and the time-matched paired image is obtained. Subsequently, all images were normalized to their maximum and minimum values based on adaptive constraints of brightness distribution. To avoid the influence of outliers, the radiation range of different sensors was unified to 0 to 1. Then, the reference solar extreme ultraviolet image was resampled to the same size as the solar extreme ultraviolet image to be repaired. A geometric registration algorithm based on SIFT feature detection was used to register the reference image pixel by pixel onto the image to be repaired by estimating the homography transformation matrix, thereby completing the geometric matching and obtaining a spatiotemporally aligned paired dataset.
4. The method for quality restoration of solar extreme ultraviolet images based on deep learning as described in claim 3, characterized in that: The maximum and minimum value normalization based on the adaptive constraint of brightness distribution includes: By statistically analyzing the distribution characteristics of image pixel radiance, and constructing a cumulative distribution function for effective pixels, low-brightness boundary values are adaptively determined based on the cumulative distribution function. High brightness boundary value Subsequently, using the determined low-brightness boundary value High brightness boundary value As the upper and lower limits of normalization, the image pixel radiance values are cropped and subjected to maximum and minimum value normalization, and the expression is as follows: ; in Indicates the image at the pixel position The radiance value, the result after normalization The radiation range is strictly limited to This suppresses interference from pixels with abnormally extreme brightness.
5. The method for quality restoration of solar extreme ultraviolet images based on deep learning as described in claim 3, characterized in that: The geometric registration algorithm based on SIFT feature detection extracts feature points from the image using scale-invariant feature transformation and matching SIFT algorithm, and calculates the feature descriptor for each feature point to characterize its local texture features. Subsequently, the FLANN fast approximate nearest neighbor search library is used to perform preliminary matching between the feature descriptors of the solar extreme ultraviolet image to be restored and the reference solar extreme ultraviolet image, eliminating ambiguous or erroneous matches. After obtaining a sufficient number of reliable feature correspondence points, the Random Sample Consensus Algorithm (RANSAC) is used to suppress abnormal matching points, and the homography transformation matrix H is estimated to achieve spatial mapping from the reference image coordinate system to the image coordinate system to be restored. The transformation relationship is as follows: Among them, symbols This represents a similarity equivalence relation in the sense of homogeneous coordinates; and The homography transformation matrix represents the spatial position of pixels in the reference solar extreme ultraviolet image and the solar extreme ultraviolet image to be restored in a two-dimensional coordinate system, respectively. It is a 3×3 projection transformation matrix used to describe the two-dimensional spatial mapping relationship between the reference solar extreme ultraviolet image coordinate system and the solar extreme ultraviolet image coordinate system to be repaired.
6. The method for quality restoration of solar extreme ultraviolet images based on deep learning as described in claim 1, characterized in that: The generator consists of a downsampling stage, several residual modules with embedded channel attention, and an upsampling stage connected in series. The residual modules with embedded channel attention are used for deep modeling of the features output from the downsampling stage. Each residual module includes one residual branch and one skip connection branch. Let the input features be... The residual branch consists of two convolutional layers. The first convolutional layer is followed by a batch normalization layer and a ReLU activation function, and the second convolutional layer is followed by a batch normalization layer to obtain the residual feature representation. ( To achieve adaptive feature enhancement along the channel dimension, the residual module embeds a channel attention mechanism at the output position of the residual branch, which uses residual features... ( Taking a function as input, a channel-dimensional feature description vector is obtained through global average pooling (GAP). One-dimensional convolution is then used to model the local correlations between channels. Subsequently, a channel attention weight vector is generated using a sigmoid activation function. Based on this weight vector, the residual features are... ( Channel-by-channel weighting is performed, and skip connections are made between the branch and the input features. Element-by-element addition is performed to obtain the output of the residual module.
7. The method for quality restoration of solar extreme ultraviolet images based on deep learning as described in claim 1, characterized in that: The specific implementation process of step 4 is as follows: After radiometric normalization of any image to be repaired, it is input into the generator of the trained conditional generative adversarial network. G In the process, the initial restoration results are output; then, radiometric inverse normalization is performed to adjust the radiometric values to the same range as the original image to be restored, and finally, the restoration results of the solar extreme ultraviolet image with the correct range and clear structure are obtained. The calculation process for radiation anti-normalization is as follows: in This represents the repair result output by the generative adversarial network, and its value range is... ; This represents the restoration result after radiometric inverse normalization; L and U represent the low-brightness boundary value and high-brightness boundary value, respectively, determined adaptively based on the image brightness distribution during the radiometric normalization stage.
8. A solar extreme ultraviolet image quality restoration system based on deep learning, characterized in that: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the program instructions in the memory to execute the deep learning-based solar extreme ultraviolet image quality restoration method as described in any one of claims 1-7.
Citation Information
Patent Citations
Image restoration method and device based on generative adversarial network
CN117495724A
