A method for stabilizing a full-disk extreme ultraviolet time-series image of the sun

Abstract

The application discloses a method for stabilizing a full-disk extreme ultraviolet time sequence image of the sun, and relates to the technical field of solar observation image processing. The method comprises the following steps: acquiring a full-disk extreme ultraviolet time sequence image of the sun, and extracting gradient information of each frame of image; determining a solar edge region of each frame of image based on the gradient information; calculating a regional gradient difference value of a frame to be stabilized and a reference frame in the solar edge region, and minimizing the regional gradient difference value to obtain a first offset; extracting a matching point pair between the two frames; adjusting the matching point pair based on the first offset, determining an offset change amount based on a residual of the matching point pair and a matching point pair weight, and obtaining a second offset; and compensating the frame to be stabilized by using the second offset to obtain a stabilized frame. The application significantly suppresses the problems of mismatching and error accumulation caused by the rapid evolution of the solar active region by combining gradient difference minimization and anti-difference selection weight iteration optimization of the matching point pair, and improves the robustness and accuracy of the stabilization processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of solar observation image processing, and in particular to a method for stabilizing images of a full-disk EUV time-series image of the sun. BACKGROUND

[0002] Solar activity is the main driving source of the Earth's space environment change. Intense activities such as solar flares and coronal mass ejections can have a significant impact on Earth's communication and navigation systems. Extreme ultraviolet (EUV) imaging technology has become a core means for real-time monitoring of solar activity and supporting space weather forecasting due to its fast response characteristics and excellent early warning capabilities.

[0003] Due to factors such as high-frequency jitter of the satellite platform and atmospheric turbulence interference, the acquired solar EUV image sequence has significant overall frame-to-frame offset. This offset prevents the image sequence from being accurately aligned, which severely restricts high-precision solar activity analysis based on time-series images, such as accurate flare positioning, coronal mass ejection velocity measurement, and solar surface flow field inversion. Therefore, it is necessary to perform image stabilization on the solar EUV image sequence to eliminate the overall frame-to-frame offset and ensure the spatio-temporal consistency of the image data.

[0004] Currently, the image stabilization of the current solar EUV image sequence mainly focuses on two directions: (1) local registration and alignment of time-phase images; and (2) matching and alignment of images with different resolutions. Among them, electronic image stabilization technology, as the mainstream scheme of modern image stabilization, directly establishes the frame-to-frame mapping relationship by extracting image features, and has the advantages of high precision and strong flexibility. It should be noted that the related technology has the following defects when dealing with the stabilization of solar full-disk EUV time-series images: (1) it mainly targets time-phase or fixed resolution scenarios, and lacks systematic image stabilization research for dynamic time-series full-disk images; (2) the existing feature matching strategy does not consider the dynamic evolution characteristics between time-series images, and is prone to matching errors when the solar activity region morphology changes rapidly (such as flare explosion), leading to cumulative stabilization errors. These defects result in the related technology being unable to meet the geometric consistency maintenance requirements of solar full-disk time-series images, which severely restricts the construction accuracy of space weather models and the space safety warning capability.

[0005] Therefore, there is currently a lack of an accurate and reliable EUV full-disk solar image frame stabilization method to provide technical support for real-time monitoring of solar activity and supporting space weather forecasting. SUMMARY

[0006] The application provides a method for stabilizing images of a full-disk solar EUV time sequence image, which is used to eliminate overall offset between frames, ensure the spatiotemporal consistency of image data, effectively improve the accuracy of solar activity monitoring and the reliability of space weather prediction, and solve the defects of the above related technologies. The technical solution is as follows:

[0007] In a first aspect, the application provides a method for stabilizing images of a full-disk solar EUV time sequence image, which comprises:

[0008] obtaining a full-disk solar EUV time sequence image, and extracting gradient information of each frame of the image;

[0009] determining a solar edge region of each frame of the image based on the gradient information;

[0010] calculating a regional gradient difference value of the solar edge region of a frame to be stabilized and a reference frame, adjusting the frame to be stabilized based on different offset values to minimize the regional gradient difference value, and obtaining a first offset value;

[0011] extracting a matching point pair of the frame to be stabilized and the reference frame, adjusting the matching point pair based on the first offset value, determining an offset change value based on a residual of the matching point pair and different matching point pair weights, and obtaining a second offset value;

[0012] when the offset change value is less than a preset threshold, compensating the frame to be stabilized based on the second offset value to obtain a stabilized frame;

[0013] wherein the reference frame and the frame to be stabilized are two adjacent frames of the full-disk solar EUV time sequence image.

[0014] In an optional solution of the first aspect, the extraction of the gradient information of each frame of the full-disk solar EUV time sequence image comprises:

[0015] performing multi-scale down-sampling on each frame of the full-disk solar EUV time sequence image to obtain a plurality of images with different resolutions;

[0016] arranging the original image and the down-sampled images in layers in order of resolution size to construct an image pyramid corresponding to the frame;

[0017] performing gradient calculation on each resolution layer of the image pyramid to obtain a gradient amplitude map corresponding to each resolution layer;

[0018] wherein the gradient information of each frame of the image includes the gradient amplitude map corresponding to each resolution layer.

[0019] In an optional solution of the first aspect, the determination of the solar edge region of each frame of the image based on the gradient information comprises:

[0020] For each image pyramid of the image frames, a preset annular mask is applied on a corresponding lowest resolution layer to determine a coverage area of the annular mask on the image, a gradient accumulation value of the coverage area is extracted from a corresponding gradient amplitude map, a radius and / or a center position of the annular mask are adjusted, and the annular mask at which the gradient accumulation value takes a maximum value is output.

[0021] In order from low to high according to the resolution layers of the image pyramid, the annular mask adjusted on the previous resolution layer is mapped to the next resolution layer, and the step of determining the coverage area of the annular mask on the image is turned to until the annular mask corresponding to the highest resolution layer is obtained, and the annular mask corresponding to each image frame is obtained.

[0022] The sun edge region of the corresponding frame is determined based on the radius, the center position and the coverage area of the annular mask corresponding to each image frame.

[0023] In an optional implementation of the first aspect, the region gradient difference value of the to-be-stabilized frame and the reference frame in the sun edge region is calculated, the to-be-stabilized frame is adjusted based on different offset amounts to minimize the region gradient difference value, and the first offset amount is obtained, including:

[0024] The pixel point pairs with the same coordinates in the sun edge region of the to-be-stabilized frame and the reference frame are determined, the gradient values of the to-be-stabilized frame and the reference frame at the corresponding coordinate positions are extracted respectively, the gradient value difference of each pixel point pair is obtained by difference, and the region gradient difference value is obtained by summing the gradient value differences of all pixel point pairs.

[0025] The coordinates of the pixel point belonging to the to-be-stabilized frame in each pixel point pair are adjusted by the offset amount, the region gradient difference value is calculated based on different offset amounts respectively, and the first objective function is constructed.

[0026] When the first objective function takes a minimum value, the first offset amount is output.

[0027] In an optional implementation of the first aspect, the matching point pairs of the to-be-stabilized frame and the reference frame are extracted, including:

[0028] The sun radius and the sun center position are determined according to the sun edge region corresponding to each image frame, a circular mask with a radius of the sun radius is constructed with the sun center position as a circle, and a circular coverage area of the circular mask on the corresponding image is determined.

[0029] The circular coverage area is input into the trained feature extraction model, and the feature points and the descriptors of the corresponding image are extracted.

[0030] Feature matching is performed on the to-be-stabilized frame and the reference frame in combination with the extracted feature points and the descriptors, and the matching point pairs of the to-be-stabilized frame and the reference frame are output.

[0031] In an optional implementation of the first aspect, the adjusting the matching point pair based on the first offset includes:

[0032] constructing a translation transformation model with the first offset as an offset parameter, and adjusting coordinates of pixel points belonging to the to-be-stabilized frame in the matching point pair based on the translation transformation model;

[0033] The determining the offset change based on the residual sum of squares of the matching point pair and the different matching point pair weights includes:

[0034] calculating a residual sum of squares and a standard deviation of each matching point pair based on the adjusted matching point pair, and determining a corresponding matching point pair weight based on a ratio of the residual sum of squares and the standard deviation;

[0035] constructing a second objective function based on the residual sum of squares and the matching point pair weight of each matching point pair;

[0036] dynamically adjusting the matching point pair weight of each matching point pair to minimize the second objective function;

[0037] determining a corresponding residual sum of squares and matching point pair weight when the second objective function takes a minimum value, and determining a corresponding offset change;

[0038] updating the offset parameter based on the offset change to obtain the second offset.

[0039] In an optional implementation of the first aspect, after the second offset is obtained, the offset change and an offset change threshold are compared, and if the offset change is greater than or equal to the offset change threshold, the method further includes:

[0040] comparing the iteration number and an iteration number threshold, and if the iteration number is less than the iteration number threshold, taking the second offset as a new offset parameter, and turning to the step of constructing the translation transformation model;

[0041] until the offset change is less than the offset change threshold, or the iteration number reaches the iteration number threshold.

[0042] In a second aspect, the present application further provides a stabilizing device for a solar full-disk extreme ultraviolet time sequence image, including:

[0043] an image processing unit, configured to acquire a solar full-disk extreme ultraviolet time sequence image, and extract gradient information of each frame of image in the solar full-disk extreme ultraviolet time sequence image;

[0044] a first offset calculation unit, configured to determine a solar edge region of each frame of image based on the gradient information;

[0045] The first offset calculation unit is also used to calculate the regional gradient difference between the image frame to be stabilized and the reference frame in the solar edge region, and adjust the image frame to be stabilized based on different offsets to minimize the regional gradient difference, thereby obtaining the first offset.

[0046] The second offset calculation unit is used to extract the matching point pairs between the image frame to be stabilized and the reference frame, adjust the matching point pairs based on the first offset, and determine the offset change based on the residual of the matching point pairs and the different weights of the matching point pairs to obtain the second offset.

[0047] The image stabilization unit is used to compensate the image frame to be stabilized based on the second offset when the offset change is less than a preset threshold, so as to obtain a stabilized image frame.

[0048] The reference frame and the image frame to be stabilized are two adjacent frames in the solar disk extreme ultraviolet time-series image.

[0049] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method provided by the first aspect of this application or any implementation thereof.

[0050] Fourthly, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of this application or any implementation thereof.

[0051] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following:

[0052] This application provides a method for stabilizing solar extreme ultraviolet time-series images. By extracting gradient information from each frame and determining the solar edge region, a gradient-driven geometric consistency constraint mechanism is constructed, effectively enhancing the expression and utilization of image structural features. Furthermore, by combining gradient difference minimization with robust weighted iterative optimization of matching point pairs, the method significantly suppresses mismatches and error accumulation caused by rapid evolution of solar active regions (such as flare eruptions), improving the robustness and accuracy of image stabilization. Compared to methods in related technologies, this application can still achieve sub-pixel-level registration accuracy under strong dynamic interference, significantly improving the stabilization success rate and error control capability. It can effectively eliminate overall inter-frame offset, ensuring good spatiotemporal consistency of solar extreme ultraviolet time-series images, thus providing a more reliable data foundation for high-precision solar activity analysis, coronal mass ejection tracking, solar flow field inversion, and space weather forecasting. Attached Figure Description

[0053] In order to more clearly illustrate the technical solutions in the application or the related art, the accompanying drawings needed to be used in the description of the embodiments or the related art will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0054] Figure 1 is a flowchart of a sun full-disk extreme ultraviolet time sequence image image stabilization method provided by an embodiment of the present application;

[0055] Figure 2 is a schematic diagram of one frame of image in a sun full-disk extreme ultraviolet time sequence image provided by the present application;

[0056] Figure 3 is a schematic diagram of a gradient amplitude graph provided by an embodiment of the present application;

[0057] Figure 4 is a schematic diagram of a circular ring mask provided by an embodiment of the present application;

[0058] Figure 5 is a schematic diagram of a sun edge region provided by an embodiment of the present application

[0059] Figure 6 is a flowchart of a sun full-disk extreme ultraviolet time sequence image image stabilization method provided by an embodiment of the present application;

[0060] Figure 7 is a structural schematic diagram of a sun full-disk extreme ultraviolet time sequence image image stabilization device provided by an embodiment of the present application;

[0061] Figure 8 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely in the following with reference to the accompanying drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0063] The terms "comprise", "comprising", "include", "including", "have" and "having" and any variations thereof in the Specification and in the Claims herein are intended to cover both the express stated features or steps and also other features or steps which are inherent in the process, method, system, product or apparatus.

[0064] It should be noted that the terms "first" and "second" in the present application are only to distinguish similar objects, and do not represent a specific order of the objects. Understandably, the "first" and "second" can be interchanged in a specific order or sequence as appropriate. It should be understood that the objects distinguished by "first" and "second" can be interchanged as appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those described or illustrated herein.

[0065] It should be noted that in the related art, the X-EUV imager carried by FY-3E can be used for high time resolution solar full-disk observation. Since the ultraviolet band solar observation payload is installed on the satellite sun array and is not co-benchmarked with the platform, it is affected by the sun array flexible vibration and benchmark deformation, cannot use the high-precision attitude determination information of the satellite platform, and needs to adjust the images obtained by shooting through the image stabilization method based on the image itself.

[0066] Wherein, the image stabilization processing of the time sequence image can be understood as a process of analyzing and eliminating unnecessary shaking of the picture in the video caused by some factors through computer algorithm, so as to obtain a more smooth and stable video. Unlike traditional optical load, the solar image of the ultraviolet band solar observation load is affected by the corona, and is a complex and variable irregular circular image. Moreover, it is affected by the solar rotation and solar flare, and there is no fixed reference target in the image. The image registration method in the related art is difficult to apply to the image stabilization of the solar full-disk extreme ultraviolet time sequence image.

[0067] The present application will be described in detail below with reference to specific embodiments.

[0068] Next, a solar full-disk extreme ultraviolet time sequence image stabilization method provided by an embodiment of the present application will be described below with reference to Figure 1 Figure 1 , Figure 1 A flowchart of a solar full-disk extreme ultraviolet time sequence image stabilization method provided by an embodiment of the present application is shown. As shown in Figure 1 , the method comprises the following steps:

[0069] S101, acquiring a solar full-disk extreme ultraviolet time sequence image, and extracting gradient information of each frame of image in the solar full-disk extreme ultraviolet time sequence image; ​

[0070] S102, Determine the solar edge region of each frame image based on gradient information;

[0071] S103, calculate the regional gradient difference between the image frame to be stabilized and the reference frame in the solar edge region, and adjust the image frame to be stabilized based on different offsets to minimize the regional gradient difference, thus obtaining the first offset;

[0072] S104, Extract the matching point pairs between the image frame to be stabilized and the reference frame;

[0073] S105, adjust the matching point pairs based on the first offset, determine the offset change based on the residual of the matching point pairs and the different weights of the matching point pairs, and obtain the second offset;

[0074] When the change in offset is less than a preset threshold, S106 is executed to compensate the image frame to be stabilized based on the second offset, thereby obtaining a stabilized image frame.

[0075] In some embodiments, in S101, the sun can be observed via satellite, ground observation station, etc., and the continuous multi-frame images captured during the observation period constitute the solar disk extreme ultraviolet time-series image, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of one frame from the solar disk extreme ultraviolet time-series images provided in this application.

[0076] Specifically, in S101, the process of extracting gradient information from each frame of the solar disk extreme ultraviolet time-series image includes:

[0077] S1011, perform multi-scale downsampling on each frame of the solar disk extreme ultraviolet time-series image to obtain multiple images of different resolutions.

[0078] Specifically, different downsampling ratios can be set to obtain images of multiple different scales. For example, downsampling each frame of the original resolution image can yield low-resolution images at scales of 1 / 4 and 1 / 8.

[0079] S1012, the original image and the downsampled image are arranged in layers according to their resolution to construct the image pyramid of the corresponding frame.

[0080] Specifically, the high-resolution layer and other low-resolution layers of the image pyramid can be set sequentially in order of resolution from high to low. The image pyramid includes at least the image of the original resolution layer and the image of the low-resolution layer.

[0081] S1013, perform gradient calculation on each resolution layer in the image pyramid to obtain the gradient magnitude map corresponding to each resolution layer.

[0082] Specifically, the Sobel operator can be used to calculate the gradient of each resolution layer in the image pyramid layer by layer, and the formula for calculating the gradient of the Sobel operator is as follows:

[0083]

[0084] wherein, represents the calculated gradient amplitude, represents any pixel in the image, represents the gradient in the x direction, that is, the horizontal direction, for detecting vertical edges; represents the gradient in the y direction, that is, the vertical direction, for detecting horizontal edges.

[0085] Exemplarily, the gradient operator convolution kernel can be:

[0086]

[0087] Understandably, the gradient information of each frame of image includes the gradient amplitude map corresponding to each resolution layer in the image pyramid constructed based on the frame of image.

[0088] Exemplarily, as shown in Figure 3 , the gradient amplitude map provided by the embodiments of the present application is a schematic diagram. Figure 3

[0089] In some embodiments, in S102, the solar edge region of each frame of image is determined based on the gradient information, and the following steps are performed on the image pyramid of each frame of image:

[0090] S1021, applying a preset annular mask on the corresponding lowest resolution layer to determine the coverage area of the annular mask on the image.

[0091] Specifically, when applying the preset annular mask, the center of the image of the lowest resolution layer is taken as the center position initial value of the annular mask, and an empirical value is taken as the radius initial value of the preset annular mask.

[0092] Understandably, with the given center position initial value as the center of the circle, a reference circle of the annular mask is constructed based on the given annular mask radius, and at least two concentric circles are constructed by extending a predetermined number of pixels inward and outward along the radial direction of the reference circle, thereby constructing the annular annular mask.

[0093] It should be noted that the radius of the annular mask refers to the radius of the reference circle corresponding to the annular mask.

[0094] Specifically, the same number of pixels, for example, 2 pixels, can be expanded inward and outward along the radial direction of the reference circle, thereby constructing an annular mask with a width of 4 pixels. ​​​

[0095] It can be understood that the coverage area of the circular ring mask on the image is the area composed of the pixels on the image intersecting with the circular ring profile of the circular ring mask and the pixels within the circular ring profile.

[0096] Exemplarily, as shown in Figure 4 , Figure 4 Fig. 1 is a schematic diagram of a circular ring mask provided by an embodiment of the present application, the red dot is the center position of the circular ring mask, the green square is the pixel where the center position of the circular ring mask is located, and the light green area is the coverage area of the circular ring mask on the image, wherein the solid line circle represents the reference circle of the circular ring mask, R is the radius of the reference circle, and the dashed line circle is a concentric circle obtained based on the reference circle, wherein R 内环 , R 外环 are the inner ring radius and the outer ring radius of the circular ring mask, respectively.

[0097] S1022, extracting the gradient cumulative value of the coverage area from the corresponding gradient amplitude map.

[0098] Specifically, the gradient value of each pixel point in the coverage area can be extracted, and the gradient cumulative value of the coverage area can be obtained by accumulation calculation.

[0099] In some embodiments, the at least three concentric circles corresponding to the circular ring mask can be respectively assigned weights according to the normal distribution from inside to outside, and the weight of the reference circle is the largest.

[0100] The pixel points in the surrounding area of each concentric circle are determined, and the corresponding gradient cumulative value is calculated; wherein the surrounding area of the concentric circle can be understood as the pixel area intersecting with the concentric circle on the image.

[0101] Further, based on the given weight of each concentric circle, a weighted average value of the gradient cumulative value of each concentric circle is obtained by weighted calculation, and the weighted average value is taken as the gradient cumulative value of the coverage area.

[0102] In some embodiments, additional concentric circles can also be constructed between the inner ring and the reference circle and between the outer ring and the reference circle based on the circular ring mask, and the steps of calculating the gradient cumulative value and the weighted average value are respectively performed, which are not limited by the embodiments of the present application.

[0103] By S1022, multiple concentric circles are selected to more comprehensively consider the pixel information of different ranges around the reference circle, and the weighted average value is calculated by comprehensively accumulating the pixel gradient values in multiple radius ranges, which can more accurately reflect the gradient characteristics in the region. In addition, the setting of different radii can capture the gray value change of the circular edge region, prevent the influence of the possible blurred region, and avoid missing important features due to considering only a single radius of the circular edge region. It should be noted that the weight of each concentric circle is set, because in the image with a circular edge, the edge information near the circle can better represent the essential characteristics of the circular edge, and giving it a higher weight can highlight the influence of these key regions, which helps to find the best circular contour of the sun image.

[0104] S1023, adjust the radius and / or center position of the annular mask, and output the annular mask when the gradient accumulation value is maximum.

[0105] Specifically, the radius and / or center position of the annular mask can be adjusted within a given search range, and different combinations of center positions and radii of the annular mask are selected to calculate corresponding gradient accumulation values.

[0106] For example, the search range can be understood as: based on the coordinates of the center point of the image, an offset is set, for example, set to ±10 pixels, and the adjustment amount of the radius can be determined according to the empirical value, so as to obtain multiple different parameter combinations of the annular mask.

[0107] Specifically, by continuously adjusting the parameter combination of the annular mask, the gradient accumulation value is maximized, the annular mask with the maximum gradient accumulation value is extracted, and the optimal annular mask of the current resolution layer is obtained.

[0108] S1024, according to the order from low to high of the resolution layers of the image pyramid, map the annular mask adjusted in the last resolution layer to the next resolution layer, and go to S1021 to determine the covering area of the annular mask on the image, until the annular mask corresponding to the highest resolution layer is obtained.

[0109] S1025, the annular mask of the highest resolution layer is taken as the annular mask corresponding to the current frame.

[0110] Wherein, the highest resolution layer is the original resolution corresponding to the current frame.

[0111] S1026, based on the radius, center position and covering area of the annular mask corresponding to each frame of image, the solar edge region of the corresponding frame is determined.

[0112] Specifically, the radius of the sun disk in the current image frame can be determined based on the radius of the annular mask of the current image frame, the sun center position of the sun disk in the current image frame can be determined based on the center position of the annular mask, and the reference circle of the coverage area of the annular mask determined as the optimal circular profile of the sun disk is extracted, so as to obtain the sun edge region of the current image frame.

[0113] Further, the steps of S1021-S1026 are performed for each image frame respectively, so as to extract the sun edge region of each image frame respectively.

[0114] In some embodiments, after the sun edge region is calculated in the first frame of the sun disk extreme ultraviolet time sequence image, in the process of searching for the sun edge region of the subsequent frame, in order to save the search time, the radius of the annular template can be fixed based on the determined sun radius in S1023, and only the center position of the annular template is adjusted.

[0115] Exemplarily, the extracted sun edge region is as shown in Figure 5 Figure 5 Exemplary diagram of the sun edge region provided by the embodiments of the present application is shown.

[0116] Further, the steps of S103 are performed, specifically including:

[0117] S1031, determining the pixel point pairs with the same coordinates in the respective sun edge regions of the to-be-stabilized frame and the reference frame.

[0118] It should be noted that the reference frame and the to-be-stabilized frame are two adjacent frames in the sun disk extreme ultraviolet time sequence image, and the reference frame is the frame adjacent to the to-be-stabilized frame.

[0119] Based on this understanding, the to-be-stabilized frame and the reference frame can be aligned, and a plurality of pixel point pairs with the same coordinates on the two frames can be easily determined.

[0120] Optionally, for the first frame in the sun disk extreme ultraviolet time sequence image, when the first frame is taken as the to-be-stabilized frame, a copy frame can be generated based on the first frame, and the copy frame is taken as the reference frame of the first frame.

[0121] S1032, respectively extracting the gradient values of the to-be-stabilized frame and the reference frame at the corresponding coordinate positions, and obtaining the gradient value difference of each pixel point pair by difference, and summing the gradient value differences of all pixel point pairs to obtain the regional gradient difference value.

[0122] Specifically, according to the coordinates of the pixel points in each pixel point pair, the gradient values can be extracted from the gradient amplitude maps of the to-be-stabilized frame and the reference frame respectively, and the gradient value difference can be obtained by further difference.

[0123] ​S1033, adjust the coordinates of the pixel points belonging to the frame to be stabilized in each pixel point pair by the offset.

[0124] Specifically, a given offset is applied to the pixel points belonging to the frame to be stabilized in each pixel point pair, and the value of the offset can be given based on a preset search range, for example, the search range is set to [-10 pixels, +10 pixels].

[0125] The offset can be represented as , is the offset in the x direction, is the offset in the y direction.

[0126] It can be understood that the coordinates of the pixel points belonging to the frame to be stabilized in the adjusted pixel point pair can be represented as , is the coordinates of the corresponding pixel point.

[0127] S1034, calculate the regional gradient difference value based on different offsets respectively, construct a first target function, realize the consistency constraint of the solar edge region between frames, and apply the formula:

[0128] ;

[0129] S1035, output the first offset when the first target function takes the minimum value.

[0130] wherein, represents the first target function, which represents the sum of the gradient information difference on the annular mask between the two frames when the offset is , represents the gradient of the pixel point position with coordinates on the frame to be stabilized.

[0131] In some embodiments, the step of extracting matching point pairs in S104 specifically includes:

[0132] S1041, determine the solar radius and the solar center position according to the solar edge region corresponding to each frame, construct a circular mask with a circular shape and a radius of the solar radius with the solar center position as the center, and determine the circular coverage area of the circular mask on the corresponding image.

[0133] In this way, the interference area located at the solar edge and outside can be shielded, thereby reducing semantic ambiguity matching.

[0134] S1042, take the circular coverage area extracted in S1041 as the input of the trained feature extraction model, and extract the feature points and descriptors through the trained feature extraction model.

[0135] The selected feature extraction model can be a SuperPoint network, which is not limited in the embodiments of the application.

[0136] S1043, in combination with the extracted feature points and descriptors, performing feature matching on the to-be-stabilized frame and the reference frame to output a matching point pair of the to-be-stabilized frame and the reference frame.

[0137] Specifically, the LightGlue algorithm can be selected to perform feature matching calculation on the adjacent to-be-stabilized frame and the reference frame, so as to determine each matching point pair, and a matching point pair set M can be output, which is represented as:

[0138] ;

[0139] wherein, represents a matching point of the reference frame, represents a matching point of the to-be-stabilized frame, is the number of each matching point pair.

[0140] It should be noted that the coordinates of the processed matching points are sub-pixel accuracy.

[0141] Further, the step S105 is performed to obtain a higher-precision inter-frame offset, as shown in the following formula: Figure 6 The step specifically includes:

[0142] S1051, setting the first offset as an offset parameter.

[0143] S1052, constructing a translation transformation model based on the set offset parameter, and adjusting the coordinates of the pixel points belonging to the to-be-stabilized frame in the matching point pair based on the translation transformation model, which is represented as:

[0144] ;

[0145] wherein, , is the coordinates of the pixel points belonging to the to-be-stabilized frame in the i th matching point pair, and , is the matching point coordinates of the to-be-stabilized frame after adjustment based on the translation transformation model, represents the offset.

[0146] S1053, calculating the residual and standard deviation of each matching point pair based on the adjusted matching point pair.

[0147] wherein, the formula for calculating the residual is:

[0148] wherein, represents the residual of the i th matching point pair in the matching point pair set, The matching point coordinates belonging to the reference frame.

[0149] It can be understood that the calculation of the standard deviation is the same as the calculation process in the related art, and details are not repeated here.

[0150] S1054, based on the ratio of the residual and the standard deviation, the corresponding matching point pair weight is determined, and the formula is applied:

[0151] ;

[0152] ;

[0153] wherein, is a preset weight initial value, is the matching point pair weight determined by S1054, , , is a weight parameter, is the standard deviation.

[0154] The value of may be set , may be adjusted according to the actual situation, and the embodiments of the present application do not limit it.

[0155] S1055, based on the residual and the matching point pair weight of each matching point pair, a second target function is constructed, and the second target function is represented as:

[0156] .

[0157] S1056, dynamically adjusting the matching point pair weight of each matching point pair.

[0158] Through iterative convergence and parameter updating by S1056, the second target function is minimized, including:

[0159] By dynamically adjusting the value of the standard deviation, the matching point pair weight of each matching point pair is dynamically adjusted, and the formula is applied:

[0160] ;

[0161] wherein, is a new standard deviation, is a function for calculating the standard deviation;

[0162] Based on the new standard deviation go to the step of S1054, update the matching point pair weight, until the value of the second target function takes the minimum value.

[0163] S1057, determine the residual and the matching point pair weight corresponding to the minimum of the second target function, and determine the corresponding offset change amount At.

[0164] S1058, update the offset parameter by the offset change amount At to obtain a second offset.

[0165] S1059, compare the offset change amount with an offset change amount threshold.

[0166] Specifically, whether the iteration converges or not is determined by comparing the offset change amount with the offset change amount threshold. The offset change amount in the x direction and the offset change amount in the y direction can be compared with the offset change amount threshold, respectively. In the case of convergence, the offset change amount in the x direction and the offset change amount in the y direction should both be less than the offset change amount threshold.

[0167] Exemplarily, the offset change amount threshold can be 0.01 pixel.

[0168] If the offset change amount is less than the offset change amount threshold, it indicates that the current offset parameter iteration converges, and the step of S106 is performed.

[0169] If the offset change amount is greater than or equal to the offset change amount threshold, the following steps are performed:

[0170] S10510, compare the iteration number with an iteration number threshold.

[0171] If the maximum iteration number is reached, the iteration process is terminated, the second offset obtained in the current iteration is output, and the step of S106 is performed.

[0172] If the iteration number is less than the iteration number threshold, the following steps are performed:

[0173] S10511, take the second offset as a new offset parameter, and perform the steps of S1052-S1059 again based on the second offset obtained in the current iteration, until the offset change amount is less than the offset change amount threshold, or until the iteration number reaches the maximum iteration number.

[0174] Specifically, by the above steps, the offset can be continuously optimized, the sub-pixel level offset estimation can be realized, and the second offset finally output is taken as the fine correction sub-pixel translation parameter to perform the step of S106.

[0175] In some embodiments, in S106, the second offset can be applied to the coordinates of each pixel point of the frame to be stabilized to compensate for the overall inter-frame offset between the frame to be stabilized and the reference frame, so that the two frames can be accurately aligned.

[0176] Further, the steps of S101-S106 are repeatedly performed for each frame of the solar full-disk extreme ultraviolet time-series image, so as to complete inter-frame image stabilization of each frame of the solar full-disk extreme ultraviolet time-series image.

[0177] It should be noted that, by gradient-driven geometric consistency constraint modeling and robust selection weight iterative optimization, the embodiment of the present application can effectively suppress dynamic feature interference, and can improve the image stabilization precision to a sub-pixel level (RMSE better than 0.5 pixels) by combining deep feature matching and robust optimization. Experiments show that, compared with the traditional RANSAC method, the image stabilization success rate of the embodiment of the present application can be improved to 100% in a scene with intense solar activity, and the error standard deviation is reduced by 61.2%, which significantly improves the stability of the solar extreme ultraviolet time-series image, and can provide accurate and high-precision data support for the construction of a spatial weather model and the early warning of space safety.

[0178] The following is a device embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0179] Next, please refer to Figure 7 The following is a structure schematic diagram of a solar full-disk extreme ultraviolet time-series image stabilization device provided by an exemplary embodiment of the present application. The device can be realized by software, hardware or a combination of the two to become all or part of a terminal, and can also be integrated as an independent module on a server. The device in the embodiment of the present application can be applied to a terminal or cloud, and the device comprises:

[0180] An image processing unit is configured to acquire a solar full-disk extreme ultraviolet time-series image and extract gradient information of each frame of the solar full-disk extreme ultraviolet time-series image.

[0181] A first offset calculation unit is configured to determine a solar edge region of each frame based on the gradient information.

[0182] The first offset calculation unit is further configured to calculate a region gradient difference value of the solar edge region of a frame to be stabilized and a reference frame, adjust the frame to be stabilized based on different offset amounts to minimize the region gradient difference value, and obtain a first offset amount.

[0183] A second offset calculation unit is configured to extract a matching point pair of the frame to be stabilized and the reference frame, adjust the matching point pair based on the first offset amount, determine an offset variation amount based on a residual error of the matching point pair and different matching point pair weights, and obtain a second offset amount.

[0184] An image stabilization unit is configured to compensate the frame to be stabilized based on the second offset amount when the offset variation amount is less than a preset threshold, and obtain a stabilized frame.

[0185] The reference frame and the frame to be stabilized are two adjacent frames in the full-disk extreme ultraviolet time sequence image of the sun.

[0186] It should be noted that the apparatus provided in the above embodiments is only used to execute the method provided in the above embodiments, and the above functions are completed by the above functional modules, and in actual applications, the above functions can be completed by different functional modules according to requirements, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the implementation process is described in detail in the method embodiments, which will not be described here.

[0187] The embodiment of the present application further provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the program to realize the steps of the method of any one of the above embodiments.

[0188] Please refer to Figure 8 The structure block diagram of an electronic device provided in the embodiment of the present application is shown in the figure.

[0189] As Figure 8 shown, the electronic device 800 includes a processor 801 and a memory 802.

[0190] In the embodiment of the present application, the processor 801 is the control center of the computer system, which can be the processor of a physical machine or the processor of a virtual machine. The processor 801 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 can be implemented in at least one of the hardware forms of a DSP (Digital Signal Processing), a FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array).

[0191] The processor 801 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state.

[0192] The memory 802 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 802 can also include high-speed random access memory and can include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction for being executed by the processor 801 to implement the method in the embodiments of the present application.

[0193] In some embodiments of the present application, the electronic device 800 further includes a peripheral device interface 803 and at least one peripheral device 804. The processor 801, the memory 802 and the peripheral device interface 803 can be connected through a bus or a signal line. Each peripheral device 804 can be connected to the peripheral device interface 803 through a bus, a signal line or a circuit board. Specifically, the peripheral device interface 803 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802.

[0194] In some embodiments of the present application, the processor 801, the memory 802 and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 801, the memory 802 and the peripheral device interface 803 can be implemented on a separate chip or circuit board. The embodiments of the present application do not make specific limitations in this regard.

[0195] The structure block diagram of the electronic device shown in the embodiments of the present application does not constitute a limitation on the electronic device 800, and the electronic device 800 can include more or fewer components than shown, or combine certain components, or adopt a different arrangement of components.

[0196] The embodiments of the present application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the preceding embodiments. The computer-readable storage medium can include, but is not limited to, any type of disk including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro-drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic or optical card, a nanosystem (including molecular memory IC), or any type of media or device suitable for storing instructions and / or data.

[0197] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform from the above description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions or the part that contributes to the related art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments.

[0198] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for stabilizing a solar full-disk extreme ultraviolet time-series image, characterized in that, include: A time-series image of the solar disk extreme ultraviolet radiation is acquired. Each frame of the solar disk extreme ultraviolet time-series image is downsampled at multiple scales to obtain multiple images of different resolutions. The original image and the downsampled image are arranged in layers according to their resolution to construct an image pyramid for the corresponding frame. Gradient calculation is performed on each resolution layer in the image pyramid to obtain the gradient magnitude map corresponding to each resolution layer as the gradient information of the corresponding frame; For each frame of the image pyramid, a preset circular mask is applied to the lowest resolution layer to determine the coverage area of ​​the circular mask on the image. The gradient accumulation value of the coverage area is extracted from the corresponding gradient magnitude map. The radius and / or center position of the circular mask are adjusted, and the circular mask with the maximum gradient accumulation value is output. Following the image pyramid resolution layers from low to high, the circular template obtained by adjusting the previous resolution layer is mapped onto the next resolution layer, and the process continues to the step of determining the coverage area of ​​the circular mask on the image, until the circular mask corresponding to the highest resolution layer is obtained, thus obtaining the circular mask corresponding to each frame of the image. The solar edge region of the corresponding frame is determined based on the radius, center position, and coverage area of ​​the circular mask corresponding to each frame image. Calculate the regional gradient difference between the image frame to be stabilized and the reference frame in the solar edge region, and adjust the image frame to be stabilized based on different offsets to minimize the regional gradient difference, thus obtaining the first offset. Extract matching point pairs between the image frame to be stabilized and the reference frame; The matching point pairs are adjusted based on the first offset, and the offset change is determined based on the residual of the matching point pairs and the weights of different matching point pairs to obtain the second offset. When the change in offset is less than a preset threshold, the image frame to be stabilized is compensated based on the second offset to obtain a stabilized image frame; The reference frame and the image frame to be stabilized are two adjacent frames in the solar disk extreme ultraviolet time series image; When applying the preset annular mask, the center of the image of the lowest resolution layer is taken as the initial value of the center position of the annular mask. Based on the preset initial value of the radius of the annular mask, a reference circle of the annular mask is constructed. A preset number of pixels are extended inward and outward along the radial direction of the reference circle to construct at least two concentric circles, thus constructing an annular mask. The step of extracting the cumulative gradient value of the covered area from the corresponding gradient magnitude map includes: The at least three concentric circles corresponding to the annular mask are weighted according to the normal distribution, and the maximum weight corresponds to the reference circle among the at least three concentric circles. Identify the pixels in the region surrounding each concentric circle and calculate the corresponding gradient accumulation value; where the region surrounding the concentric circle is the pixel region where the image intersects with the concentric circle. A weighted average of the gradient accumulation values ​​of each concentric circle is calculated based on the weight of each concentric circle, and this weighted average is used as the gradient accumulation value of the covered area.

2. The image stabilization method for a solar full-disk extreme ultraviolet time-series image according to claim 1, characterized in that, The calculation of the regional gradient difference between the image frame to be stabilized and the reference frame in the solar edge region, and the adjustment of the image frame to be stabilized based on different offsets to minimize the regional gradient difference, to obtain the first offset, includes: The image frame to be stabilized and the reference frame are identified as having the same coordinates within their respective solar edge regions. The gradient values ​​of the image frame to be stabilized and the reference frame at their corresponding coordinate positions are extracted. The gradient value difference of each pixel pair is obtained by subtracting the gradient value differences of all pixel pairs. The gradient value difference of the region is obtained by summing the gradient value differences of all pixel pairs. The coordinates of the pixels belonging to the image frame to be stabilized in each pixel pair are adjusted by offset, and the regional gradient difference is calculated based on different offsets to construct the first objective function. When the first objective function reaches its minimum value, the first offset is output.

3. The image stabilization method for a solar full-disk extreme ultraviolet time-series image according to claim 1, characterized in that, The extraction of matching point pairs between the image frame to be stabilized and the reference frame includes: The solar radius and solar center position are determined based on the solar edge region corresponding to each frame image. A circular mask with the solar center position as the circle and the radius of the solar center position as the circle is constructed. The circular coverage area of ​​the circular mask on the corresponding image is then determined. The circular coverage area is input into the trained feature extraction model to extract the feature points and descriptors of the corresponding image; By combining the extracted feature points and descriptors, feature matching is performed on the image frame to be stabilized and the reference frame, and the matching point pairs between the image frame to be stabilized and the reference frame are output.

4. A method for stabilizing a solar full-disk extreme ultraviolet time-series image according to any one of claims 1-3, characterized in that, The adjustment of the matching point pair based on the first offset includes: Using the first offset as the offset parameter, a translation transformation model is constructed, and the coordinates of the pixels belonging to the image frame to be stabilized in the matching point pair are adjusted based on the translation transformation model. The determination of the offset change based on the residuals of matching point pairs and different weights of matching point pairs to obtain the second offset includes: The residuals and standard deviations of each matching point pair are calculated based on the adjusted matching point pairs, and the corresponding matching point pair weights are determined based on the ratio of the residuals to the standard deviations. A second objective function is constructed based on the residuals and weights of each matching point pair. The weights of each matching point pair are dynamically adjusted to minimize the second objective function; Determine the residual and the weight of the matching point pair when the second objective function reaches its minimum value, and then determine the corresponding change in offset. The offset parameter is updated based on the change in offset to obtain the second offset.

5. The image stabilization method for a solar full-disk extreme ultraviolet time-series image according to claim 4, characterized in that, After obtaining the second offset, the method compares the offset change with an offset change threshold. If the offset change is greater than or equal to the offset change threshold, the method further includes: Compare the number of iterations with the iteration number threshold. If the number of iterations is less than the iteration number threshold, then use the second offset as the new offset parameter and proceed to the step of constructing the translation transformation model. Continue until the change in offset is less than the threshold for the change in offset, or the number of iterations reaches the threshold for the number of iterations.

6. An apparatus for stabilizing a solar full-disk extreme ultraviolet time-series image based on any one of claims 1-5, characterized in that, The device includes: An image processing unit is used to acquire a time-series image of the solar disk's extreme ultraviolet radiation and extract gradient information from each frame of the time-series image of the solar disk's extreme ultraviolet radiation. The first offset calculation unit is used to determine the solar edge region of each frame image based on gradient information; The first offset calculation unit is also used to calculate the regional gradient difference between the image frame to be stabilized and the reference frame in the solar edge region, and adjust the image frame to be stabilized based on different offsets to minimize the regional gradient difference, thereby obtaining the first offset. The second offset calculation unit is used to extract the matching point pairs between the image frame to be stabilized and the reference frame, adjust the matching point pairs based on the first offset, and determine the offset change based on the residual of the matching point pairs and the different weights of the matching point pairs to obtain the second offset. The image stabilization unit is used to compensate the image frame to be stabilized based on the second offset when the offset change is less than a preset threshold, so as to obtain a stabilized image frame. The reference frame and the image frame to be stabilized are two adjacent frames in the solar disk extreme ultraviolet time-series image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images