An aurora image spatial scale alignment method based on polar space low rank

Abstract

A method of aurora image spatial scale alignment based on polar space low rank. Firstly, according to the light intensity of aurora in longitude, the aurora image is mapped to polar coordinates with the weakest meridian as the starting point, realizing the polar representation of the full sky view. The adjacent frames are also mapped to the polar space with the same starting point. Then, in the polar space-time, the non-local similarity is calculated, and the similar blocks are weighted and fused by using the asymptotic non-local mean algorithm. Finally, the multi-frame image blocks with continuity in the polar space-time are embedded into the low rank regularization constraint to realize the scale change of the image. This method can be used for spatial scale alignment of aurora images of different scales obtained by different stations, and provides a data processing method for joint observation of aurora by global observation stations.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for spatial scale alignment of aurora images based on low-rank epipolar space; Background Technology

[0002] The aurora is the response of solar activity to the Earth's upper atmosphere. It is the most concentrated manifestation of polar solar-terrestrial physical processes (especially magnetosphere-ionosphere interaction) and the only geophysical phenomenon with polar characteristics that can be directly observed by people. Currently, there are various ways to obtain aurora data. The advantage of the all-sky aurora data used in this invention compared to other data is that it can obtain two-dimensional morphological information, enabling continuous observation of aurora activity.

[0003] All-sky aurora images possess high temporal and spatial resolution and rich texture features, thus allowing the introduction of ideas and methods from the field of dynamic texture mapping into ASI aurora image sequences. For DTI, some image structures may not reappear during observation, making it difficult to discover periodic patterns in a single image. However, the dynamic processes in the spatiotemporal dimensions exhibit regular trends, indicating that these images have high correlation and nonlocal similarity. Therefore, we need to study more suitable reconstruction algorithms based on the image's imaging rules, shape features, edge features, and spatiotemporal relationship features to achieve better fusion results. To maintain spatiotemporal continuity, we need to fully utilize the connection between consecutive frames and not be limited to a single frame. Changes at the same point in continuously acquired image sequences can be considered dynamically continuous. We can introduce this spatiotemporal continuity into aurora image sequence reconstruction to achieve spatial scale alignment of aurora images.

[0004] In previous research on aurora image reconstruction techniques, a large number of models and algorithms with research significance have been proposed. Among them, three categories are the most representative: interpolation-based methods, learning-based algorithms, and reconstruction-based algorithms. With the development of optimization theory, in order to improve the accuracy of image reconstruction, an image reconstruction method based on epipolar space low-rank matrix recovery has been proposed. Since aurora image sequences have spatiotemporal continuity, much similar information is likely to be linearly represented by neighborhood or combinations of similar pixels in the neighborhood. In 2005, Baudes et al. proposed a nonlocal mean denoising algorithm (NLM) based on the characteristics of a large number of similar image structures. This algorithm uses the idea of ​​weighted average to estimate the value of the pixel to be processed. The weight is calculated by the similarity between two image patches, and the similarity is measured by Gaussian weighted Euclidean distance. Nercessian et al. proposed a multi-scale NLM algorithm (MSNLM). Liu et al. proposed a new multi-scale nonlocal mean (MNLM) image denoising method by introducing multi-scale decomposition of the image. In addition, Wang et al. provided a new nonlocal weight function based on structural similarity index.

[0005] Currently, we have implemented aurora classification and event detection using deep learning methods on a large number of aurora images. However, these methods only consider single-resolution images from the same observation platform. When studying aurora images with complex structures, such as integrating multi-scale images from different observation devices into a convolutional neural network, the different resolutions become a major problem hindering the successful implementation of the algorithm. In this case, it is necessary to perform a uniform resolution operation on the original aurora image data to achieve spatiotemporal scale alignment of aurora images. Therefore, we consider using image reconstruction technology to achieve resolution alignment of multi-scale aurora images, thereby enabling joint observation of data from multiple platforms and synchronous information fusion of observation image data at different resolutions.

[0006] In summary, existing methods for achieving resolution alignment of multi-scale aurora images using image reconstruction techniques mainly suffer from the following problems:

[0007] 1) Starting from the overall image sequence, the spatiotemporal structure and spatial neighborhood relationship between adjacent frame image data are ignored, and the useful prior information of the dataset is not utilized to the maximum extent. In previous studies, only single-resolution images from the same observation platform were considered. When using multi-observation platform data and facing a large number of images with different resolutions, it is necessary to align the spatiotemporal scales of the image data acquired by different observation platforms to achieve joint observation of multi-platform data and synchronization of information fusion of different resolutions of the observed image data. At this time, the misalignment of scale and region is the main challenge to processing aurora image data and achieving satisfactory performance.

[0008] 2) The aurora arcs in the entire sky aurora image have different curvatures. If the image is reconstructed directly, the shape of the scaled aurora arc will change, increasing the number of pixels stuck between multiple arcs, making it impossible to completely separate each arc in the image. Summary of the Invention

[0009] The purpose of this invention is to provide a spatial scale alignment method for aurora images based on low-rank polar space, specifically a method based on polarimetric nonlocal spatiotemporal information fusion and low-rank representation. This method introduces spatiotemporal continuity into the aurora image spatial scale alignment method and explores the nonlocal similarity between adjacent frames in the form of image patches. Similar patches are then weighted and fused based on a progressive nonlocal algorithm. Finally, the low-rank aurora image spatial scale alignment method is applied to a global image containing spatiotemporal continuity to obtain reconstructed images at different resolutions. Furthermore, for all-sky images, we propose an adaptive polar coordinate mapping strategy that alters their neighborhood relationships to achieve complete aurora arc segmentation on ASI images based on the reconstruction results.

[0010] To achieve the above objectives, the technical solution of the present invention is as follows:

[0011] A method for spatial scale alignment of aurora images based on low-rank piriform space, comprising the following steps:

[0012] Step 1: Input a set of all-sky aurora images. Preprocess each frame of the image. The sequence of aurora images to be monitored, after preprocessing, forms the following sequence: X = {f(x,y,t), x,y∈Ω,t=1,2,…T}, where Ω represents the circular mask region of the all-sky aurora image, T represents the length of the aurora image sequence, and x,y represent the coordinates of a point at a certain time t. In the preprocessing, the largest inscribed circle is used as a mask to crop the size of the original image without losing the main image information. Then, the excess area outside the mask is removed after cropping.

[0013] Step 2: Based on the intensity of aurora emission along longitude, starting from the weakest meridian, the aurora image is mapped to polar coordinates. This means that any point in the image is transformed from Cartesian coordinates X(x,y) to polar coordinates Z(ρ,θ) to achieve a polarized representation of the entire sky view. For the sequence of images obtained after mask preprocessing, adjacent frames of the target image sequence are accessed in the order of adjacent frames. Using the same longitude starting point, adjacent frames are also transformed to polar coordinates and mapped to polar space to merge the information of consecutive frames in the aurora sequence. Therefore, the aurora image reconstruction model is represented as follows:

[0014]

[0015] Step 3: Using the result after polar coordinate transformation as input, divide each frame of the image into overlapping sub-blocks. The spatial size and sliding window step size of each sub-block are the same. Explore the spatiotemporal continuity and polarization characteristics of aurora images in the form of image blocks. For each block, search and select three types of non-local similar blocks from the local window and neighboring windows surrounding the block: spatial local neighborhood blocks, spatiotemporal local neighborhood blocks, and spatiotemporal non-local neighborhood blocks.

[0016] Step 4: Define an operator ξ k The algorithm searches and selects different numbers of similar blocks from the three types of reference blocks mentioned above. After dynamically selecting reference blocks from multiple images, it obtains the block with the highest similarity from the three neighborhoods based on Euclidean distance. It performs non-local self-similarity between different neighborhood blocks of the image and estimates the similarity between pixels by measuring the similarity between blocks.

[0017] Step 5: Employ a progressive nonlocal averaging algorithm to perform weighted fusion on similar blocks to estimate the pixel values ​​to be processed. The weights ω(x,y) are calculated from the similarity between two image blocks. The filtering parameter h is the same for the first iteration, and the weights from the first iteration are used as the filtering parameters for the second iteration. After the above operations, the aurora image reconstruction model can be expressed as:

[0018]

[0019] Step 6: After the above processing, the objective function model of the image is used to calculate the low-rank matrix recovery using the continuous SVD low-rank decomposition algorithm, introducing a redundant variable M. k (k) is used to simulate each dimension of the image, so that the reconstruction model of the objective function of the processed image can be represented as a new optimization problem, namely:

[0020]

[0021] Step 7: Use the ADMM algorithm, which decomposes the cost function of the original problem into multiple solvable subproblems, solves each subproblem in parallel, and finally adjusts the solutions of each subproblem to obtain the global solution. Solve the above optimization model and obtain the final reconstructed image to achieve spatial scale alignment of aurora images at different scales.

[0022] Step 2 involves performing a polar coordinate transformation on the cropped image to convert it to a polar space, including the following steps:

[0023] 2a) For the distribution area of ​​the aurora arc in the target image, we propose an adaptive polarity transformation strategy; based on the intensity of aurora emission along longitude, we take the weakest meridian as the starting point and use it as the starting angle for polar coordinate transformation, so that a complete aurora arc can be obtained after transformation.

[0024] 2b) Perform polar coordinate mapping on the aurora image. If the Cartesian coordinates of any point in an image are represented as X(x,y), then the polar coordinates obtained by the following transformation are Z(ρ,θ):

[0025]

[0026] Where (x0, y0) are the coordinates of the center point of the image transformation, ρ and θ correspond to the polar radius and polar angle in polar coordinates, respectively, and then the image Z after polar coordinate transformation is used as the input for the next step;

[0027] 2c) For the sequence of images obtained after mask preprocessing, the adjacent frames of the target image sequence are accessed in the order of adjacent frames. Using the same longitude starting point, the adjacent frames are also subjected to polar coordinate transformation using a unified polar transformation rule and mapped to polarization space to merge the information of the preceding and following frames of the aurora sequence.

[0028] 2d) Based on the degradation model and reconstruction framework of the aurora image described in step one, the reconstruction model can now be rewritten as:

[0029] Y = DSZ + N

[0030]

[0031] Here, F(Z) represents low-rank regularization related to polar coordinates, which is usually used to remove useless background information, change the spatial relationship of auroral arcs, and enhance the zenith area.

[0032] In step 3, each frame of the image is divided into overlapping sub-blocks. For each block, non-locally similar blocks are searched and selected from the local window and neighboring windows surrounding that block; this is done as follows:

[0033] 3a) First, select r consecutive frames of images, each image Z∈R m×n It is divided into overlapping sub-blocks with a spatial size of o. Multiple overlapping blocks; it is worth noting that dividing the image into overlapping image blocks can reduce artifacts in the reconstructed image; the spatial size of each sub-block is q×q, and the sliding step size is s;

[0034] 3b) Consider a square target block Z (θ,ρ) ={z(m+g m ,n+g n)}, where (m,n) is the position of the target pixel (block center), g m and g n This is the square spatial offset relative to the target pixel;

[0035] 3c) For the target block, non-locally similar blocks are searched and selected by overlapping scans in a spatiotemporally continuous polar space from the local window and adjacent windows surrounding the block. There are three types of reference blocks: spatial local neighborhood blocks, spatiotemporally local neighborhood blocks, and spatiotemporally non-local neighborhood blocks.

[0036] 3d) All central blocks and reference blocks together form a polarization subspace; we use nonlocal techniques to select relevant blocks to subdivide the dynamic polarization subspace into two smaller subspaces; then, we select spatiotemporally similar blocks and spatially similar blocks respectively to combine them to obtain the final polarization subspace.

[0037] In step 4, different numbers of similar blocks are searched and selected from three types of reference blocks. This invention defines an operator ξ. k Condition X is satisfied. k,r =ξ k X r To extract non-locally similar overlapping blocks, we use the following method: k is the number of similar blocks in each image; the block with the highest similarity is obtained from three neighborhoods based on Euclidean distance; non-local self-similarity is performed between different neighborhood blocks of the image, and the similarity between pixels is estimated by measuring the similarity between blocks; the Euclidean distance formula is as follows:

[0038]

[0039] in, and Represents different blocks at locations on the image, u and v are the indices of the matrix, and t represents multiple images with spatiotemporal continuity.

[0040] The preprocessing of the original image consists of five steps:

[0041] (1) Subtract the dark current;

[0042] (2) Remove edge noise;

[0043] (3) Grayscale stretching;

[0044] (4) Image rotation;

[0045] (5) Image cropping.

[0046] In the preprocessing of the original image, the reconstruction of the high-resolution image sequence of a general degradation model can be represented by the following observation model: Y = DSX + N. Low-rank regularization is used to realize the scale change of the image, so the reconstruction model of the aurora image is represented as follows:

[0047]

[0048] The first term represents the fidelity of Gaussian noise, the second term is the low-rank constraint term, and F(X) is the regularization parameter. The performance of the reconstruction model depends heavily on the choice of regularization. F(X) is the regularization parameter used to combine these two terms to balance their contributions to the final result.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] This invention takes into account the imaging patterns, texture features, edge features, and spatiotemporal relationship features inherent in the image dataset itself, and successfully integrates the different characteristics of aurora images in spatiotemporal structure and polarization space, thereby obtaining better reconstruction results.

[0051] This invention takes into account the different trends and distributions of aurora arcs. To avoid altering the neighborhood relationships of the aurora arcs, it proposes an adaptive polarity transformation strategy to transform the aurora image into a polarization space. This transforms the arcs, which originally exhibit a curved trend, into a "row / column" orientation, thus making it easier to achieve complete arc segmentation based on the reconstruction results.

[0052] This invention explores the nonlocal similarity of aurora images in time and space in the form of image patches. This not only allows for the further fusion of nonlocal information in the spatiotemporal structure of aurora images, but also enables spatial scale alignment of aurora images of different scales acquired from different sites.

[0053] Experimental results on an all-sky aurora image dataset demonstrate that the proposed aurora image spatial scale alignment method preserves details and boundary information, outperforming other reconstruction methods both subjectively and objectively. The reconstructed results were used as the basis for aurora arc segmentation experiments to test the practicality of the invention. Experimental results show that the invention can effectively complete the aurora arc segmentation task. Attached Figure Description

[0054] Figure 1 This is a flowchart of spatiotemporal nonlocal information fusion and low-rank regularization in this invention;

[0055] Figure 2 This is the adaptive polarity mapping map of the entire sky image in this invention;

[0056] Figure 3These are reconstruction results of different methods on ASI aurora image sequences. (a) Original image; (b) NN method; (c) LMMSE method; (d) LRTV method; (e) CNN method; (f) VDSR method; (g) +NLM method; (h) The method proposed in this invention.

[0057] Figure 4 These are reconstruction results of different methods under high-intensity aurora images, 2012-11-23 12:36:40. (a) Original image; (b) NN method; (c) LMMSE method; (d) LRTV method; (e) CNN method; (f) VDSR method; (g) +NLM method; (h) The method proposed in this invention.

[0058] Figure 5 These are reconstruction results of different methods under low-intensity aurora images, 2012-11-23 12:36:40. (a) Original image; (b) NN method; (c) LMMSE method; (d) LRTV method; (e) CNN method; (f) VDSR method; (g) +NLM method; (h) The method proposed in this invention.

[0059] Figure 6 This is the segmentation result of the ASI aurora arc using the method proposed in this invention. Specific implementation methods

[0060] The present invention will be further illustrated below through specific embodiments:

[0061] The technical approach to implementing this invention is as follows: First, the input raw low-resolution all-sky aurora image is preprocessed, using the largest inscribed circle as a mask to crop the original image. The cropped image is then transformed to polar coordinates, converting it to polar space. Next, each frame is divided into overlapping sub-blocks. Non-locally similar blocks are searched from the local window and adjacent windows surrounding each block, dynamically extracting locally similar blocks. Then, similar blocks are weighted and fused based on a progressive non-local algorithm. Finally, the ADMM algorithm is used to optimize the ASI aurora image sequence reconstruction model until convergence. The specific implementation steps are as follows:

[0062] like Figure 1 As shown, the steps of the aurora image spatial scale alignment method based on low-rank piriform space of the present invention are as follows:

[0063] Step 1: Input a set of all-sky aurora images and preprocess each frame. The original all-sky aurora image taken from the Yellow River Station in the Arctic is 512×512 in size, and the dynamic range of pixel grayscale is [0, 18000]. The preprocessing process of the original image consists of five steps: (1) dark current subtraction; (2) edge noise removal; (3) grayscale stretching; (4) image rotation; (5) image cropping. In order to remove useless information such as noise from the lights and mountains around the Yellow River Station, the size of the original image is cropped using the largest inscribed circle as a mask without losing the main image information. The radius of the circular mask is set to 220, and then the excess area outside the mask is removed by cropping. The size of the preprocessed image is 440×440, and the dynamic range of grayscale is [0, 4000]. The sequence of images obtained after preprocessing the aurora image sequence to be monitored is X = f(x,y,t), x,y∈Ω,t=1,2,…T, where Ω represents the circular mask region of the all-sky aurora image, T represents the length of the aurora image sequence, and x,y represent the coordinates of a point at a certain time t in the image. The reconstructed high-resolution image sequence of a general degradation model can be represented by the following observation model:

[0064] Y = DSX + N

[0065] Low-rank regularization is used to achieve image scaling. Therefore, the reconstruction model of aurora images is represented as follows:

[0066]

[0067] The first term represents the fidelity of Gaussian noise, and the second term is a low-rank constraint term. The performance of the reconstruction model depends heavily on the choice of regularization. F(X) is the regularization parameter used to combine these two terms to balance their contributions to the final result.

[0068] Step 2: Perform a polar coordinate transformation on the cropped image to convert it to polar space. That is, the Cartesian coordinates X(x,y) of any point in the image are transformed to polar coordinates Z(ρ,θ). Using the result of the polar coordinate transformation as input, each frame of the image is divided into overlapping sub-blocks. For each block, non-locally similar blocks are searched and selected from the local window and neighboring windows surrounding that block. There are three types of reference blocks: spatially local neighborhood blocks, spatiotemporally local neighborhood blocks, and spatiotemporally non-local neighborhood blocks. Then, an operator ξ is introduced. k The algorithm searches and selects different numbers of similar blocks from three types of reference blocks, and obtains the block with the highest similarity from the three neighborhoods based on the Euclidean distance.

[0069] 2.1) Perform polar coordinate transformation on the cropped image to convert the image to polar space.

[0070] 2.1.1) For the distribution area of ​​the aurora arc in the target image, we propose an adaptive polarity transformation strategy. Based on the intensity of aurora emission along longitude, the weakest meridian is taken as the starting point for polar coordinate transformation, so that a complete aurora arc can be obtained after transformation.

[0071] 2.1.2) Perform polar coordinate mapping on the aurora image. If the Cartesian coordinates of any point in an image are represented as X(x,y), then the polar coordinates obtained after the transformation by the following formula are Z(ρ,θ):

[0072]

[0073] Where (x0, y0) are the coordinates of the center point of the image transformation, and ρ and θ correspond to the polar radius and polar angle in polar coordinates, respectively. Then, the image Z after polar coordinate transformation is used as the input for the next step.

[0074] 2.1.3) For the sequence of images obtained after mask preprocessing, the adjacent frames of the target image sequence are accessed in the order of adjacent frames. Using the same longitude starting point, the adjacent frames are also transformed into polar coordinates using a unified polar transformation rule and mapped to polarization space to merge the information of the preceding and following frames of the aurora sequence.

[0075] 2.1.4) Based on the degradation model and reconstruction framework of the aurora image described in step one, the reconstruction model can now be rewritten as:

[0076] Y = DSZ + N

[0077]

[0078] Here, F(Z) represents low-rank regularization related to polar coordinates, which is usually used to remove useless background information, change the spatial relationship of auroral arcs, and enhance the zenith area.

[0079] 2.2) Using the polar coordinate transformation result as input, each frame of the image is divided into overlapping sub-blocks, with each sub-block having the same spatial size and sliding window step size. The spatiotemporal continuity and polarization characteristics of aurora images are explored in the form of image blocks. For each block, non-locally similar blocks are searched and selected from the local window and neighboring windows surrounding that block.

[0080] 2.2.1) First, we select images from r consecutive frames, each image Z∈R m×n It is divided into overlapping sub-blocks with a spatial size of o. Multiple overlapping blocks. Notably, dividing the image into overlapping image blocks can reduce artifacts in the reconstructed image. Each sub-block has a spatial size of q×q and a sliding step size of s.

[0081] 2.2.2) We consider a square target block Z (m,n) =z(m+g m ,n+g n ), where (m,n) is the position of the target pixel (block center), g m and g n This is the square spatial offset relative to the target pixel.

[0082] 2.2.3) For the target block, we extract non-locally similar blocks from the local window and neighboring windows surrounding the block by overlapping scan search in a spatiotemporally continuous polar space. There are three types of reference blocks: spatial local neighborhood blocks, spatiotemporally local neighborhood blocks, and spatiotemporally non-local neighborhood blocks.

[0083] 2.2.4) All center blocks and reference blocks together form a polarization subspace. We use nonlocal techniques to select relevant blocks, subdividing the dynamic polarization subspace into two smaller subspaces. Then, spatiotemporally similar blocks and spatially similar blocks are selected and combined to obtain the final polarization subspace. To incorporate the spatiotemporal continuity information of the ASI image, multiple spatiotemporally local and spatiotemporally nonlocal neighborhood blocks need to be forcibly selected. Because spatially local neighborhood blocks are more similar to the center blocks, more similar blocks are selected within the spatially local neighborhood when extracting similar blocks.

[0084] 2.3) Define an operator ξ k The algorithm searches and selects different numbers of similar blocks from three reference blocks, and then obtains the block with the highest similarity from the three neighborhoods based on Euclidean distance. Nonlocal self-similarity is performed between different neighborhood blocks of the image, and the similarity between pixels is estimated by measuring the similarity between blocks.

[0085] 2.3.1) An operator ξ is defined. k such that X satisfies k,r =ξ k X r The condition is used to extract non-locally similar overlapping blocks, where k is the number of similar blocks in each image.

[0086] 2.3.2) After dynamically selecting reference blocks from multiple images, each block finds the group of most similar blocks with the highest similarity in its current frame and adjacent frames based on Euclidean distance. The Euclidean distance formula is as follows:

[0087]

[0088] in, and Represents different blocks at locations on the image, u and v are the indices of the matrix, and t represents multiple images with spatiotemporal continuity.

[0089] 2.3.3) For several similar blocks obtained in consecutive frames, we use the same metric to obtain the k most similar blocks Z. k .

[0090] Step 3: A progressive nonlocal averaging algorithm is used to perform weighted fusion on similar blocks to estimate the pixel values ​​to be processed. The weights ω(x,y) are calculated from the similarity between two image blocks. After the above processing, the objective function model of the image is calculated using the continuous SVD low-rank decomposition algorithm to recover the low-rank matrix. The ADMM algorithm is used to solve the optimization model and obtain the final reconstructed image.

[0091] 3.1) The progressive nonlocal averaging algorithm is used to perform weighted fusion on similar blocks to estimate the pixel value to be processed.

[0092] 3.1.1) An Asymptotic Nonlocal Averaging (ANLM) algorithm is used to weightedly fuse the spatiotemporal continuity and nonlocal similarity information of the polarization space in ASI aurora images. When using the ANLM algorithm for measurement, the higher the similarity, the higher the weighting applied during image processing. Its basic formula is expressed as:

[0093]

[0094] Where ω(x,y) represents the weight, which indicates the similarity between pixel x and pixel y in the original image v, Ω x It is the neighborhood of pixel x.

[0095] 3.1.2) The weight values ​​must satisfy the requirement of being greater than 0 and summing to 1, which can be expressed by the formula:

[0096]

[0097] Furthermore, the concept of weighted average is reflected in the weight formula as follows:

[0098]

[0099] Where u(.) is the original image, m(x) is the normalization factor, and h > 0 is the filtering parameter.

[0100] 3.1.3) When performing the first filtering, all points use the same filtering parameters:

[0101] h1(i)=0.5σ

[0102] 3.1.4) During the second filtering, the filter parameter h2(i) is calculated point-by-point using the weights from the first filtering (different filter parameters are used at each point). Assume X i Let i = 1, 2, ..., n, where n is an independent and identically distributed random variable, then we have:

[0103]

[0104] Where D represents variance, Cov(X) i ,X j ) represents covariance.

[0105] 3.1.5) Due to X1, X2, ..., X i If they are mutually independent, then Cov(X) i ,X j Since ) = 0, i ≠ j, therefore:

[0106]

[0107] 3.1.6) Since the noise is independent and identically distributed, according to the above formula, after the first filtering, the noise standard deviation at each pixel i∈Ω is:

[0108]

[0109] 3.1.7) In the second filtering, the smoothing parameter is taken as the noise standard deviation at each pixel, that is:

[0110]

[0111] 3.1.8) After the original image undergoes PT, polarization, nonlocal spatiotemporal intensity fusion, and asymptotic weighting of similar blocks, the corresponding constraint problem is as follows:

[0112]

[0113] 3.2) After the above processing, the objective function model of the image is calculated using the continuous SVD low-rank decomposition algorithm to recover the low-rank matrix.

[0114] 3.2.1) Through the above operations, the image reconstruction algorithm in this paper can be expressed as:

[0115]

[0116] This model consists of two parts. The first part is a Gaussian noise fidelity term, which is used to consider noise removal and edge preservation. The second part is a nonlocal decomposition term based on spatiotemporal structure, which is used to characterize the nonlocal self-similarity of the image and the high correlation between the image sequence with spatiotemporal continuity.

[0117] 3.2.2) Use the continuous SVD low-rank decomposition algorithm to calculate the low-rank matrix recovery.

[0118] 3.2.3) Based on this, we introduce a redundant variable M. k(k)To simulate each dimension of the image so that it satisfies X k =ξ k X. Using the ALM algorithm, the above model can be written as:

[0119]

[0120] 3.3) The ADMM algorithm is used to decompose the cost function of the original problem into multiple solvable subproblems, solve each subproblem in parallel, and finally adjust the solutions of each subproblem to obtain the global solution, thereby solving the optimization model and obtaining the final reconstructed image.

[0121] 3.3.1) First, introduce N-dimensional redundant variables. To simulate the unfolding of the image in each dimension, the new cost function is as follows:

[0122]

[0123]

[0124] 3.3.2) Based on ADMM, the ALM form of the above cost function is expressed as follows, where the Lagrangian parameter is Y. i :

[0125]

[0126] 3.3.3) According to ADMM, the above formula can be broken down into the following three subproblems. The subproblems can be solved iteratively to address the variable update problem.

[0127] 3.3.4) Subproblem 1: Keep Y and M unchanged, update X (k+1) :

[0128]

[0129] 3.3.5) Sub-problem 2: Update X (k+1) Substitute the values, keep Y constant, and update.

[0130]

[0131] 3.3.6) Sub-problem 3: Update X (k+1) M i (k+1) Import, update

[0132] Y i (k+1) =Y i (k) +(X (k+1) -Mi (k+1 ))

[0133] 3.3.7) Repeat the above iterative operation until the algorithm converges, find the optimal feasible solution of this image reconstruction model, and obtain the final reconstructed image, thereby realizing the spatial scale alignment of aurora images of different scales.

[0134] The advantages of this invention can be further illustrated by the following experiments:

[0135] To demonstrate the effectiveness of the spatial scale alignment method proposed in this invention, we selected five representative image reconstruction methods for comparison: NN, LMMSE, LRTV, CNN, and VDSR. We also compared the +NLM method with our method. For the parameters used in this paper, the model regularization parameter was specified as λ = 0.5, and the noise variance was set to 1. The number of dynamically selected images with spatiotemporal continuity was r = 3, the sliding window radius was t = 60, the image patch size was f = 5, and the number of iterations was l = 4. Experiments were conducted on ASI aurora image sequences using the aforementioned different methods. To verify the effectiveness of this invention, the role of the proposed method in achieving resolution alignment of multi-scale aurora images was evaluated both qualitatively and quantitatively. Furthermore, the aurora image spatial scale alignment method was applied to ASI aurora image processing, and its practicality was tested in aurora arc segmentation experiments to study the direction and number of aurora arcs.

[0136] Experiment 1: Using qualitative examples to illustrate the effectiveness of the invention.

[0137] first, Figure 3 This image shows a comparison of the reconstruction results of the present invention and six different reconstruction methods on ASI aurora image sequences. (a) shows the original image; (b) the result using the NN method; (c) the result using the LMMSE method; (d) the result using the LRTV method; (e) the result using the CNN method; (f) the result using the VDSR method; (g) the result using the +NLM method; and (h) the result using the method proposed in this invention. To more intuitively illustrate the reconstruction results, we have magnified the results of the present invention and the six different reconstruction methods, and enlarged the green box area in the upper right corner of the image. From the comparison of reconstruction results of different methods, we can observe that all methods can improve image resolution to some extent, but the method proposed in this invention achieves the best results and the highest resolution among all methods.

[0138] Secondly Figure 4 and Figure 5 This presents our experimental results for ASI aurora image sequences at different aurora intensities (including high-intensity and low-intensity aurora images). Figure 4 This invention presents the reconstruction results of high-intensity aurora arc images using six different methods. Figure 5 This paper presents the reconstruction results of low-intensity aurora arc images using the present invention and six different methods. (a) shows the original image; (b) the result using the NN method; (c) the result using the LMMSE method; (d) the result using the LRTV method; (e) the result using the CNN method; (f) the result using the VDSR method; (g) the result using the +NLM method; and (h) the result using the method proposed in this invention. Similar to the experiments described above, we have magnified the green box area in the upper right corner (red box). By observing the magnified area, we can draw the same conclusion as the experiments above: under different aurora intensities (ASI), whether high-intensity or low-intensity aurora images, all methods can improve image resolution to some extent. However, the method proposed in this invention achieves the best results and the highest resolution among all methods.

[0139] In summary, the visualization experiments above demonstrate that all the comparative methods improved image resolution and eliminated some noise to some extent. Among them, the LMMSE method had the most severe impact on the image, mainly manifested in the blurring of aurora arc details, leading to unclear contours. CNN and VSDR also exhibited smoothing phenomena in some areas. However, the method proposed in this invention not only effectively suppressed noise but also preserved image details and contours. It achieved the highest resolution and best results among all the methods tested in this invention's comparative experiments, effectively demonstrating the effectiveness of this invention in spatial scale alignment.

[0140] Experiment 2: A quantitative comparison of the reconstruction results of seven different methods was conducted to demonstrate the effectiveness of the present invention.

[0141] In this experiment, we used five commonly used quantitative image quality assessment metrics (IQAs), including signal-to-noise ratio (SNR), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), mean squared error (MSE), and mean absolute error (MAE), to quantitatively evaluate the reconstruction results of different methods. Generally, higher SNR, PSNR, and SSIM values, and lower MSE and MAE values, indicate better results.

[0142] Table 1 summarizes the values ​​of five commonly used image quality evaluation metrics for the reconstruction results of different reconstruction methods in ASI aurora image sequences. Among them, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Mean Absolute Error (MAE) achieved the best results in the method proposed in this invention.

[0143] Table 1 compares the objective performance of different reconstruction methods on ASI aurora image sequences.

[0144]

[0145] Table 2 summarizes the objective evaluation results of different reconstruction methods for ASI aurora image sequences with different aurora intensities (including high-intensity and low-intensity aurora images). The table shows the values ​​of five commonly used quantitative image quality evaluation indicators for the reconstruction results of the present invention and six different methods under high-intensity and low-intensity aurora images. Among the different methods, the proposed method achieved the best results in signal-to-noise ratio (SNR), structural similarity (SSIM), mean squared error (MSE), and mean absolute error (MAE) for the reconstruction results under high-intensity aurora images; and the proposed method also achieved the best results in peak signal-to-noise ratio (PSNR), mean squared error (MSE), and mean absolute error (MAE) for the reconstruction results under low-intensity aurora images.

[0146] Table 2. Quantitative comparison results of different reconstruction methods in aurora images.

[0147]

[0148] The table also shows that the +NLM method is not significantly different from the method presented in this paper. The main difference lies in the weighting of similar blocks within the reconstruction framework; the rest of the algorithms are the same. Tables 1 and 2 also clearly demonstrate that while our algorithm is not entirely superior to other algorithms, it still offers certain advantages. This clearly indicates that the method proposed in this invention has advantages in spatial scale alignment of aurora images with temporal continuity, exhibiting better reconstruction capabilities.

[0149] Experiment 3: Multi-arc Aurora Image Segmentation

[0150] This experiment applies the proposed method to the processing of ASI aurora images and verifies its practicality in a segmentation experiment. To study the orientation and number of aurora arcs, ASI aurora image segmentation methods are commonly used for aurora arc detection. We use a U-Net network to achieve semantic segmentation of the aurora. The network output is a probability map, representing the probability that a pixel belongs to an aurora. Since there are contact pixels between multiple aurora arcs, it is intuitively difficult to completely segment each arc. Our proposed method, which magnifies the contact pixels, effectively solves this problem. Furthermore, the morphology of the aurora is extremely complex; by dynamically fusing information from two consecutive image frames, multiple aurora arcs can be better segmented.

[0151] Using the method proposed in this invention, results of the original size (440×440) and doubled size (880×880) were obtained. Then, the U-Net network was used to obtain probability maps from the original and reconstructed images, respectively, as shown below. Figure 6 The second and third rows are shown. Finally, to more intuitively demonstrate the effect of the segmentation experiment, we binarize the dual-size segmentation results (threshold set to 0.5), obtaining the binarized probability map from the reconstructed image, as shown below. Figure 6 The last line shows that the boundary information is enhanced and the segmented aurora arcs have better integrity, which again proves that the polarization embedding multi-frame fusion model can incorporate potential aurora motion information.

Claims

1. A method for spatial scale alignment of aurora images based on low-rank epipolar space, characterized in that, The steps are as follows: Step 1: Input a set of all-sky aurora images, preprocess each frame, and the sequence of images obtained after preprocessing the aurora image sequence to be monitored is as follows: X = {f(x,y,t), x,y∈Ω,t=1,2,…T}, where Ω represents the circular mask region of the all-sky aurora image, T represents the length of the aurora image sequence, and x,y represent the coordinates of a point at a certain time t in the image. In the preprocessing, the largest inscribed circle is used as a mask to crop the size of the original image without losing main image information. Then, the excess area outside the mask is removed after cropping. Step 2: Based on the intensity of aurora emission along longitude, starting from the weakest meridian, the aurora image is mapped to polar coordinates. This means that any point in the image is transformed from Cartesian coordinates X(x,y) to polar coordinates Z(ρ,θ) to achieve a polarized representation of the entire sky view. For the sequence of images obtained after mask preprocessing, adjacent frames of the target image sequence are accessed in the order of adjacent frames. Using the same longitude starting point, adjacent frames are also transformed to polar coordinates and mapped to polar space to merge the information of consecutive frames in the aurora sequence. Therefore, the aurora image reconstruction model is represented as follows: Step 3: Using the result after polar coordinate transformation as input, divide each frame of the image into overlapping sub-blocks. The spatial size and sliding window step size of each sub-block are the same. Explore the spatiotemporal continuity and polarization characteristics of aurora images in the form of image blocks. For each block, search and select three types of non-local similar blocks from the local window and neighboring windows surrounding the block: spatial local neighborhood blocks, spatiotemporal local neighborhood blocks, and spatiotemporal non-local neighborhood blocks. Step 4: Define an operator ξ k The algorithm searches and selects different numbers of similar blocks from the three types of reference blocks mentioned above. After dynamically selecting reference blocks from multiple images, it obtains the block with the highest similarity from the three neighborhoods based on Euclidean distance. It performs non-local self-similarity between different neighborhood blocks of the image and estimates the similarity between pixels by measuring the similarity between blocks. Step 5: Employ a progressive nonlocal averaging algorithm to perform weighted fusion on similar blocks to estimate the pixel values ​​to be processed. The weights ω(x,y) are calculated from the similarity between two image blocks. The filtering parameter h is the same for the first iteration, and the weights from the first iteration are used as the filtering parameters for the second iteration. After the above operations, the aurora image reconstruction model can be expressed as: Step 6: After the above processing, the objective function model of the image is used to calculate the low-rank matrix recovery using the continuous SVD low-rank decomposition algorithm, introducing a redundant variable M. k (k) is used to simulate each dimension of the image, so that the reconstruction model of the objective function of the processed image can be represented as a new optimization problem, namely: Step 7: Use the ADMM algorithm, which decomposes the cost function of the original problem into multiple solvable subproblems, solves each subproblem in parallel, and finally adjusts the solutions of each subproblem to obtain the global solution. Solve the above optimization model and obtain the final reconstructed image to achieve spatial scale alignment of aurora images at different scales.

2. The aurora image spatial scale alignment method based on low-rank piriform space according to claim 1, characterized in that, Step 2 involves performing a polar coordinate transformation on the cropped image to convert it to a polar space, including the following steps: 2a) For the distribution area of ​​the aurora arc in the target image, we propose an adaptive polarity transformation strategy; based on the intensity of aurora emission along longitude, we take the weakest meridian as the starting point and use it as the starting angle for polar coordinate transformation, so that a complete aurora arc can be obtained after transformation. 2b) Perform polar coordinate mapping on the aurora image. If the Cartesian coordinates of any point in an image are represented as X(x,y), then the polar coordinates obtained by the following transformation are Z(ρ,θ): Where (x0, y0) are the coordinates of the center point of the image transformation, ρ and θ correspond to the polar radius and polar angle in polar coordinates, respectively, and then the image Z after polar coordinate transformation is used as the input for the next step; 2c) For the sequence of images obtained after mask preprocessing, the adjacent frames of the target image sequence are accessed in the order of adjacent frames. Using the same longitude starting point, the adjacent frames are also subjected to polar coordinate transformation using a unified polar transformation rule and mapped to polarization space to merge the information of the preceding and following frames of the aurora sequence. 2d) Based on the degradation model and reconstruction framework of the aurora image described in step one, the reconstruction model can now be rewritten as: Y = DSZ + N Here, F(Z) represents low-rank regularization related to polar coordinates, which is usually used to remove useless background information, change the spatial relationship of auroral arcs, and enhance the zenith area.

3. The method for spatial scale alignment of aurora images based on low-rank piriform space according to claim 1, characterized in that, In step 3, each frame of the image is divided into overlapping sub-blocks. For each block, non-locally similar blocks are searched from the local window and neighboring windows surrounding that block; this is performed as follows: 3a) First, select r consecutive frames of images, each image Z∈R m×n It is divided into overlapping sub-blocks with a spatial size of o. Multiple overlapping blocks; it is worth noting that dividing the image into overlapping image blocks can reduce artifacts in the reconstructed image; the spatial size of each sub-block is q×q, and the sliding step size is s; 3b) Consider a square target block Z (θ,ρ) ={z(m+g m ,n+g n )}, where (m,n) is the position of the target pixel (block center), g m and g n This is the square spatial offset relative to the target pixel; 3c) For the target block, non-locally similar blocks are searched and selected by overlapping scans in a spatiotemporally continuous polar space from the local window and adjacent windows surrounding the block. There are three types of reference blocks: spatial local neighborhood blocks, spatiotemporally local neighborhood blocks, and spatiotemporally non-local neighborhood blocks. 3d) All center blocks and reference blocks together form a polarization subspace; we use nonlocal techniques to select relevant blocks, subdividing the dynamic polarization subspace into two smaller subspaces; Then, spatiotemporal similarity blocks and spatial similarity blocks are selected and combined to obtain the final polarization subspace.

4. The method for spatial scale alignment of aurora images based on low-rank piriform space according to claim 1, characterized in that, In step 4, a search is conducted among three types of reference blocks, and different numbers of similar blocks are selected, defining an operator ξ. k Condition X is satisfied. k,r =ξ k X r To extract non-locally similar overlapping blocks, we use the following method: k is the number of similar blocks in each image; the block with the highest similarity is obtained from three neighborhoods based on Euclidean distance; non-local self-similarity is performed between different neighborhood blocks of the image, and the similarity between pixels is estimated by measuring the similarity between blocks; the Euclidean distance formula is as follows: in, and Represents different blocks at locations on the image, u and v are the indices of the matrix, and t represents multiple images with spatiotemporal continuity.

5. The method for spatial scale alignment of aurora images based on low-rank piriform space according to claim 1, characterized in that, The preprocessing of the original image consists of five steps: (1) Subtract the dark current; (2) Remove edge noise; (3) Grayscale stretching; (4) Image rotation; (5) Image cropping.

6. The method for spatial scale alignment of aurora images based on low-rank piriform space according to claim 1, characterized in that, In the preprocessing of the original image, the reconstruction of the high-resolution image sequence of a general degradation model can be represented by the following observation model: Y = DSX + N. Low-rank regularization is used to realize the scale change of the image, so the reconstruction model of the aurora image is represented as follows: The first term represents the fidelity of Gaussian noise, the second term is the low-rank constraint term, and F(X) is the regularization parameter. The performance of the reconstruction model depends heavily on the choice of regularization. F(X) is the regularization parameter used to combine these two terms to balance their contributions to the final result.

Images