Small-scale auroral structure positioning and measurement method for full-sky view

Abstract

The present invention discloses a method for locating and measuring small-scale auroral structures for full-sky vision, which mainly solves the technical problem that the diverse appearance features and complex motion patterns of auroral images lead to insufficient acquisition of key features. The solution includes: 1) extracting features from the input original image to obtain a feature map of the original image; 2) dividing the feature map into blocks to obtain a columnar feature map; 3) performing average pooling operations on the columnar feature map in space and channels to obtain a mean, and arranging them according to the position of the corresponding columnar feature map in the original image to obtain a new feature map; 4) performing bilinear interpolation on the new feature map to restore it to the size of the original feature map in space to obtain weights; 5) multiplying the obtained weights and the original feature map to obtain an enhanced feature representation, thereby realizing end-to-end target detection. The present invention realizes efficient and accurate positioning and measurement of small-scale structures in full-sky auroral images.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology and relates to the positioning and measurement of small-scale auroral structures. Specifically, it is a method for positioning and measuring small-scale auroral structures with a full sky field of view, which can be used for image classification, image super-resolution, target detection, etc. Background Art

[0002] In the field of deep learning, the attention mechanism is a method inspired by human vision and cognition that allows neural networks to focus on relevant parts of input data. Initially, the attention mechanism was used as part of the encoder-decoder framework in recurrent neural networks (RNNs) to encode long input sentences. It has since been widely used in various neural network models, particularly in the RNN field. The core idea of ​​the attention mechanism is to enable the model to automatically learn and selectively focus on important information in the input data. This ability enables the neural network to dynamically learn and weigh the importance of different parts of the input data, thereby better capturing the connections and context between the data, thereby improving the model's performance and generalization ability.

[0003] In the field of natural language processing, attention mechanisms are widely used in tasks such as machine translation and text summarization. In machine translation, the introduction of attention mechanisms enables models to better understand different parts of the input sentence and selectively translate the target language, thereby improving translation accuracy and fluency. In text summarization tasks, attention mechanisms help models focus on the most important content in the input text, generating more informative and coherent summaries. Attention mechanisms also play a vital role in computer vision. In image processing tasks, attention mechanisms enable models to automatically focus on the most important areas of an image, improving task accuracy. In object detection tasks, attention mechanisms help models locate and identify objects in images, achieving more accurate object detection and localization. Overall, the introduction of attention mechanisms has significantly improved the performance and robustness of deep learning models across a wide range of tasks. They enable models to better handle complex real-world data and dynamically learn and adapt to the requirements of different tasks, leading to significant progress in deep learning in fields such as natural language processing and computer vision. Research and application of attention mechanisms will continue to drive the development of deep learning technology, providing powerful tools and methods for solving more complex real-world problems.

[0004] In previous studies, many different attention mechanisms have been proposed, the more common ones are spatial attention and channel attention.

[0005] Spatial attention (SA) can be viewed as an adaptive spatial region selection mechanism. It is an attention mechanism used to calculate the correlation between different spatial locations in a neural network. It improves model performance by emphasizing or weakening information at different locations in the input feature map. Spatial attention can be viewed as an adaptive spatial region selection mechanism. By learning weights for each spatial location, the network can adaptively adjust the importance of different locations, thereby achieving more effective feature representation and information extraction. In 2015, Max Jaderberg et al. proposed a representative spatial attention model, the Spatial Transformer Network (STN). The STN can transform various deformable data in space and automatically capture important regional features. The input information first passes through a 1×1 convolutional layer to fuse cross-channel information. It then passes through a ReLU activation layer and a DepthWise convolutional layer to learn spatial weights for each channel. Finally, a sigmoid layer maps the dynamic range of the spatial weights to [0, 1]. The input information is multiplied pixel by pixel, and features at different locations are weighted and fused according to the learned weights, resulting in an activation feature map that carries spatial information. The core idea of ​​the spatial attention mechanism is to assign attention weights based on the spatial location of the input data. Through spatial attention, the model can learn to selectively focus on important areas in the image, thereby improving the performance of visual tasks.

[0006] Channel attention (CA) is an attention mechanism used to enhance the interaction and importance of features between different channels in a neural network, aiming to improve the network's utilization of different feature channels. This mechanism is commonly used in convolutional neural networks. By learning weights for each channel, the network can adaptively adjust the importance of each channel, thereby focusing on certain feature channels and improving feature representation and generalization performance. In neural networks, different channels in different feature maps typically represent different objects and have varying degrees of importance. A representative model is the Squeeze-and-Excitation Network (SENet), which consists of two parts: compression and excitation. First, each feature map is compressed and integrated using global average pooling or a fully connected layer to obtain a real number that reflects global information. Next, an activation function (such as ReLU) is used to process the global representation to learn the weight or importance of each channel, amplifying the weights of more important feature maps and reducing those of less important ones. The learned channel weights are then applied to the corresponding channels to perform a weighted fusion of the channel features. This step can be achieved through multiplication, where the features of each channel are multiplied by the corresponding weight. Finally, the weighted features are recombined to obtain the feature representation adjusted by the channel attention mechanism.

[0007] Due to the complex content and random motion patterns of auroral images, many smaller, critical local information is obscured by other brightness patterns during auroral motion. Traditional attention mechanisms, when used with these datasets, struggle to capture and integrate detailed details and features. Summary of the Invention

[0008] The present invention proposes a small-scale auroral structure positioning and measurement method for a full-sky field of view, which is mainly used to solve the technical problem of insufficient acquisition of key features due to the diverse appearance characteristics and complex motion patterns of auroral images.

[0009] The technical idea of ​​implementing the present invention is: firstly, extract the features of the input original picture to obtain the feature map of the original image; secondly, divide the obtained feature map into blocks to obtain blocks of size H M ×W M ×C DThe method is to generate a columnar feature map; then perform an Average pooling operation on the obtained columnar feature map in space and channels to obtain the mean of each columnar feature map, and arrange the means according to the position of the corresponding columnar feature map in the original image to obtain a new feature map; perform bilinear interpolation on the obtained new feature map to restore it to the size of the original feature map in space and obtain the weight; finally, multiply the obtained weight by the original feature map to obtain an enhanced feature representation, and use the enhanced feature for various downstream tasks. The present invention can ignore the random distribution problem of aurora in space, improve the network's ability to represent aurora features, and enable it to focus on more critical semantic information in the image.

[0010] The present invention achieves the above-mentioned purpose by the following specific steps:

[0011] (1) Input an aurora image taken by a full-sky lens with an effective field of view of 360°×180°. The shallow feature map of the full-sky aurora image is obtained through the shallow feature extraction layer, and the full-sky field of view space is divided into K regions of size H. M ×W M Then, on the channel, the region block is further divided into D sub-blocks. According to this spatial division rule, K×D columnar feature maps are obtained, where each columnar feature map corresponds to an area in the shallow feature map, and the size of each columnar feature map is H. M ×W M ×C D , where H M 、W M 、C D are the height, width, and number of channels of the columnar feature map respectively; then, a pooling operation is performed on each columnar feature to obtain the regional-level quantized attention value;

[0012] (2) Arrange the regional-level quantized attention values ​​according to the full sky field of view spatial division rule and restore them to the center position of the original full sky area. The size of the restored feature map is Where P = H / H M =W / W M , D=C / C D , where H, W, and C correspond to the height, width, and channels of the shallow features, respectively. P is the number of blocks divided by height and width on the feature map, and D is the number of blocks divided on the channel. The bilinear interpolation method is used to generate a reconstructed feature map with the same size as the original full sky image in space, but the channel is still D. That is, the smooth attention matrix related to the neighborhood information is reconstructed to obtain the smooth attention matrix.

[0013] (3) Using the smooth attention matrix, the shallow feature map is multiplied pixel by pixel in the whole sky domain to obtain the fused feature map after spatial enhancement and channel screening;

[0014] (4) Add the fused feature map to the shallow feature map to complete the residual learning of the potential significant features and enhance the representation ability of the deep feature extraction network;

[0015] (5) Based on the deep feature extraction layer, the detailed information of the auroral features of the entire sky domain is amplified, and the fuzzy representation of the deformation features is achieved; then the entire sky field of view is divided, and the presence of small-scale auroral structures is predicted on each grid to achieve end-to-end target detection.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] First, since the present invention adopts a columnar feature map partitioning strategy, global information is decomposed into local details, so that the neural network can focus on the more subtle content within the aurora image.

[0018] Second, since the present invention uses the Average pooling method to compress each columnar feature map into a one-dimensional tensor that can reflect the columnar feature map, it effectively reduces information redundancy and enhances the robustness and representativeness of the features.

[0019] Third, since the present invention adopts a bilinear interpolation method, the reconstructed feature map not only considers the values ​​of the two pixels closest to the target pixel, but also considers the values ​​of their adjacent pixels, thereby better maintaining the spatial continuity of the image, reducing the blurred or jagged edges that appear in the upsampled image, and producing smoother, higher-quality images.

[0020] Fourth, since the present invention performs weighted fusion of quantized features and shallow features, the neural network can dynamically learn weights, thereby enhancing features that are important to the current task and suppressing features that are not important to the current task. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart for realizing the method of the present invention;

[0022] Figure 2 Schematic diagram of the basic framework of the attention mechanism in the present invention;

[0023] Figure 3 Schematic diagram of the relationship between the observation target and the observation station in the present invention;

[0024] Figure 4This is a visualization result diagram of the present invention and other attention methods on the aurora image target detection task, where a is the detection result under the present invention, b is the detection result under the channel attention mechanism CA, c is the detection result under the spatial attention parallel channel attention CA+SA, d is the detection result under the convolutional attention mechanism CBAM, e is the detection result under the spatial attention SA, and f is the detection result under the Baseline model;

[0025] Figure 5 It is a heat map of the present invention and other attention methods in the task of aurora image target detection; where a is the heat map under the present invention, b is the heat map under the channel attention mechanism CA, c is the heat map under the spatial attention parallel channel attention CA+SA, d is the heat map under the convolutional attention mechanism CBAM, e is the heat map under the spatial attention SA, and f is the heat map under the Baseline model. Specific implementation methods

[0026] The present invention will be further described below with reference to the accompanying drawings.

[0027] Example 1: Refer to the attached Figure 1 The present invention proposes a method for locating and measuring small-scale auroral structures with a full-sky field of view, comprising the following steps:

[0028] Step 1. Input an aurora image taken by a full-sky lens with an effective field of view of 360°×180°. Obtain the shallow feature map of the full-sky aurora image through the shallow feature extraction layer, and divide the full-sky field of view space into K blocks of size H. M ×W M Then, on the channel, the region block is further divided into D sub-blocks. According to this spatial division rule, K×D columnar feature maps are obtained, where each columnar feature map corresponds to an area in the shallow feature map, and the size of each columnar feature map is H. M ×W M ×C D , where H M 、W M 、C D are the height, width, and number of channels of the columnar feature map respectively; then, a pooling operation is performed on each columnar feature to obtain the regional-level quantized attention value.

[0029] The region division and pooling operations of this embodiment are performed according to the following steps:

[0030] (1.1) Intensity scaling of the original image is performed to make the aurora visible, and all ASI images are preprocessed, including dark current subtraction, masking, and cropping. The preprocessed aurora images are then channel expanded.

[0031] (1.2) Perform shallow feature extraction on the image obtained in step (1.1) to obtain shallow features X, divide it into regions, and obtain columnar feature maps of different regions. Set the size of each columnar feature map to H M ×W M ×C D ;

[0032] (1.3) Perform an average pooling operation on the obtained columnar feature map to convert a columnar feature map into a value that reflects the importance of local columnar information, that is, the regional level quantized attention value. The implementation is as follows:

[0033]

[0034] Among them, M i Represents the attention value of the region-level quantization of the i-th columnar feature map, i = 1, 2, 3, ..., K × D.

[0035] Step 2. Arrange the regional-level quantized attention values ​​according to the full sky field of view spatial division rules and restore them to the center position of the original full sky area. The size of the restored feature map is Where P = H / H M =W / W M , D=C / C D , where H, W, and C correspond to the height, width, and channels of the shallow features, respectively. P is the number of blocks divided according to height and width on the feature map, and D is the number of blocks divided on the channel. The bilinear interpolation method is used to generate a reconstructed feature map with the same size as the original full sky image in space, but the channel is still D. That is, the smooth attention matrix related to the neighborhood information is reconstructed to obtain the smooth attention matrix. The specific implementation steps are as follows:

[0036] (2.1) Rearrange each attention value according to the position of its columnar feature in the shallow feature X, and restore it to the center position of the original full sky area. The restored feature size is

[0037] (2.2) Use the bilinear interpolation method to reconstruct the restored features and generate a smooth attention matrix F' that is equal to the size of the shallow feature X in space, and the size of F' is equal to F in the channel:

[0038] F′=F inter (F)

[0039] in F inter (·) indicates that · is reconstructed using the bilinear interpolation method.

[0040] Step 3. Use the smooth attention matrix to perform pixel-by-pixel multiplication on the shallow feature map in the entire sky domain to obtain a fused feature map after spatial enhancement and channel screening; in this embodiment, the fused feature map is obtained according to the following steps:

[0041] (3.1) Divide the shallow features X into D groups on the channel, and the size of each group of features is H×W×C D ;

[0042] (3.2) Divide the generated features F' into D groups on the channel, and the size of each group of features is H×W×1;

[0043] (3.3) Perform dot multiplication on each pixel feature of each set of generated features and its corresponding set of shallow features, as shown in the following formula:

[0044]

[0045] Where m = 1, 2, ..., D;

[0046] (3.4) Cascade the dot product results on the channel to obtain the enhanced features As shown in the following formula:

[0047] X′={cat(X′ m )|m=1,2,3...D}

[0048] Among them, cat(·) represents the concatenation operation.

[0049] Step 4. Add the fused feature map to the shallow feature map to complete the residual learning of the potential significant features and enhance the representation ability of the deep feature extraction network; this is achieved according to the following formula:

[0050]

[0051] Among them, X′ represents the fused features, X represents the shallow features, Represents a pixel-by-pixel addition operation.

[0052] Step 5. Based on the deep feature extraction layer, the full-sky auroral feature details are magnified, and the deformation features are fuzzy represented. The full-sky field of view is then divided into grids, and the presence of small-scale auroral structures is predicted on each grid, achieving end-to-end target detection. The specific steps are as follows:

[0053] (5.1) The deep feature extraction layer amplifies the information contained in the small-scale auroral structure relative to other regions and blurs the deformation features;

[0054] (5.2) The full sky field of view is divided into grids. The size of each grid is determined according to the size of the small-scale auroral structure to be detected. Within each grid, the small-scale auroral structure is detected based on the features enhanced by the neural network, as shown in the following formula:

[0055]

[0056] Among them, x t It is a small-scale auroral structure, t represents the t-th grid. When a small-scale auroral structure is detected in the grid, it is represented as 1, otherwise it is 0, completing end-to-end target detection;

[0057] (5.3) Based on the actual geographical location, when conducting joint observation and analysis of the aurora with other equipment, it is projected into geomagnetic coordinates for comparison. The geocentric angle γ = θ - α, where α is the angle between the line of sight and the normal to the ionospheric aurora luminous surface. Using the sine theorem, we can obtain:

[0058] sinγ / R E = sinθ / (R E +h)

[0059] The relationship between the distance d from any pixel to the zenith and the geocentric angle γ and its corresponding zenith angle θ is obtained as follows:

[0060]

[0061] d=(R E +h)γ

[0062] Among them, R E represents the radius of the Earth, and h is the height of the aurora.

[0063] Example 2: Refer to the attached Figure 1 The overall implementation steps of this embodiment are as follows: Example 1, which uses YOLOv5s as the backbone network and the selected dataset is the Aurora dataset. After preprocessing the Aurora dataset, a specific example of using it for small-scale target detection tasks is given, and the implementation process of the present invention is further described in detail:

[0064] Step 1: Preprocess the input image, then go through a series of convolutions to obtain shallow features, and perform region division and pooling operations on the shallow features.

[0065] 1.1) To make the auroras visible in the raw images, the intensity was linearly stretched from 0 to a threshold (the dynamic range of the raw images was 0-17,000) to 0-255. The image bars are at the bottom, with a Rayleigh range of 0-4,000 Rayleighs at 557.7 and 630.0 nm, and a Rayleigh range of 0-300 Rayleighs at 427.8 nm. In addition to intensity scaling, all ASI images require additional preprocessing: dark current subtraction, masking, and cropping, followed by channel expansion of the processed auroral images.

[0066] 1.2) Perform shallow feature extraction on the processed image to obtain shallow features X, divide the shallow features X into regions, and obtain columnar feature maps of different regions. Set the size of each columnar feature map to H M ×W M ×C D (Set according to actual situation), where H M , W M , C D are the height, width and number of channels of the columnar feature map, respectively. In this experiment, the size of the shallow feature X is 8×8×512, and the size of each columnar feature is 2×2×16.

[0067] 1.3) Perform an average pooling operation on the obtained columnar feature map to convert a columnar feature map into a value that can reflect the importance of local columnar information, according to the following formula:

[0068]

[0069] At this point, a batch of region-level quantized attention values ​​are obtained.

[0070] Step 2: Sort and reconstruct the region-level quantized attention values, as follows:

[0071] 2.1) Each columnar feature map has its own unique position information relative to other columnar feature maps. In order to facilitate the subsequent acquisition of more coherent features, each attention value is reordered according to the position of its columnar feature in the shallow feature X and restored to the center of the original full sky area. The restored feature size is In this embodiment, the restored feature size is 4×4×32.

[0072] 2.2) Calculate the total number of columnar feature maps. In this example, there are 512 columnar feature maps of the same size. The calculation formula is as follows:

[0073]

[0074] 2.3) Use bilinear interpolation to reconstruct the restored features and generate a feature F' that is equal to the shallow feature X in space. However, it is worth noting that the size of F' is equal to F in the channel. This step is performed according to the following formula:

[0075] F′=F inter (F)

[0076] In this embodiment, the reconstructed feature size is 8×8×32.

[0077] Step 3: Multiply each pixel feature of the original full sky view point by point using the smooth attention matrix. Follow these steps:

[0078] 3.1) Divide the shallow features X into D groups on the channel, and the size of each group of features is H×W×C D ;

[0079] 3.2) Divide the generated features F' into D groups on the channel, and the size of each group of features is H×W×1;

[0080] 3.3) Perform dot multiplication of each set of generated features and each pixel feature of each corresponding set of shallow features X, as shown in the following formula:

[0081]

[0082] Where i represents the i-th group of features of the shallow feature X and the generated feature F';

[0083] 3.4) X after dot multiplication i 'Cascade on the channel to obtain enhanced features As shown in the following formula:

[0084]

[0085] Step 4: Add the enhanced features. The operation is as follows:

[0086] The enhanced feature X' is added pixel by pixel to the shallow feature X, as shown in the following formula:

[0087]

[0088] Through this step, the fused feature map can have richer information and stronger representation ability, ensuring that important information in the original image is not lost, while enhancing the filtered and enhanced feature information, such as [V1, V2, V3, V4...V n ]The bold part is the enhanced part.

[0089] Step 5: Locate and measure the obtained features as follows:

[0090] 5.1) Through the deep feature extraction layer, the information contained in small-scale auroral structures is amplified relative to other regions, making the small-scale auroral structures more prominent, and the deformation features are blurred to reduce their interference with target detection accuracy;

[0091] 5.2) Divide the full sky field of view into multiple small grids. The size of each grid is determined by the size of the small-scale auroral structure to be detected. Within each grid, the small-scale auroral structure is detected based on the enhanced features, as follows:

[0092]

[0093] where x i is the small-scale auroral structure, i is the i-th grid, when a small-scale auroral structure is detected in the grid, it is represented as 1, otherwise it is 0, thus completing the end-to-end target detection;

[0094] 5.3) Based on the actual geographical location of the Yellow River Station, when conducting joint observations and analysis of the aurora with other equipment, it is necessary to project the observations into geomagnetic coordinates for comparison. Since the solid angle of each pixel is the same, but the geocentric angle γ is different, it can be expressed as γ = θ - α, where α is the angle between the line of sight and the normal to the ionospheric aurora luminous surface. Using the sine theorem, we can obtain:

[0095] sinγ / R E = sinθ / (R E +h)

[0096] Therefore, the relationship between the distance d from any pixel to the zenith and the geocentric angle γ and its corresponding zenith angle θ can be obtained.

[0097]

[0098] d=(R E +h)γ

[0099] 5.4) In practice, the geocentric angle increases as the zenith angle increases. Therefore, the distance between pixels does not satisfy a linear relationship, but a nonlinear mapping relationship, so the distance between pixels also increases.

[0100] The effects of the present invention will be further described below in conjunction with simulation experiments.

[0101] To fully validate the effectiveness of our invention, we conducted a series of in-depth experiments, comparing it in detail with classic attention mechanisms. These include spatial attention (SA), channel attention (CA), spatial attention plus channel attention (CA+SA), and the convolutional attention mechanism (CBAM). We conducted a comprehensive evaluation of these different mechanisms, aiming to reveal the unique advantages of our invention and its contribution to improving model performance.

[0102] During the experiments, we carefully selected a set of parameter settings to ensure the reliability and comparability of the experimental results. Specifically, we set the learning rate to 0.01, which is to maintain a moderate parameter update speed during training to avoid overly fast or slow convergence problems. The batch size is set to 16. This choice takes into account the limitations of computing resources and the stability of model training, while also balancing training speed and memory consumption. The momentum is 0.937. Finally, we choose 300 training epochs. This number ensures that the model is fully trained while avoiding the risk of overfitting.

[0103] Through these meticulously designed experiments and parameter settings, we aim to fully demonstrate the superiority of our proposed method over classic approaches. We will carefully analyze the experimental results, delving into the performance differences between different mechanisms and exploring the unique contributions of our proposed method to improving model performance. These experimental data will provide valuable reference and guidance for further understanding and application of attention mechanisms, helping to advance research and application in related fields.

[0104] Experiment 1: Comparison with other methods to verify the effectiveness of the present invention

[0105] In order to ensure the effectiveness of the experiment, we chose to conduct the experiment under the same network framework, that is, the experiment was conducted under the YOLOv5s network architecture, and all parameters were selected the same. The present invention uses the full-sky image from the Yellow River Station (YRS) in Yalesund, Svalbard Islands to divide the dayside aurora into four active areas. The geographical coordinates of YRS are located at 78.92°N, 11.93°E. The YRS optical system consists of three full-sky cameras, which have been developed since December 2003 to measure light emissions at 427.8, 557.7 and 630.0nm. In the experiment, the aurora images can be divided into 4 categories, namely, the dayside aurora is divided into arc aurora, drapey corona aurora, radial corona aurora and hot-spot aurora. Our comparison results are shown in Table 1:

[0106] Table 1 Comparison results

[0107]

[0108] We compare our proposed method for localizing and measuring small-scale auroral structures with full-sky view to improve representation capabilities with channel attention (CA) and spatial attention (SA), spatial attention plus channel attention (CA+SA), baseline, and convolutional attention mechanism (CBAM), demonstrating the advantages of our proposed block attention for aurora image feature extraction.

[0109] After rigorous experimental comparisons, as shown in Table 1, this method demonstrates significant advantages in extracting features from aurora images. Compared to channel attention (CA), this method, through its unique design, is able to more accurately capture key information in the image, thereby improving accuracy by 0.8%. Compared to spatial attention (SA), this method not only focuses on the spatial location information of the image but also combines local and global contextual information, achieving a 0.6% improvement in accuracy.

[0110] Compared to the baseline model, this method significantly enhances the model's representational capabilities, making it more efficient and accurate in extracting auroral information, achieving a 1% improvement in accuracy. Compared to the convolutional attention mechanism (CBAM), this method further enhances the model's representational capabilities while maintaining efficiency, achieving a 0.4% improvement in accuracy.

[0111] Finally, compared to the combination of channel attention and spatial attention in parallel (CA+SA), this invention, with its unique advantages, not only outperforms a single attention mechanism, but also demonstrates superior performance when multiple attention mechanisms are combined, achieving a 0.8% accuracy improvement. This result fully demonstrates the advantages and potential of this invention in the task of extracting features from aurora images.

[0112] Figure 4This is a visualization of the results of six methods, where a is the detection result under our method, b is the detection result under the channel attention mechanism (CA), c is the detection result under the spatial attention parallel channel attention (CA+SA), d is the detection result under the convolutional attention mechanism (CBAM), e is the detection result under the spatial attention (SA), and f is the detection result under the baseline model. By observing the detection results of each model, it can be found that compared with other attention mechanisms, our method has a more refined attention allocation capability and can accurately and efficiently focus the model's attention on local micro-regions in the image. This concentration not only enhances the model's ability to capture subtle and key features, but also greatly enhances the depth and accuracy of the description of these local features.

[0113] Figure 5 It is also a heat map visualization result of six methods, where a is the heat map under the present invention, b is the heat map under the channel attention mechanism (CA), c is the heat map under spatial attention parallel channel attention (CA+SA), d is the heat map under the convolutional attention mechanism (CBAM), e is the heat map under spatial attention (SA), and f is the heat map under the Baseline model.

[0114] Experiment 2: Ablation experiment to verify the residual

[0115] To comprehensively evaluate the effectiveness of our proposed method on the aurora object detection task, we conducted a series of in-depth ablation experiments. The primary purpose of these experiments is to accurately measure the independent performance of each component in the model and its contribution to the overall performance. By comparing the performance of models under different configurations, we gain a clearer understanding of how each module works together to optimize the final detection results.

[0116] We introduced residual connections to further analyze their role in model training stability and performance improvement.

[0117] After introducing residual connections, we observed that the model was more stable during training and was able to converge to a better solution more quickly. This is because residual connections allow the model to skip certain layers, thus avoiding the problem of vanishing or exploding gradients, making it easier to train deep networks.

[0118] Table 2 Impact of residual connections

[0119]

[0120] Experiment 3: Ablation experiment, the impact of the number of channels

[0121] In order to explore the impact of the number of channels on experimental accuracy, we conducted a third experiment by setting different numbers of channels to study the impact on detection accuracy. The experimental results are shown in Table 3.

[0122] We performed various operations on shallow feature maps along the channel dimension, including no partitioning, partitioning into 2, 4, 8, 16, and 32 blocks. Our results show that the impact of the number of channel partitions on model performance exhibits a complex trend, deviating from a direct linear relationship. Partitioning into 16 blocks yields the best results, potentially suggesting that a moderate channel partitioning can improve detection accuracy. These results highlight the importance of adjusting the number of channel partitions to optimize model performance based on specific application scenarios and datasets. Therefore, in practical applications, it is crucial to determine the optimal channel partitioning strategy for the task at hand through experimentation and evaluation.

[0123] Table 3 Influence of the number of channels

[0124]

[0125] Experiment 4: Ablation experiment, the impact of the number of spatial blocks

[0126] In order to explore the impact of the number of spatial blocks on experimental accuracy, we conducted a third experiment by setting different numbers of spatial blocks to study the impact on detection accuracy. The experimental results are shown in Table 3.

[0127] We partitioned the shallow feature maps in the spatial dimension into blocks of no partitioning, two blocks, and four blocks. Our results show that the impact of the number of spatial partitions on model performance exhibits a complex trend, deviating from a direct linear relationship. This may indicate that a moderate spatial partitioning strategy can improve detection accuracy. These results highlight the importance of adjusting the number of channel partitions to optimize model performance based on specific application scenarios and datasets. Therefore, in practical applications, it is crucial to determine the optimal spatial partitioning strategy for the task at hand through experimentation and evaluation.

[0128] Table 4 The impact of the number of spatial blocks

[0129]

[0130] Experiment 5: Ablation experiment, the impact of different upsampling methods

[0131] In the field of image processing, bilinear interpolation has shown significant advantages over other interpolation methods, such as nearest neighbor interpolation, especially in the pursuit of image smoothness and quality.

[0132] The core advantage of bilinear interpolation is that it considers the influence of the four neighboring pixels surrounding the interpolated point. By performing linear interpolation between these four points, the pixel values ​​along a transition line are first calculated in one direction (usually horizontally or vertically), and then a similar interpolation is performed in the other direction to finally obtain the value of the target pixel. This bidirectional linear interpolation method is not only relatively simple to calculate, but also effectively captures the gradual change of pixel values ​​in space.

[0133] Compared to nearest neighbor interpolation, bilinear interpolation avoids the "hard jump" of using the nearest neighbor pixel directly, thus reducing the jagged edges commonly seen when enlarging images. By smoothing the transition, it makes the interpolation result more natural, the image edges softer, and improves the overall visual quality of the image.

[0134] Table 5 Impact of different upsampling methods

[0135]

[0136] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.

[0137] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A method for locating and measuring small-scale auroral structures with a full-sky field of view, characterized in that: The steps include: (1) Input an aurora image taken by a full-sky lens with an effective field of view of 360°×180°. The shallow feature map of the full-sky aurora image is obtained through the shallow feature extraction layer, and the full-sky field of view space is divided into K regions of size H. M ×W M Then, on the channel, the region block is further divided into D sub-blocks. According to this spatial division rule, K×D columnar feature maps are obtained, where each columnar feature map corresponds to an area in the shallow feature map, and the size of each columnar feature map is H. M ×W M ×C D , where H M 、W M 、C D are the height, width, and number of channels of the columnar feature map respectively; then, a pooling operation is performed on each columnar feature to obtain the regional-level quantized attention value; (2) Arrange the regional-level quantized attention values ​​according to the full sky field of view spatial division rule and restore them to the center position of the original full sky area. The size of the restored feature map is Where P = H / H M =W / W M , D=C / C D , where H, W, and C correspond to the height, width, and channels of the shallow features, respectively. P is the number of blocks divided by height and width on the feature map, and D is the number of blocks divided on the channel. The bilinear interpolation method is used to generate a reconstructed feature map with the same size as the original full sky image in space, but the channel is still D. That is, the smooth attention matrix related to the neighborhood information is reconstructed to obtain the smooth attention matrix. (3) Using the smooth attention matrix, the shallow feature map is multiplied pixel by pixel in the whole sky domain to obtain the fused feature map after spatial enhancement and channel screening; (4) Add the fused feature map to the shallow feature map to complete the residual learning of the potential significant features and enhance the representation ability of the deep feature extraction network; (5) Based on the deep feature extraction layer, the detailed information of the auroral features of the entire sky domain is amplified, and the fuzzy representation of the deformation features is achieved; then the entire sky field of view is divided, and the presence of small-scale auroral structures is predicted on each grid to achieve end-to-end target detection.

2. The method according to claim 1, wherein: The region division and pooling operation described in step (1) is performed as follows: (1.1) Intensity scaling of the original image is performed to make the aurora visible, and all ASI images are preprocessed, including dark current subtraction, masking, and cropping. The preprocessed aurora images are then channel expanded. (1.2) Perform shallow feature extraction on the image obtained in step (1.1) to obtain shallow features X, divide it into regions, and obtain columnar feature maps of different regions. Set the size of each columnar feature map to H M ×W M ×C D ; (1.3) Perform an average pooling operation on the obtained columnar feature map to convert a columnar feature map into a value that can reflect the importance of local columnar information, that is, the regional-level quantized attention value.

3. The method according to claim 2, wherein: The average pooling operation is performed on the obtained columnar feature map in step (1.3), which is implemented as follows: Among them, M i Represents the attention value of the region-level quantization of the i-th columnar feature map, i = 1, 2, 3, ..., K × D.

4. The method according to claim 1, wherein: The reconstruction of the smooth attention matrix in step (2) is achieved as follows: (2.1) Rearrange each attention value according to the position of its columnar feature in the shallow feature X, and restore it to the center position of the original full sky area. The restored feature size is (2.2) Use the bilinear interpolation method to reconstruct the restored features and generate a smooth attention matrix F' that is equal to the size of the shallow feature X in space, and the size of F' is equal to F in the channel: F′=F inter (F) in F inter (·) indicates that · is reconstructed using the bilinear interpolation method.

5. The method according to claim 4, characterized in that: The fused feature map in step (3) is obtained specifically according to the following steps: (3.1) Divide the shallow features X into D groups on the channel, and the size of each group of features is H×W×C D ; (3.2) Divide the generated features F' into D groups on the channel, and the size of each group of features is H×W×1; (3.3) Perform dot multiplication on each pixel feature of each set of generated features and its corresponding set of shallow features, as shown in the following formula: Where m = 1, 2, ..., D; (3.4) Cascade the dot product results on the channel to obtain the enhanced features As shown in the following formula: X′={cat(X' m )|m=1,2,3...D} Among them, cat(·) represents the concatenation operation.

6. The method according to claim 1, wherein: Step (4) adds the fused feature map to the shallow feature map, which is implemented according to the following formula: Among them, X′ represents the fused features, X represents the shallow features, Represents a pixel-by-pixel addition operation.

7. The method according to claim 1, wherein: The end-to-end target detection in step (5) is specifically achieved by positioning and measuring the obtained features, and the implementation steps are as follows: (5.1) The deep feature extraction layer amplifies the information contained in the small-scale auroral structure relative to other regions and blurs the deformation features; (5.2) The full sky field of view is divided into grids. The size of each grid is determined according to the size of the small-scale auroral structure to be detected. Within each grid, the small-scale auroral structure is detected based on the features enhanced by the neural network, as shown in the following formula: Among them, x t It is a small-scale auroral structure, t represents the t-th grid. When a small-scale auroral structure is detected in the grid, it is represented as 1, otherwise it is 0, completing end-to-end target detection; (5.3) Based on the actual geographical location, when conducting joint observation and analysis of the aurora with other equipment, it is projected into geomagnetic coordinates for comparison. The geocentric angle γ = θ - α, where α is the angle between the line of sight and the normal to the ionospheric aurora luminous surface. Using the sine theorem, we can obtain: sinγ / R E =sinθ / (R E +h) The relationship between the distance d from any pixel to the zenith and the geocentric angle γ and its corresponding zenith angle θ is obtained as follows: d=(R E +h)c Among them, R E represents the radius of the Earth, and h is the height of the aurora.

Images