Small-scale aurora structure positioning and measuring method for all-sky view

Abstract

The invention discloses a small-scale aurora structure positioning and measuring method for an all-sky visual field, and mainly solves the technical problem that key features are insufficiently acquired due to diversified appearance features and complex motion modes of aurora images. According to the scheme, the method comprises the steps of 1) performing feature extraction on an input original image to obtain a feature map of the original image; 2) dividing the feature map into blocks to obtain a columnar feature map; 3) carrying out average pooling operation on the columnar feature maps in space and channels to obtain an average value, and arranging the average value according to the positions of the corresponding columnar feature maps in the original map to obtain a new feature map; 4) bilinear interpolation is carried out on the new feature map, the new feature map is reduced to the size of the original feature map in space, and a weight is obtained; and 5) multiplying the obtained weight by the original feature map to obtain enhanced feature representation, and realizing end-to-end target detection. According to the invention, efficient and accurate positioning and measurement of the small-scale structure in the all-sky aurora image are realized.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology and relates to the positioning and measurement of small-scale aurora structures. Specifically, it is a method for positioning and measuring small-scale aurora structures for a full sky field of view, which can be used for image classification, image super-resolution, target detection, etc. Background Art

[0002] In the field of deep learning, the attention mechanism is a method inspired by the human visual and cognitive system that allows neural networks to focus on relevant parts when processing input data. Initially, the attention mechanism was used as part of the encoder-decoder framework in recurrent neural networks (RNNs) to encode long input sentences, and was later widely used in various neural network models, especially in the RNN field. The core idea of ​​the attention mechanism is to enable the model to automatically learn and selectively focus on important information in the input data. This ability enables the neural network to dynamically learn and weigh the importance of different parts of the input data, so as to better capture the associations and contextual information between the data, thereby improving the performance and generalization ability of the model.

[0003] In the field of natural language processing, attention mechanisms are widely used in tasks such as machine translation and text summarization. In machine translation, by introducing the attention mechanism, the model can better understand the different parts of the input sentence and selectively translate the corresponding target language, thereby improving the accuracy and fluency of the translation. In the text summarization task, the attention mechanism can help the model focus on the most important content in the input text and generate a more informative and coherent summary. In the field of computer vision, the attention mechanism also plays an important role. In image tasks, by introducing the attention mechanism, the model can automatically focus on the most important areas in the image, thereby improving the accuracy of the task. In the object detection task, the attention mechanism can help the model locate and identify objects in the image, thereby achieving more accurate object detection and positioning. In general, the introduction of the attention mechanism has greatly improved the performance and robustness of deep learning models in various tasks. It enables the model to better handle complex real-world data, dynamically learn and adapt to the requirements of different tasks, so that deep learning has made significant progress in natural language processing, computer vision and other fields. The research and application of the attention mechanism will continue to promote the development of deep learning technology and provide powerful tools and methods for solving more complex problems in the real world.

[0004] In previous studies, many different attention mechanisms have been proposed, the more common ones are spatial attention and channel attention.

[0005] Spatial Attention (SA) can be seen as an adaptive spatial region selection mechanism. It is an attention mechanism used to calculate the correlation between different spatial positions in a neural network. It improves the model performance by strengthening or weakening the information at different positions in the input feature map. Spatial attention can be seen as an adaptive spatial region selection. By learning the weight of each spatial position, the network can adaptively adjust the importance of different positions, thereby achieving more effective feature representation and information extraction. In 2015, Max Jaderberg et al. proposed a representative spatial attention model, Spatial Transformer Networks (STN). STN can transform various deformation data in space and automatically capture important regional features. The input information first passes through a 1×1 convolution layer to fuse cross-channel information, and then passes through a ReLU activation layer and a DepthWise convolution layer, and each channel learns its spatial weights separately. Finally, the Sigmoid layer maps the dynamic range of the spatial weight to [0, 1], multiplies the input information pixel by pixel, and performs weighted fusion of features at different positions according to the learned weights, thereby obtaining an activated feature map carrying spatial information. The core idea of ​​the spatial attention mechanism is to assign attention weights according to the spatial location of the input data. Through spatial attention, the model can learn to selectively focus on important areas in the image, thereby improving the performance of visual tasks.

[0006] Channel Attention (CA) is an attention mechanism used to enhance the feature interaction and importance between different channels of a neural network, aiming to improve the network's utilization efficiency of different feature channels. The channel attention mechanism is usually applied to convolutional neural networks. By learning the weight of each channel, the network can adaptively adjust the importance of each channel, so that the neural network can focus on certain feature channels and improve the ability of feature representation and generalization performance. In a neural network, different channels of different feature maps usually represent different objects, and different channels have different importance. Its representative model is the Squeeze-and-Excitation Networks (SENet), which is divided into two parts: compression and excitation. First, each feature map is compressed and integrated through global average pooling or a fully connected layer to obtain a real number that reflects global information. Then, an activation function (such as ReLU) is used to process the global representation to learn the weight or importance of each channel, amplify the weight of more important feature maps, and reduce the weight of unimportant ones. Then, the learned channel weights are applied to the corresponding channels to perform weighted fusion of channel features. This step can be achieved through multiplication operations, that is, multiplying the features of each channel by the corresponding weights. Finally, the weighted features are recombined to obtain the feature representation adjusted by the channel attention mechanism.

[0007] Due to the complex content and random motion patterns of aurora images, many small key local information is submerged by other brightness when the aurora moves. When facing the above datasets, the traditional attention mechanism's comprehensive capture and integration strategy seems to be inadequate and it is difficult to fully capture detailed details and features. Summary of the invention

[0008] The present invention proposes a small-scale auroral structure positioning and measurement method for a full-sky field of view, which is mainly used to solve the technical problem of insufficient acquisition of key features due to the diverse appearance characteristics and complex motion patterns of auroral images.

[0009] The technical idea of ​​implementing the present invention is: firstly, extract the features of the input original image to obtain the feature map of the original image; secondly, divide the obtained feature map into blocks to obtain blocks of size H M ×W M ×C D; then the obtained columnar feature map is subjected to Average pooling operation in space and channel to obtain the mean of each columnar feature map, and the mean is arranged according to the position of the corresponding columnar feature map in the original image to obtain a new feature map; bilinear interpolation is performed on the obtained new feature map to restore it to the original feature map size in space to obtain weights; finally, the obtained weights are multiplied by the original feature map to obtain an enhanced feature representation, and the enhanced features are used for various downstream tasks. The present invention can ignore the random distribution problem of aurora in space, improve the network's ability to represent aurora features, and enable it to focus on more critical semantic information in the image.

[0010] The present invention achieves the above-mentioned purpose by the following specific steps:

[0011] (1) Input an aurora image taken by a full-sky lens with an effective field of view of 360°×180°. The shallow feature map of the full-sky aurora image is obtained through the shallow feature extraction layer, and the full-sky field of view space area is divided into K blocks of size H. M ×W M Then, on the channel, the region block is further divided into D sub-blocks. According to this spatial division rule, K×D columnar feature maps are obtained, where each columnar feature map corresponds to an area in the shallow feature map, and the size of each columnar feature map is H. M ×W M ×C D , where H M , W M , C D are the height, width and number of channels of the columnar feature map respectively; then, a pooling operation is performed on each columnar feature to obtain the regional-level quantized attention value;

[0012] (2) Arrange the regional-level quantized attention values ​​according to the full sky field of view spatial division rules and restore them to the center position of the original full sky area. The size of the restored feature map is Where P = H / H M =W / W M , D=C / C D , where H, W and C correspond to the height, width and channel of the shallow features respectively, P is the number of blocks divided according to the height and width on the feature map, and D is the number of blocks divided on the channel; the bilinear interpolation method is used to generate a reconstructed feature map with the same size as the original full sky image in space, but the channel is still D, that is, the smooth attention matrix related to the neighborhood information is reconstructed to obtain the smooth attention matrix;

[0013] (3) Use the smooth attention matrix to perform pixel-by-pixel multiplication on the shallow feature map in the entire sky domain to obtain a fused feature map after spatial enhancement and channel screening;

[0014] (4) Add the fused feature map to the shallow feature map to complete the residual learning of the potential significant features and enhance the representation ability of the deep feature extraction network;

[0015] (5) Based on the deep feature extraction layer, the detailed information of the aurora features in the entire sky area is magnified, and the fuzzy representation of the deformation features is achieved. The entire sky field of view is then divided, and the existence of small-scale aurora structures is predicted on each grid to achieve end-to-end target detection.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] First, since the present invention adopts a columnar feature map division strategy, global information is decomposed into local details, so that the neural network can focus on the more subtle content within the aurora image.

[0018] Second, since the present invention uses the Average pooling method to compress each columnar feature map into a one-dimensional tensor that can reflect the columnar feature map, it effectively reduces information redundancy and enhances the robustness and representativeness of the features.

[0019] Third, since the present invention adopts a bilinear interpolation method, the reconstructed feature map not only considers the values ​​of the two pixels closest to the target pixel, but also considers the values ​​of their adjacent pixels, thereby better maintaining the spatial continuity of the image, reducing the blurred or jagged edges that appear in the upsampled image, and producing smoother and higher quality images.

[0020] Fourth, since the present invention performs weighted fusion of quantized features and shallow features, the neural network can dynamically learn weights, thereby enhancing features that are important to the current task and suppressing features that are not important to the current task. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart for realizing the method of the present invention;

[0022] Figure 2 Schematic diagram of the basic framework of the attention mechanism in the present invention;

[0023] Figure 3 It is a schematic diagram of the relationship between the observation target and the observation station in the present invention;

[0024] Figure 4: is a visualization result diagram of the present invention and other attentions on the aurora image target detection task, where a is the detection result of the present invention, b is the detection result of the channel attention mechanism CA, c is the detection result of the spatial attention parallel channel attention CA+SA, d is the detection result of the convolutional attention mechanism CBAM, e is the detection result of the spatial attention SA, and f is the detection result of the Baseline model;

[0025] Figure 5 It is a heat map of the present invention and other attention mechanisms on the task of aurora image target detection; wherein a is the heat map under the present invention, b is the heat map under the channel attention mechanism CA, c is the heat map under the spatial attention parallel channel attention CA+SA, d is the heat map under the convolutional attention mechanism CBAM, e is the heat map under the spatial attention SA, and f is the heat map under the Baseline model. Specific implementation methods

[0027] The present invention will be further described below in conjunction with the accompanying drawings.

[0028] Embodiment 1: Refer to the attached Figure 1 The present invention proposes a method for positioning and measuring small-scale auroral structures with a full sky view, comprising the following steps:

[0029] Step 1. Input an aurora image taken by a full-sky lens with an effective field of view of 360°×180°. Obtain the shallow feature map of the full-sky aurora image through the shallow feature extraction layer, and divide the full-sky field of view space into K blocks of size H M ×W M Then, on the channel, the region block is further divided into D sub-blocks. According to this spatial division rule, K×D columnar feature maps are obtained, where each columnar feature map corresponds to an area in the shallow feature map, and the size of each columnar feature map is H. M ×W M ×C D , where H M , W M , C D are the height, width and number of channels of the columnar feature map respectively; then, a pooling operation is performed on each columnar feature to obtain the regional-level quantized attention value.

[0030] The region division and pooling operations of this embodiment are performed according to the following steps:

[0031] (1.1) Intensity scaling of the original image is performed to make the aurora visible, and all ASI images are preprocessed, including dark current subtraction, masking, and cropping, followed by channel expansion of the preprocessed aurora images;

[0032] (1.2) Perform shallow feature extraction on the image obtained in step (1.1) to obtain shallow features X, divide it into regions, obtain columnar feature maps of different regions, and set the size of each columnar feature map to H M ×W M ×C D ;

[0033] (1.3) Perform an average pooling operation on the obtained columnar feature map to convert a columnar feature map into a value that reflects the importance of local columnar information, that is, the regional-level quantized attention value. The implementation is as follows:

[0034]

[0035] Among them, M i Represents the attention value of the quantized region level of the i-th columnar feature map, i = 1, 2, 3, ..., K × D.

[0036] Step 2. Arrange the regional-level quantized attention values ​​according to the full sky field of view space division rules and restore them to the center of the original full sky area. The size of the restored feature map is Where P = H / H M =W / W M , D=C / C D , where H, W and C correspond to the height, width and channel of the shallow features respectively, P is the number of blocks divided according to the height and width on the feature map, and D is the number of blocks divided on the channel; the bilinear interpolation method is used to generate a reconstructed feature map with the same size as the original full sky image in space, but the channel is still D, that is, the reconstruction of the smooth attention matrix related to the neighborhood information is completed, and the smooth attention matrix is ​​obtained. The specific implementation steps are as follows:

[0037] (2.1) Reorder each attention value according to the position of its columnar feature in the shallow feature X, and restore it to the center of the original full sky area. The restored feature size is

[0038] (2.2) The restored features are reconstructed using the bilinear interpolation method to generate a smooth attention matrix F' that is equal to the size of the shallow feature X in space, and the size of F' is equal to F in the channel:

[0039] F′=F inter (F)

[0040] in F inter (·) indicates that · is reconstructed using the bilinear interpolation method.

[0041] Step 3. Use the smooth attention matrix to perform pixel-by-pixel multiplication on the shallow feature map in the entire sky domain to obtain a fused feature map after spatial enhancement and channel screening; in this embodiment, the fused feature map is obtained specifically according to the following steps:

[0042] (3.1) Divide the shallow features X into D groups on the channel, and the size of each group of features is H×W×C D ;

[0043] (3.2) Divide the generated features F' into D groups on the channel, and the size of each group of features is H×W×1;

[0044] (3.3) Perform point multiplication on each pixel feature of each set of generated features and its corresponding set of shallow features, as shown in the following formula:

[0045]

[0046] Wherein, m = 1, 2, ... D;

[0047] (3.4) Cascade the dot product results on the channel to obtain the enhanced features As shown below:

[0048] X′={cat(X′ m )|m=1,2,3...D}

[0049] Among them, cat(·) represents the concatenation operation.

[0050] Step 4. Add the fused feature map to the shallow feature map to complete the residual learning of the potential significant features and enhance the representation ability of the deep feature extraction network; this is achieved according to the following formula:

[0051]

[0052] Among them, X′ represents the fused features, X represents the shallow features, Represents a pixel-by-pixel addition operation.

[0053] Step 5. Based on the deep feature extraction layer, the detailed information of the aurora features in the entire sky area is magnified, and the fuzzy representation of the deformation features is realized; then the entire sky field of view is divided, and the existence of small-scale aurora structures is predicted on each grid to achieve end-to-end target detection; the specific steps are as follows:

[0054] (5.1) Through the deep feature extraction layer, the information contained in the small-scale aurora structure is amplified relative to other areas, and the deformation features are blurred;

[0055] (5.2) The whole sky field of view is divided into grids, and the size of each grid is determined according to the size of the small-scale auroral structure to be detected, and the small-scale auroral structure is detected in each grid according to the features enhanced by the neural network, as shown in the following formula:

[0056]

[0057] Among them, x t is a small-scale aurora structure, t represents the t-th grid. When a small-scale aurora structure is detected in the grid, it is represented as 1, otherwise it is 0, completing end-to-end target detection.

[0058] (5.3) According to the actual geographical location, when conducting joint observation and analysis of the aurora with other equipment, it is projected into the geomagnetic coordinates for comparison. The geocentric angle γ = θ-α, where α is the angle between the line of sight and the normal to the ionospheric aurora luminous surface. Using the sine theorem, we can obtain:

[0059] sinγ / R E = sinθ / (R E +h)

[0060] The relationship between the distance d from any pixel to the zenith and the geocentric angle γ and its corresponding zenith angle θ is obtained as follows:

[0061]

[0062] d=(R E +h)γ

[0063] Among them, R E represents the radius of the earth, and h is the height of the aurora.

[0064] Embodiment 2: Refer to the attached Figure 1 , the overall implementation steps of this embodiment are also implemented in Example 1. Now, a specific example of using YOLOv5s as the backbone network, selecting the aurora data set, and using the aurora data set for small-scale target detection tasks after preprocessing the aurora data set is given, and the implementation process of the present invention is further described in detail:

[0065] Step 1: Preprocess the input image, then go through a series of convolutions to obtain shallow features, and perform region division and pooling operations on the shallow features.

[0066] 1.1) To make the auroras visible in the raw images, the intensity was linearly stretched from 0 to a threshold (the dynamic range of the raw images was 0-17000) to 0-255. The image bars are at the bottom, with a Rayleigh range of 0-4000 Rayleigh at 557.7 and 630.0nm and a Rayleigh range of 0-300 Rayleigh at 427.8nm. In addition to intensity scaling, all ASI images require other preprocessing: dark current subtraction, masking and cropping, followed by channel expansion of the processed auroral images.

[0067] 1.2) Perform shallow feature extraction on the processed image to obtain shallow feature X, divide the shallow feature X into regions, obtain columnar feature maps of different regions, and set the size of each columnar feature map to H M ×W M ×C D (Set according to actual situation), where H M , W M , C D are the height, width and number of channels of the columnar feature map respectively. In this experiment, the size of the shallow feature X is 8×8×512, and the size of each columnar feature is 2×2×16.

[0068] 1.3) Perform an average pooling operation on the obtained columnar feature map to convert a columnar feature map into a value that can reflect the importance of local columnar information, according to the following formula:

[0069]

[0070] At this point, a batch of region-level quantized attention values ​​are obtained.

[0071] Step 2: Sort and reconstruct the region-level quantized attention values, as follows:

[0072] 2.1) Each columnar feature map has its own unique position information relative to other columnar feature maps. In order to facilitate the subsequent acquisition of more coherent features, each attention value is reordered according to the position of its columnar feature in the shallow feature X, and restored to the center of the original full sky area. The restored feature size is In this embodiment, the restored feature size is 4×4×32.

[0073] 2.2) Calculate the total number of columnar feature maps. In this example, there are 512 columnar feature maps of the same size. The calculation formula is as follows:

[0074]

[0075] 2.3) Use the bilinear interpolation method to reconstruct the restored features and generate a feature F' that is equal to the shallow feature X in space. However, it is worth noting that the size of F' is equal to F in the channel. This step is performed according to the following formula:

[0076] F′=F inter (F)

[0077] In this embodiment, the reconstructed feature size is 8×8×32.

[0078] Step 3: Use the smooth attention matrix to multiply each pixel feature of the original full sky view point by point. Follow these steps:

[0079] 3.1) Divide the shallow features X into D groups on the channel, and the size of each group of features is H×W×C D ;

[0080] 3.2) Divide the generated features F' into D groups on the channel, and the size of each group of features is H×W×1;

[0081] 3.3) Perform dot multiplication of each set of generated features and each pixel feature of each set of shallow features X, as shown in the following formula:

[0082]

[0083] Where i represents the i-th group of features of the shallow feature X and the generated feature F';

[0084] 3.4) X after dot multiplication i 'Cascade on the channel to obtain enhanced features As shown below:

[0085]

[0086] Step 4: Add the enhanced features. The operation is as follows:

[0087] The enhanced feature X' is added pixel by pixel to the shallow feature X, as shown in the following formula:

[0088]

[0089] Through this step, the fused feature map can have richer information and stronger representation ability, ensuring that the important information in the original image is not lost, and at the same time enhancing the feature information that has been screened and enhanced, such as [V1, V2, V3, V4...V n ]The bold part is the enhanced part.

[0090] Step 5: Locate and measure the obtained features as follows:

[0091] 5.1) Through the deep feature extraction layer, the information contained in the small-scale aurora structure will be amplified relative to other areas, making the small-scale aurora structure more prominent, and the deformation features are blurred to reduce their interference with the accuracy of target detection;

[0092] 5.2) The whole sky field of view is divided into multiple small grids. The size of each grid is determined according to the size of the small-scale auroral structure to be detected. The small-scale auroral structure is detected in each grid according to the enhanced features, as shown in the following formula:

[0093]

[0094] where x i is a small-scale aurora structure, i is the i-th grid, when a small-scale aurora structure is detected in the grid, it is represented as 1, otherwise it is 0, and end-to-end target detection is completed;

[0095] 5.3) According to the actual geographical location of the Yellow River Station, when conducting joint observation and analysis of the aurora with other equipment, it is necessary to project it into the geomagnetic coordinates for comparison. Since the solid angle of each pixel is the same, but the geocentric angle γ is different, it can be expressed as γ = θ-α, where α is the angle between the line of sight and the normal direction of the ionospheric aurora luminous surface. Using the sine theorem, we can get:

[0096] sinγ / R E = sinθ / (R E +h)

[0097] Therefore, the relationship between the distance d from any pixel to the zenith and the geocentric angle γ and its corresponding zenith angle θ can be obtained:

[0098]

[0099] d=(R E +h)γ

[0100] 5.4) In practice, the geocentric angle increases as the zenith angle increases, so the distance between pixels does not satisfy a linear relationship, but a nonlinear mapping relationship, so the distance between pixels also becomes larger and larger.

[0101] The effect of the present invention is further described below in conjunction with simulation experiments.

[0102] In order to fully verify the effectiveness of the present invention, we conducted a series of in-depth experiments and compared it in detail with classic attention mechanisms. These classic attention mechanisms include spatial attention (SA), channel attention (CA), spatial attention and channel attention (CA+SA), and convolutional attention mechanism (CBAM). We conducted a comprehensive evaluation of these different mechanisms to reveal the unique advantages of the present invention and its contribution to improving model performance.

[0103] When conducting the experiments, we carefully selected a set of parameter settings to ensure the reliability and comparability of the experimental results. Specifically, we set the learning rate to 0.01, in order to maintain a moderate parameter update speed during training to avoid too fast or too slow convergence problems. The batch size is set to 16. This choice takes into account the limitations of computing resources and the stability of model training, while also taking into account the balance between training speed and memory consumption. The momentum is 0.937. Finally, we selected the training cycle (epoch) as 300. This number ensures that the model is fully trained while avoiding the risk of overfitting.

[0104] Through these careful experimental designs and parameter settings, we aim to fully demonstrate the superiority of this invention over classical methods. We will carefully analyze the experimental results, deeply explore the performance differences between different mechanisms, and the unique contribution of this invention in improving model performance. These experimental data will provide us with valuable reference and guidance for further understanding and application of attention mechanisms, and help promote the development of research and application in related fields.

[0105] Experiment 1: Comparison with other methods to verify the effectiveness of the present invention

[0106] In order to ensure the effectiveness of the experiment, we chose to conduct the experiment under the same network framework, that is, all the experiments were conducted under the YOLOv5s network architecture, and all the parameters were selected the same. The present invention uses the full sky image from the Yellow River Station (YRS) in Ya°lesund, Svalbard Islands, and divides the dayside aurora into four active areas. The YRS geographical coordinates are located at 78.92°N, 11.93°E. The YRS optical system consists of three full sky cameras, which have been developed since December 2003 to measure light emissions at 427.8, 557.7 and 630.0nm. In the experiment, the aurora images can be divided into 4 categories, namely, the dayside aurora is divided into arc aurora, drapery corona aurora, radial corona aurora and hot-spot aurora. Our comparison results are shown in Table 1:

[0107] Table 1 Comparison results

[0108]

[0109] We compared our proposed small-scale auroral structure localization and measurement method for the full sky view with channel attention (CA) and spatial attention (SA), spatial attention plus channel attention (CA+SA), Baseline, and convolutional attention mechanism (CBAM) to improve representation capabilities, demonstrating the advantages of our proposed block attention for aurora image feature extraction.

[0110] After rigorous experimental comparison, as shown in Table 1, the present invention has shown significant advantages in extracting features from aurora images. Compared with channel attention (CA), the present invention can more accurately capture key information in the image through its unique design, thereby improving the accuracy by 0.8%. Compared with spatial attention (SA), the present invention not only focuses on the spatial position information of the image, but also combines local and global contextual information, achieving a 0.6% improvement in accuracy.

[0111] Compared with the Baseline model, the proposed method significantly enhances the representation capability of the model, making it more efficient and accurate in extracting aurora information, achieving a 1% accuracy improvement. Compared with the Convolutional Attention Mechanism (CBAM), the proposed method further enhances the representation capability of the model while maintaining high efficiency, achieving a 0.4% accuracy improvement.

[0112] Finally, compared with the combination of channel attention and spatial attention in parallel (CA+SA), the present invention, with its unique advantages, not only performs well on a single attention mechanism, but also shows excellent performance when multiple attention mechanisms are fused, achieving an accuracy improvement of 0.8%. This result fully demonstrates the advantages and potential of the present invention in the task of feature extraction of aurora images.

[0113] Figure 4is a visualization result of six methods, where a is the detection result under the present invention, b is the detection result under the channel attention mechanism (CA), c is the detection result under the spatial attention parallel channel attention (CA+SA), d is the detection result under the convolutional attention mechanism (CBAM), e is the detection result under the spatial attention (SA), and f is the detection result under the Baseline model. By observing the detection results of each model, it can be found that compared with other attention mechanisms, our method has a more sophisticated attention allocation ability, and can accurately and efficiently focus the model's attention on the local micro-areas in the image. This concentration not only enhances the model's ability to capture subtle and key features, but also greatly enhances the depth and accuracy of the description of these local features.

[0114] Figure 5 It is also a heat map visualization result of six methods, where a is the heat map under the present invention, b is the heat map under the channel attention mechanism (CA), c is the heat map under spatial attention parallel channel attention (CA+SA), d is the heat map under the convolutional attention mechanism (CBAM), e is the heat map under spatial attention (SA), and f is the heat map under the Baseline model.

[0115] Experiment 2: Ablation experiment to verify the residual

[0116] In order to comprehensively evaluate the actual effect of our proposed method on the aurora target detection task, we conducted a series of in-depth ablation experiments. The main purpose of these experiments is to accurately measure the independent performance of each component in the model and its contribution to the overall performance of the model. By comparing the model performance under different configurations, we can more clearly understand how each module works together to optimize the final detection results.

[0117] We introduced residual connections to further analyze their role in model training stability and performance improvement.

[0118] After introducing residual connections, we observed that the model was more stable during training and could converge to a better solution faster. This is because residual connections allow the model to skip certain layers, thus avoiding the problem of gradient disappearance or explosion, making it easier to train deep networks.

[0119] Table 2 Impact of residual connection

[0120]

[0121] Experiment 3: Ablation experiment, the impact of the number of channels

[0122] In order to explore the impact of the number of channels on experimental accuracy, we conducted a third experiment by setting different numbers of channels to study the impact on detection accuracy. The experimental results are shown in Table 3.

[0123] We performed various operations on the shallow feature maps in the channel dimension, including no partitioning, partitioning into 2 blocks, 4 blocks, 8 blocks, 16 blocks, and 32 blocks. Our results show that the impact of the number of channel partitions on model performance shows a complex trend, deviating from a direct linear relationship. The best results were obtained with 16 blocks, which may indicate that moderate channel partitioning helps improve detection accuracy. These results highlight the importance of adjusting the number of channel partitions to optimize model performance according to specific application scenarios and datasets. Therefore, in practical applications, it is crucial to determine the best channel partitioning strategy for the task at hand through experiments and evaluation.

[0124] Table 3 Influence of the number of channels

[0125]

[0126] Experiment 4: Ablation experiment, the impact of the number of spatial blocks

[0127] In order to explore the impact of the number of spatial blocks on experimental accuracy, we conducted a third experiment by setting different numbers of spatial blocks to study the impact on detection accuracy. The experimental results are shown in Table 3.

[0128] We partitioned the shallow feature maps in the spatial dimension into blocks, including no partition, 2 blocks, and 4 blocks. Our results show that the impact of the number of spatial partitions on model performance shows a complex trend, deviating from a direct linear relationship. This may indicate that moderate spatial partitioning helps improve detection accuracy. These results highlight the importance of adjusting the number of channel partitions to optimize model performance according to specific application scenarios and datasets. Therefore, in practical applications, it is crucial to determine the best spatial partitioning strategy for the task at hand through experiments and evaluation.

[0129] Table 4 The impact of the number of spatial blocks

[0130]

[0131] Experiment 5: Ablation experiment, the impact of different upsampling methods

[0132] In the field of image processing, bilinear interpolation shows significant advantages over other interpolation methods, such as nearest neighbor interpolation, especially in the pursuit of image smoothness and quality.

[0133] The core advantage of bilinear interpolation is that it takes into account the influence of the four neighboring pixels around the interpolation point. By performing linear interpolation between these four points, the pixel value on a transition line is first calculated along one direction (usually horizontal or vertical), and then similar interpolation is performed along the other direction to finally obtain the value of the target pixel. This bidirectional linear interpolation method is not only relatively simple to calculate, but also can effectively capture the gradual trend of pixel values ​​in space.

[0134] Compared with nearest neighbor interpolation, bilinear interpolation avoids the "hard jump" of directly using the nearest neighbor pixel, thereby reducing the jagged edge phenomenon commonly seen when enlarging an image. It makes the interpolation result more natural and the image edge softer through smooth transition, thus improving the overall visual quality of the image.

[0135] Table 5 The impact of different upsampling methods

[0136]

[0137] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.

[0138] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A method for locating and measuring small-scale auroral structures with full sky vision, characterized in that: The steps include: (1) Input an aurora image taken by a full-sky lens with an effective field of view of 360°×180°. The shallow feature map of the full-sky aurora image is obtained through the shallow feature extraction layer, and the full-sky field of view space area is divided into K blocks of size H. M ×W M Then, on the channel, the region block is further divided into D sub-blocks. According to this spatial division rule, K×D columnar feature maps are obtained, where each columnar feature map corresponds to an area in the shallow feature map, and the size of each columnar feature map is H. M ×W M ×C D , where H M , W M , C D are the height, width and number of channels of the columnar feature map respectively; then, a pooling operation is performed on each columnar feature to obtain the regional-level quantized attention value; (2) Arrange the regional-level quantized attention values ​​according to the full sky field of view spatial division rules and restore them to the center position of the original full sky area. The size of the restored feature map is Where P = H / H M =W / W M , D=C / C D , where H, W and C correspond to the height, width and channel of the shallow features respectively, P is the number of blocks divided according to the height and width on the feature map, and D is the number of blocks divided on the channel; the bilinear interpolation method is used to generate a reconstructed feature map with the same size as the original full sky image in space, but the channel is still D, that is, the smooth attention matrix related to the neighborhood information is reconstructed to obtain the smooth attention matrix; (3) Use the smooth attention matrix to perform pixel-by-pixel multiplication on the shallow feature map in the entire sky domain to obtain a fused feature map after spatial enhancement and channel screening; (4) Add the fused feature map to the shallow feature map to complete the residual learning of the potential significant features and enhance the representation ability of the deep feature extraction network; (5) Based on the deep feature extraction layer, the detailed information of the aurora features in the entire sky area is magnified, and the fuzzy representation of the deformation features is achieved. The entire sky field of view is then divided, and the existence of small-scale aurora structures is predicted on each grid to achieve end-to-end target detection.

2. The method according to claim 1, characterized in that: The region division and pooling operation described in step (1) is performed according to the following steps: (1.1) Intensity scaling of the original image is performed to make the aurora visible, and all ASI images are preprocessed, including dark current subtraction, masking, and cropping, followed by channel expansion of the preprocessed aurora images; (1.2) Perform shallow feature extraction on the image obtained in step (1.1) to obtain shallow features X, divide it into regions, obtain columnar feature maps of different regions, and set the size of each columnar feature map to H M ×W M ×C D ; (1.3) Perform an average pooling operation on the obtained columnar feature map to convert a columnar feature map into a value that can reflect the importance of local columnar information, that is, the regional-level quantized attention value.

3. The method according to claim 2, characterized in that: The average pooling operation is performed on the obtained columnar feature map in step (1.3), which is implemented as follows: Among them, M i Represents the attention value of the quantized region level of the i-th columnar feature map, i = 1, 2, 3, ..., K × D.

4. The method according to claim 1, characterized in that: The reconstruction of the smooth attention matrix in step (2) is achieved as follows: (2.1) Reorder each attention value according to the position of its columnar feature in the shallow feature X, and restore it to the center of the original full sky area. The restored feature size is (2.2) The restored features are reconstructed using the bilinear interpolation method to generate a smooth attention matrix F' that is equal to the size of the shallow feature X in space, and the size of F' is equal to F in the channel: F′=F inter (F) in F inter (·) indicates that · is reconstructed using the bilinear interpolation method.

5. The method according to claim 4, characterized in that: The fused feature map in step (3) is obtained according to the following steps: (3.1) Divide the shallow features X into D groups on the channel, and the size of each group of features is H×W×C D ; (3.2) Divide the generated features F' into D groups on the channel, and the size of each group of features is H×W×1; (3.3) Perform point multiplication on each pixel feature of each set of generated features and its corresponding set of shallow features, as shown in the following formula: Wherein, m = 1, 2, ... D; (3.4) Cascade the dot product results on the channel to obtain the enhanced features As shown below: X′={cat(X' m )|m=1,2,3...D} Among them, cat(·) represents the concatenation operation.

6. The method according to claim 1, characterized in that: Step (4) adds the fused feature map to the shallow feature map, which is implemented according to the following formula: Among them, X′ represents the fused features, X represents the shallow features, Represents a pixel-by-pixel addition operation.

7. The method according to claim 1, characterized in that: The end-to-end target detection in step (5) is specifically obtained by positioning and measuring the obtained features, and the implementation steps are as follows: (5.1) The information contained in the small-scale auroral structure is amplified relative to other regions through the deep feature extraction layer, and the deformation features are blurred; (5.2) The whole sky field of view is divided into grids, and the size of each grid is determined according to the size of the small-scale auroral structure to be detected, and the small-scale auroral structure is detected in each grid according to the features enhanced by the neural network, as shown in the following formula: Among them, x t is a small-scale aurora structure, t represents the t-th grid. When a small-scale aurora structure is detected in the grid, it is represented as 1, otherwise it is 0, completing end-to-end target detection. (5.3) According to the actual geographical location, when conducting joint observation and analysis of the aurora with other equipment, it is projected into the geomagnetic coordinates for comparison. The geocentric angle γ = θ-α, where α is the angle between the line of sight and the normal to the ionospheric aurora luminous surface. Using the sine theorem, we can obtain: sinγ / R E =sinθ / (R E +h) The relationship between the distance d from any pixel to the zenith and the geocentric angle γ and its corresponding zenith angle θ is obtained as follows: d=(R E +h)c Among them, R E represents the radius of the earth, and h is the height of the aurora.

Images