Solar dynamics observation data pre-training method based on multi-scale contrast learning

Abstract

The invention discloses a solar dynamics observation data pre-training method based on multi-scale contrast learning, and belongs to the technical field of intelligent image processing. The method comprises the following steps: preprocessing SDO observation data, and constructing a pre-training framework combining an auto-encoder and contrast learning, including reconstruction loss and three levels of contrast targets: class mark alignment of global time indexes, cross-modal patch alignment of consistent positions, and spatial discrimination constraints in images at the same moment; signature logarithm intensity transformation and synchronous geometric enhancement are adopted, CNN / ViT backbones are compatible, and multi-granularity characterization with an invariable output mode and physical perception is achieved. The model trained by the method can be used as a unified visual backbone of solar image analysis, supports channel translation and missing measurement interpolation, event classification and forecast, segmentation and retrieval and other applications, reduces the labeling and computing power requirements, improves the precision and robustness, and is suitable for multi-task migration and rapid landing deployment in solar physical research and space weather services.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image intelligent processing in solar physics and space weather, and particularly relates to a multi-scale contrast learning pre-training model and method for SDO (Solar Dynamics Observatory) AIA and HMI observation data, and in particular relates to a solar dynamics observation data pre-training method based on multi-scale contrast learning. BACKGROUND

[0002] The Solar Dynamics Observatory (SDO) has provided multi-modal solar observation data with high temporal and spatial resolution since 2010, in which the Atmospheric Imaging Instrument (AIA) multi-band reflects the radiation response of different temperatures and heights, and the Helioseismic and Magnetic Imaging Instrument (HMI) records the photospheric magnetic field and velocity field.

[0003] Existing data-driven methods have made progress in tasks such as flare prediction, coronal hole segmentation, and cross-channel translation, but generally have the following problems: 1) lack of a general visual backbone suitable for solar observation, making it difficult to transfer across tasks; 2) multi-modal (AIA / HMI) co-temporal correspondence is not systematically aligned, making it difficult to share cross-modal knowledge; 3) solar rotation causes similar patterns between time series samples, with sparse and local signal activity, resulting in weak inter-class separability and strong intra-class variability, which can cause feature collapse and over-smoothing; 4) direct application of natural image pre-training (ImageNet / CLIP) and conventional enhancement can destroy physical consistency or ignore the magnetic field-dominated cross-modal relationship, resulting in limited transfer effectiveness.

[0004] Therefore, there is an urgent need for a multi-granularity cross-modal contrast pre-training method for SDO that learns discriminative representations that are temporally, spatially, and modally coordinated while preserving luminosity and spatial fidelity. SUMMARY

[0005] To address the deficiencies in the prior art, the present application aims to provide a solar dynamics observation data pre-training method based on multi-scale contrast learning that improves cross-time discrimination and local spatial discrimination while preserving physical consistency, meeting the efficient transfer and engineering deployment needs of generation and discrimination downstream tasks.

[0006] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: a solar dynamics observation data pre-training method based on multi-scale contrast learning, comprising the following steps:

[0007] Data preprocessing: Obtain AIA multi-channel data and HMI data of Solar Dynamics Observatory (SDO), standardize the resolution (uniform to 1024x1024) and intensity of the data, and use signature logarithmic transformation to compress the long-tail distribution; perform synchronous geometric enhancement on the same sample composed of AIA multi-channel data and HMI data of the same time and the same spatial region to maintain the consistency of co-time and co-site;

[0008] Encoding and decoding: build an encoder and a decoder for processing different modal AIA multi-channel data and HMI data, wherein the encoder is used to convert the input data into a sequence representation containing class label tokens and image patch tokens, and the decoder is used to reconstruct the input data according to the unprojected patch tokens to maintain photometric and spatial fidelity;

[0009] Multi-granularity contrast learning: train the encoder and the decoder through a multi-granularity contrast learning loss function, which includes class-level contrast alignment, patch-level contrast alignment, and intra-image spatial discrimination;

[0010] Joint optimization: train with the weighted sum of reconstruction loss and the above multi-granularity contrast loss to obtain pre-trained model weights that can be used for multi-task transfer.

[0011] Further, the signature logarithmic transformation is x←sign(x)·log(1+|x|) to alleviate the unstable influence of highlight sparse areas on optimization, where x represents the original intensity value of the input data.

[0012] Further, the synchronous geometric enhancement includes random rotation and flipping, and the same parameters are used on each modal data of the same sample to maintain cross-modal registration.

[0013] Further, the multi-granularity contrast learning loss function is specifically:

[0014] Class-level contrast (time index) alignment: for the same sample, build a bidirectional InfoNCE loss based on class label tokens to improve the similarity of cross-modal representations of the same time and reduce the similarity of different times, realizing global cross-modal alignment under time index;

[0015] Patch-level contrast (location consistency) alignment: build a bidirectional InfoNCE loss for patch tokens of AIA multi-channel data and HMI data according to fixed spatial index to strengthen the local feature consistency of co-site cross-modal;

[0016] Intra-image spatial discrimination: build a square bidirectional InfoNCE loss for cross-modal patch tokens of each spatial index in the same sample to prevent feature collapse by taking different spatial positions as hard negative samples, and improve spatial resolution capability.

[0017] Further, the class-level contrast, patch-level contrast and intra-image spatial discrimination are all provided with learnable temperature parameters to adaptively control the sharpness of the similarity distribution.

[0018] Further, the encoder is a convolutional neural network or a visual Transformer; the convolutional neural network obtains patch tokens by flattening the spatial dimension, and the visual Transformer obtains patch tokens by standard patching embedding.

[0019] Further, the training process using the weighted sum of the reconstruction loss and the multi-granularity contrast loss trains the sub-modal in batches.

[0020] Further, the pre-training model is used as a unified visual backbone, combined with a diffusion model control branch to realize cross-channel translation and missing data interpolation, or used through a linear classification head for flare and event classification.

[0021] The application also provides an electronic device, comprising a memory and a processor, the memory being coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the method.

[0022] The application also provides a solar dynamics observation data pre-training device based on multi-scale contrast learning, comprising a memory and one or more processors, the memory storing executable code, and the one or more processors executing the executable code to realize the method.

[0023] The application has the following advantages:

[0024] Practicality: As a unified visual backbone, it significantly improves the accuracy and robustness in cross-channel translation, missing data interpolation, flare prediction, region segmentation and event retrieval, and reduces the cost of labeling and computing power.

[0025] Novelty: For the first time, a multi-granularity cross-modal contrast pre-training framework of "time index + position index + intra-image discrimination" is proposed on SDO data, and the system uses AIA-HMI co-time registration to jointly maintain physical fidelity.

[0026] Creativity: Design data transformation, contrast structure and hard negative sample sampling strategy for solar physics, propose a multi-objective contrast loss with learnable temperature regulation, realize multi-scale representation invariant to modal but physically perceptible, and is superior to direct migration of natural image pre-training scheme. BRIEF DESCRIPTION OF DRAWINGS

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0029] Figure 2 These are schematic diagrams illustrating different loss function calculation methods according to embodiments of the present invention;

[0030] Figure 3 This is a structural block diagram of the device provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0033] like Figure 1 As shown, an embodiment of the present invention provides a pre-training method for solar dynamics observation data based on multi-scale comparative learning, comprising the following steps:

[0034] Step 1, Data Preprocessing: Acquire AIA multichannel data and HMI data from the Solar Dynamics Observatory, perform resolution standardization and intensity normalization on the data, and use signature logarithmic transformation to compress long-tailed distribution; perform synchronous geometric enhancement on the same sample composed of AIA multichannel data and HMI data from the same time and the same spatial region to maintain synchronous and co-located consistency.

[0035] Step 2, Encoding and Decoding: Construct an encoder and decoder for processing AIA multichannel data and HMI data of different modalities. The encoder is used to convert the input data into a sequence representation containing class marker tokens and image patch tokens, and the decoder is used to reconstruct the input data based on the unprojected patch tokens to maintain luminance and spatial fidelity.

[0036] Step 3, Multi-granularity contrastive learning: Train the encoder and decoder through a multi-granularity contrastive learning loss function, which includes class-level contrastive alignment, patch-level contrastive alignment, and intra-image spatial discrimination.

[0037] Step 4, Joint optimization: Train with the weighted sum of reconstruction loss and the above multi-granularity contrastive loss to obtain pre-trained model weights that can be used for multi-task transfer.

[0038] Step 5, Architecture and deployment: Encoder can be a CNN or ViT; subsampling of modalities during training to reduce computation; pre-trained model weights can be used as a unified backbone for generation (combined with diffusion / ControlNet for cross-channel translation and interpolation) and discrimination (linear head / fine-tuning for flare and event classification, segmentation, and retrieval) tasks.

[0039] As a preferred embodiment, the data preprocessing of step 1 is as follows: The original 4096x4096 full-disk images of each channel of HMI and AIA are uniformly downsampled to 1024x1024 to balance detail fidelity and computational overhead; For the characteristics of long-tail distribution of solar activity intensity and local extreme value, a sign logarithmic transformation x←sign(x)·log(1+|x|) is applied to each channel of each modality to compress the dynamic range, stabilize the optimization process, and improve the separability of samples at different activity levels; To maintain the spatial co-annotation relationship across modalities and enhance the local discrimination training signal, consistent geometric enhancement (rotation and flipping) is applied to HMI and all AIA channels at the same observation time, thereby increasing the difficulty of effective spatial position combination and negative samples without violating the physical and radiometric consistency; At the same time, appearance-based enhancements that change luminosity and overall context (such as color perturbation, random cropping, etc.) are avoided to ensure that the radiometric calibration and full-disk scene integrity of AIA / HMI are not affected.

[0040] Through the above steps, training samples with unified resolution, robust intensity distribution, and strict cross-modality alignment are constructed, providing a reliable data foundation for subsequent multi-granularity contrastive learning and autoencoding reconstruction.

[0041] As a preferred embodiment, the encoding and decoding process of step 2 is as follows: After data preprocessing, the HMI and each AIA channel data at the same observation time are sent into the corresponding single-modality encoder (i=0 represents HMI, i∈{1…10} represents AIA channels), and the input is the preprocessed single-channel 1x1024x1024 image. The encoder output is a label sequence consisting of a global class label and L local patch labels , wherein the encoder is a convolutional encoder or a Transformer encoder, the convolutional encoder is configured to realize flattening of a two-dimensional feature map according to HxW to obtain patch labels, and realizing class labels by global average pooling and linear projection; and the Transformer encoder is configured to obtain the above-mentioned labels by standard patch embedding and explicit CLS labels.

[0042] To cooperate with subsequent contrast learning, two types of linear projection heads are arranged at the end of the encoder. acting on the CLS labels to obtain contrast embedding , acting on the patch labels to obtain contrast embedding ; but the reconstruction path does not pass through the above-mentioned projection, but directly uses the unprojected patch labels as input, and is input into the decoder corresponding to the modality , which is sequentially up-sampled and restored to obtain a reconstructed image , so as to maximize the preservation of spatial details and radiation intensity information.

[0043] To constrain the fidelity of lightness and spatial structure, a pixel-level two-norm reconstruction loss is defined for each modality: ; if necessary, multi-scale or structural similarity weights can be superimposed, but in this embodiment, L2 is mainly used to obtain stable convergence and to facilitate implementation.

[0044] During training, the encoder, the decoder and related parameters are jointly optimized, wherein the reconstruction error is only returned through the unprojected branch, so as to prompt the backbone network to learn fine-grained representations friendly to dense prediction, and to provide high-fidelity global and local features for subsequent class-level contrast alignment.

[0045] As a preferred embodiment, as shown in Figure 2 , the multi-granularity contrast learning of step 3 specifically includes:

[0046] Class-level contrast alignment: for the same sample, a bidirectional InfoNCE loss is constructed based on class label tokens to improve the similarity of the same-ink cross-modality representation and reduce the similarity of different-ink, so as to realize global cross-modality alignment under the time index;

[0047] Patch-level contrast alignment: a bidirectional InfoNCE loss is constructed for patch tokens of AIA multi-channel data and HMI data according to a fixed spatial index, so as to strengthen the local feature consistency of the same-position cross-modality;

[0048] Intra-image spatial discrimination: a square bidirectional InfoNCE loss is constructed for cross-modality patch tokens of each spatial index in the same sample, so as to prevent feature collapse by taking different spatial positions as hard negative samples, and to improve the spatial resolution.

[0049] As a preferred embodiment, as shown inFigure 2 As shown, the class-level contrastive alignment is specifically: after completing the construction of the encoding and reconstruction branch, the embodiment performs cross-modal alignment on the global class label (CLS) of the same observation time. It includes: taking the CLS embedding corresponding to the HMI as the anchor (i=0), and pairing with the CLS embedding of the AIA channel set S selected at the step; for a batch of samples with a size of B, first normalize the feature dimension L2 of each sample to construct a similarity matrix , , where is a learnable temperature parameter, and m and n index the samples within the batch. The paired samples at the same time (m=n) are regarded as positive pairs, and the different time samples (m≠n) are regarded as negative pairs. The class-level contrastive loss is calculated in the form of bidirectional InfoNCE / cross-entropy: , where the label diag=[0, 1, …, B−1] represents the diagonal one-to-one correspondence.

[0050] This loss narrows the HMI-AIA representation at the same time and widens the scene representation at different times at the global semantic level, thereby improving the stability of cross-modal alignment and the separability of different time samples. During training, the gradient of this loss is only returned to the corresponding encoder and CLS projection head, and the decoder does not participate in this update. Under the condition of limited computing power, the subset channels S are aligned every training step and rotated to traverse all channels within several steps, so as to ensure the alignment coverage under the premise of controllable overhead.

[0051] As a preferred embodiment, as shown in Figure 2 , the patch-level contrastive alignment is specifically: to enhance the local spatial discriminability and align the cross-modal information at the position level, the embodiment performs position-consistent contrastive learning on the patch label. It includes: for a batch of co-temporal samples with a size of B, taking the patch contrastive embedding of the HMI and the AIA channel i selected at the step and , construct a cross-modal similarity matrix for each spatial index j∈[0, L−1] in the order agreed in the encoding stage (from left to right, from top to bottom) , where is a learnable temperature parameter, and L2 normalization is implemented in the feature dimension. The HMI-AIA patches at the same time and the same spatial position are regarded as positive pairs (diagonal elements of the matrix), and the other samples within the batch at different times are regarded as negative pairs (non-diagonal elements). The patch-level contrastive loss is defined using bidirectional cross-entropy (InfoNCE), and the average is taken over all positions: ​​where diag=[0, 1, …, B−1] denotes diagonal correspondence. Since the preprocessing stage applies identical geometric augmentation (rotation / flipping) to each modality within the same observation time, the patch index is strictly aligned across modalities, ensuring reliable positive pair construction with consistent location.

[0052] This loss narrows the co-temporal and co-location information on a local scale and suppresses the interference of different time, so that the encoder learns the fine-grained morphological and intensity features related to the magnetic topology and thermal structure, thereby providing stronger transfer representation for dense prediction tasks such as segmentation, detection, and cross-channel translation. During training, the gradient of this loss is only returned to the encoder and patch projection head, and the decoder does not participate in this update.

[0053] As a preferred embodiment, as shown in Figure 2 , the intra-image spatial discrimination is specifically: to suppress the representation collapse caused by the “pulling” of the significant area to the background patch, and to improve the separability of the quiet zone and weak signal, this embodiment constructs a cross-modality location fully connected contrast constraint within the same image. For a batch size of B and a patch number of L co-temporal samples, the patch contrast embedding of the HMI and the AIA channel i selected at this step is taken , , L2 normalized in the feature dimension; for the kth sample in the batch, an L×L similarity matrix is constructed , , where is a learnable temperature parameter, and the (p, q) element of the matrix represents the similarity between the pth patch of the HMI and the qth patch of the AIA in the same sample. The cross-modality patches at the same spatial location (p=q) are considered as positive pairs, and the other locations (p≠q) are considered as negative pairs. The intra-image contrast loss is defined using bidirectional InfoNCE / cross-entropy, and the average is taken over the samples and channels: , where diag=[0, 1, …, L−1] denotes diagonal correspondence. Since the preprocessing stage applies identical geometric augmentation to each modality within the same observation time, the patch index is strictly aligned across modalities, so the above diagonal positive pair is reliable, and the off-diagonal elements can provide rich and strong negative samples within a single image.

[0054] During training, the gradient of this loss is only returned to the encoder and patch projection head, and the decoder does not participate in this update; the intra-image contrast and the “patch-level contrast” of step 4 are complementary, the former suppresses location confusion and background collapse within a single image, and the latter strengthens the alignment of co-location information across images (within a batch), thereby improving spatial accuracy and overall training stability.

[0055] As a preferred embodiment, as shown in Figure 2 , the joint optimization of step 4 is specifically: this method jointly optimizes reconstruction and three types of contrast targets in a single-stage end-to-end manner. The total loss is defined as , where is an hyper-parameter. At each training iteration, the HMI of the same observation time and the subset S uniformly sampled from the ten AIA channels form a batch; the CLS and patch embeddings are obtained via the encoder, and the decoder is reconstructed from the un-projected patch labels is calculated as The contrastive term is based on the L2 normalized cosine similarity and calculated with temperature scaling , , (positive pairs are cross-modality pairs of the same time, and negative pairs are other samples / locations within the batch). During backpropagation, the reconstruction loss only updates the encoder and decoder, and the contrastive loss only updates the encoder and the corresponding projection head; the loss of each channel is first averaged in the channel dimension, and then weighted summed with the weight . Training continues until the validation set converges. After pre-training, the encoder can be used as the initialization weight for downstream tasks (frozen or fine-tuned with small steps).

[0056] The embodiment of the present application also provides an electronic device, including a memory and a processor, the memory is coupled with the processor; wherein the memory is used for storing program data, and the processor is used for executing the program data to realize the method.

[0057] The embodiment of the present application also provides a solar dynamics observation data pre-training device based on multi-scale contrastive learning, including a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to realize the method.

[0058] The embodiment of the solar dynamics observation data pre-training device based on multi-scale contrastive learning can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software realization as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability. From the hardware level, as shown in Figure 3 , it is a hardware structure diagram of the solar dynamics observation data pre-training device based on multi-scale contrastive learning, in addition to the processor, memory, network interface, and non-volatile memory shown in Figure 3 , the device in the embodiment is usually provided with other hardware according to the actual function of the device with data processing capability, and details are not repeated.

[0059] The above examples are only used for illustrating the design idea and characteristics of the present application, and the purpose is to enable the person skilled in the art to understand the present application and to implement it, and the protection scope of the present application is not limited to the above examples. Therefore, any equivalent changes or modifications made according to the disclosed principles and design ideas of the present application are within the protection scope of the present application.

Claims

1. A method for pre-training solar dynamics observation data based on multi-scale contrastive learning, characterized in that, The method comprises the following steps: obtaining AIA multi-channel data and HMI data of the Solar Dynamics Observatory, performing resolution standardization and intensity normalization on the data, and performing signature logarithmic transformation to compress the long-tail distribution; performing synchronous geometric enhancement on the same sample composed of AIA multi-channel data and HMI data of the same time and the same spatial region; constructing an encoder and a decoder for processing different modal AIA multi-channel data and HMI data, wherein the encoder is used to convert input data into a sequence representation containing class label tokens and image patch tokens, and the decoder is used to reconstruct the input data according to unprojected patch tokens; training the encoder and the decoder through a multi-granularity contrast learning loss function, wherein the multi-granularity contrast learning loss function comprises class-level contrast alignment, patch-level contrast alignment and intra-image spatial discrimination; training by using a weighted sum of the reconstruction loss and the multi-granularity contrast loss to obtain pre-training model weights that can be used for multi-task transfer.

2. The method of claim 1, wherein, The signature logarithmic transformation is x←sign(x)·log(1+|x|), wherein x represents the original intensity value of the input data.

3. The method of claim 1, wherein, The synchronous geometric enhancement comprises random rotation and flipping, and the same parameters are used on the modal data of the same sample to maintain cross-modal registration.

4. The method of claim 1, wherein, The multi-granularity contrast learning loss function is specifically: class-level contrast alignment: for the same sample, a bidirectional InfoNCE loss is constructed based on the class label tokens to realize global cross-modal alignment under the time index; patch-level contrast alignment: a bidirectional InfoNCE loss is constructed for the patch tokens of the AIA multi-channel data and the HMI data according to the fixed spatial index to strengthen the local feature consistency of the same-position cross-modal data; intra-image spatial discrimination: a square bidirectional InfoNCE loss is constructed for the cross-modal patch tokens of each spatial index in the same sample, different spatial positions are used as hard negative samples to prevent feature collapse, and the spatial resolution capability is improved.

5. The method of claim 4, wherein, A learnable temperature parameter is provided in the class-level contrast, the patch-level contrast and the intra-image spatial discrimination to adaptively control the sharpness of the similarity distribution.

6. The method of claim 1, wherein, The encoder is a convolutional neural network or a visual Transformer; the convolutional neural network obtains patch tokens by flattening spatial dimensions, and the visual Transformer obtains patch tokens by standard patching embedding.

7. The method of claim 1, wherein, The training process of the weighted sum of the reconstruction loss and the multi-granularity contrast loss is performed on sub-modalities in batches.

8. The method of claim 1, wherein, The pre-training model is used as a unified visual backbone, combined with a diffusion model control branch to realize cross-channel translation and missing data interpolation, or used through a linear classification head for flare and event classification. 9.A pre-training device for solar dynamics observation data based on multi-scale contrastive learning, characterized in that, The device comprises a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to implement the method of any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-8.

Images