Low-rank adaptation based few-shot coronal mass ejection segmentation method and system
By employing low-rank adaptation techniques and multi-channel input design, combined with differential fusion processing and fine-tuning on a small dataset, the high cost and performance degradation issues of existing models in CME segmentation tasks are resolved. This achieves efficient and automated CME segmentation, enhancing the model's ability to perceive and locate CME structures.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO UNIV OF SCI & TECH
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-19
AI Technical Summary
Existing machine learning and deep learning models rely heavily on large-scale labeled data for coronal mass ejection (CME) segmentation tasks, which is costly and limits the scalability and real-time performance of automated systems. Directly transferred general visual foundation models show significant performance degradation in the field of solar physics, and the lack of prior understanding of CME structures and differences in image features makes segmentation difficult.
We employ a low-rank adaptation technique (LoRA) combined with a multi-channel input design. Through differential fusion processing, we generate multi-channel images containing basic differential information and running differential information. We then insert a trainable low-rank adaptation layer into the image encoder and fine-tune the parameters of the pre-trained visual basic segmentation model using a few-sample dataset. By freezing the original weights, we construct an efficient end-to-end segmentation system.
It achieves high-quality CME segmentation with a small amount of labeled data, reduces computational and manual annotation costs, improves the model's ability to perceive and locate CME structures, builds a fully automated segmentation system, and ensures the consistency and reliability of the processing flow.
Smart Images
Figure CN121616830B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of coronal mass ejection segmentation technology, specifically relating to a method and system for segmenting few-sample coronal mass ejections based on low-rank adaptation. Background Technology
[0002] The statements herein provide only background information in relation to this invention and do not necessarily constitute prior art.
[0003] Coronal mass ejections (CMEs) are among the most energetic manifestations of solar activity, ejecting large amounts of magnetized plasma into interplanetary space during their ejection. CMEs can last from minutes to hours and appear in white-light coronagraphs as bright, discrete, and structurally intact regions of coronal enhancement that propagate outwards in their direction of travel. If they are directed towards Earth, they can trigger intense geomagnetic storms, potentially disrupting communications, navigation, power systems, and satellite operations. Therefore, accurately detecting and segmenting CME structures is crucial not only for advancing solar physics research but also for safeguarding technological infrastructure.
[0004] The inventors discovered that traditional machine learning and deep learning models (such as CNNs and Transformer segmentation networks) show potential for CME segmentation tasks. However, these methods heavily rely on large-scale, high-quality manually labeled data for training. The complex structure and varied morphology of CMEs, coupled with their low contrast with the corona background, make professional annotation time-consuming, labor-intensive, and extremely costly. During solar activity peaks, the massive amounts of observational data exacerbate this problem, severely limiting the scalability and real-time performance of automated systems.
[0005] In recent years, general visual foundation models (such as Segment Anything Model, SAM) pre-trained on large-scale natural images have demonstrated strong zero-shot generalization capabilities. However, when such models are directly applied to specialized fields such as solar physics, their performance drops significantly. The reasons are: (1) prior knowledge mismatch: SAM learns the boundaries of natural objects and lacks prior understanding of the morphology and spatial distribution of astronomical structures such as CMEs; (2) differences in image features: corona images have a high dynamic range, complex background structures (such as corona streamlines), and weak contrast between the target and the background, which are very different from the statistical characteristics of natural images, making it difficult for the model to accurately locate and segment CMEs.
[0006] To adapt the basic model to CME analysis tasks, traditional full-parameter fine-tuning methods require updating all parameters of the model, which has two major drawbacks: (1) high computational and storage costs, which contradicts the limited computing resources often faced in the astronomical field; (2) easy overfitting on small-scale professional datasets, destroying the valuable prior knowledge learned by the model on general data. Although some existing parameter-efficient fine-tuning techniques (such as LoRA) have been validated in the field of natural images, there is still no mature solution on how to combine them with the unique preprocessing methods and prompting strategies of CME images to build an end-to-end efficient adaptation scheme. Summary of the Invention
[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and system for segmenting coronal mass ejections (CMEs) with few samples based on low-rank adaptation. Through LoRA low-rank adaptation SAM and multi-channel input design, high-quality CME segmentation is achieved with a small amount of labeled data. While maintaining the prior advantages of SAM, the computational and manual annotation costs are significantly reduced, providing an efficient solution for automated CME analysis.
[0008] To achieve the above objectives, the present invention is implemented through the following technical solution:
[0009] On the one hand, the technical solution of the present invention provides a few-sample coronal mass ejection segmentation method based on low-rank adaptation, including:
[0010] Acquire white light coronagraph image sequences and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information;
[0011] The multi-channel fused image is input into a low-rank adapted visual basic segmentation model for segmentation to obtain a segmentation mask for coronal mass ejections. The visual basic segmentation model uses a low-rank matrix factorization method to fine-tune the parameters of the pre-trained image encoder using a few-sample dataset containing coronal mass ejection annotations. During the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder.
[0012] Image-level labels are determined using segmentation masks based on coronal mass ejections.
[0013] In at least one embodiment, the differential fusion process includes:
[0014] For a target coronal mass ejection event, the coronal image at the moment of eruption is subtracted from the coronal image at the moment before the eruption to obtain the basic difference map.
[0015] Subtract the previous frame's corona image from the current frame's corona image to obtain the running difference map;
[0016] The base difference map and the running difference map are fused to construct a three-channel pseudo-RGB image, resulting in a multi-channel fused image containing both base difference information and running difference information.
[0017] In at least one embodiment, a three-channel pseudo-RGB image is constructed by using a base difference image for the first and second channels and a running difference image for the third channel;
[0018] Alternatively, a three-channel pseudo-RGB image can be constructed by using a running differential image for the first and second channels and a basic differential image for the third channel.
[0019] In at least one embodiment, inserting a trainable low-rank adaptation layer into the image encoder specifically includes: inserting a low-rank adaptation layer onto the query projection matrix and value projection matrix of the multi-head attention module in each Transformer layer in the image encoder part of the visual basic segmentation model.
[0020] In at least one embodiment, for each original query projection matrix Sum projection matrix Introduce a rank of The trainable low-rank matrix factorization is expressed as:
[0021]
[0022]
[0023] In the formula, This represents the query projection matrix after low-rank adaptation; This represents the value projection matrix after low-rank fitting; and This represents trainable low-rank weights.
[0024] In at least one embodiment, the parameter fine-tuning process is as follows: during fine-tuning training, all original parameters of the visual base segmentation model image encoder and all parameters of the cue encoder are frozen, and only the parameters of the newly inserted trainable low-rank weight matrix and the lightweight mask decoder are trained.
[0025] In at least one embodiment, the fine-tuning training specifically includes: first, freezing all the original parameters of the image encoder and cue encoder of the visual base segmentation model; then, traversing each Transformer block in the image encoder; injecting a trainable low-rank weight matrix into the query projection matrix and value projection matrix in the multi-head attention module of each Transformer block; and supervising the training of the low-rank weight matrix with a few-sample training set and a set loss function; finally, producing a fine-tuned model composed of the originally frozen visual base segmentation model weights and the trained low-rank adapter weights.
[0026] In at least one embodiment, the few-sample dataset is constructed in the following ways:
[0027] Acquire historical observation images from the second camera module of the Large Angle Spectroradiometer on the Solar and Heliospheric Observatory satellite;
[0028] Based on the cataloging of coronal mass ejection events, images of events containing different types of coronal mass ejections were selected;
[0029] The selected images were manually annotated at the pixel level to generate binary segmentation masks, which were then divided into training and testing sets containing only coronal mass ejection (CME) scenes and training and testing sets containing mixed CME and non-CME scenes.
[0030] In at least one embodiment, image-level labels are determined based on the segmentation mask of a coronal mass ejection (CME) output by a visual basic segmentation model. If the segmentation mask of a CME is empty, it is determined to be a non-coronal mass ejection; otherwise, it is determined to be a CME.
[0031] On the other hand, the technical solution of the present invention also provides a few-sample coronal mass ejection segmentation system based on low-rank adaptation, comprising:
[0032] The image acquisition module is configured to: acquire a sequence of white light coronagraph images and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information;
[0033] The image segmentation module is configured to: input a multi-channel fused image into a low-rank adapted visual basic segmentation model for segmentation to obtain a segmentation mask for coronal mass ejections; wherein, the visual basic segmentation model fine-tunes the parameters of a pre-trained image encoder using a few-sample dataset containing coronal mass ejection annotations through low-rank matrix factorization; during the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder;
[0034] The output module is configured to determine image-level labels based on a segmentation mask for coronal mass ejections.
[0035] The beneficial effects of the above-described technical solution of the present invention are as follows:
[0036] 1) The low-rank adaptation-based few-sample coronal mass ejection (CME) segmentation method of this invention combines the general visual basic model (SAM) with low-rank adaptation (LoRA) fine-tuning technology. It can efficiently adapt the model to CME segmentation tasks with only a few dozen labeled images, solving the problem of traditional deep learning methods' heavy dependence on large-scale labeled data. At the same time, this invention constructs a multi-channel fusion image preprocessing pipeline for CME, which performs feature complementarity fusion between the basic difference map representing the "static existence" of CME and the running difference map representing the "dynamic motion". This provides key domain prior guidance for the SAM model originally trained on natural images, so that its input channels contain the unique morphological and evolutionary information of CME, thereby significantly improving the model's ability to perceive and locate CME structure, overcoming the problem of performance drop during direct transfer, and achieving high-quality CME segmentation.
[0037] 2) The low-rank adaptation-based few-sample coronal mass ejection segmentation method of the present invention uses a global bounding box covering the entire image as a fixed cue to transform the SAM model, which originally required manual interaction, into a fully automatic segmentation system. The process is simple and stable, and there is no need to adjust parameters or design cue for each image, which ensures the consistency and reliability of the processing flow when facing massive and continuous coronal observation data. Attached Figure Description
[0038] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0039] Figure 1 This is a schematic diagram of the few-sample coronal mass ejection segmentation method based on low-rank adaptation disclosed in Embodiment 1 of the present invention;
[0040] Figure 2 This is a comparative schematic diagram of the low-rank adaptor structure disclosed in Embodiment 1 of the present invention and the traditional full-parameter fine-tuning structure; wherein, (a) is a schematic diagram of the traditional full-parameter fine-tuning structure, and (b) is a schematic diagram of the low-rank adaptor structure disclosed in Embodiment 1.
[0041] Figure 3 This is a schematic diagram of the structure of the low-rank adapted visual basic segmentation model disclosed in Embodiment 1 of the present invention;
[0042] Figure 4 This is a schematic diagram of the visualization results obtained by using a few-sample coronal mass ejection segmentation method based on low-rank adaptation disclosed in Embodiment 1 of the present invention. Detailed Implementation
[0043] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0044] As described in the background section, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for segmenting coronal mass ejections (CMEs) with a small number of samples based on low-rank adaptation. Through LoRA low-rank adaptation SAM and multi-channel input design, high-quality CME segmentation is achieved with a small amount of labeled data. While maintaining the prior advantages of SAM, the computational and manual annotation costs are significantly reduced, providing an efficient solution for automated CME analysis.
[0045] Example 1
[0046] In a typical embodiment of the present invention, such as Figures 1 to 4 As shown, this embodiment discloses a few-sample coronal mass ejection segmentation method based on low-rank adaptation, which specifically includes the following steps:
[0047] S1. Acquire the image sequence of the white light coronagraph and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information;
[0048] S2. Input the multi-channel fused image into the low-rank adapted visual basic segmentation model for segmentation to obtain the segmentation mask of the coronal mass ejection; wherein, the visual basic segmentation model uses a low-rank matrix factorization method to fine-tune the parameters of the pre-trained image encoder using a small sample dataset containing coronal mass ejection annotations; during the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder.
[0049] S3. Determine image-level labels based on segmentation masks for coronal mass ejections.
[0050] The following detailed description of the above-mentioned low-rank adaptation-based few-sample coronal mass ejection segmentation method is provided in conjunction with specific implementation methods.
[0051] S1. Acquire the white light coronagraph image sequence and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information.
[0052] S11. Obtain the image sequence of the white light coronagraph.
[0053] In this step, the sequence of white-light solar corona images comes from the second camera module (LASCO C2 coronagraph) of the Large Angle Spectroradiometer (LASCO) on the Solar and Heliospheric Observatory (SOHO) satellite. It was acquired via SunPy's Fido interface and processed to Level 1.0. This LASCO C2 Level 1.0 image refers to an image that, after decompression / basic processing, has undergone standard instrument calibration and geometric correction (such as bias / dark field subtraction, exposure time normalization, vignetting and stray light correction, distortion correction, mask and orientation unification, etc.). At this point, the image pixel values are in a calibrated state suitable for quantitative analysis.
[0054] The LASCO C2 coronagraph has an observation range of 1.5–6 solar radii and is able to capture the structure of the CME in its early stages as it rises from the lower corona, identifying it before it enters interplanetary space.
[0055] S12. Preprocess the acquired white light coronagraph image.
[0056] Raw white-light coronagraph images obtained using LASCO C2 typically have high dynamic range and pixel values reaching [missing information]. to To enhance the visibility of faint CME structures against a bright background, this step cropped the display area of the original white-light coronagraph image to... Data units are used to enhance local contrast and suppress background noise. The cropped image is displayed in grayscale, and the image is converted from single-channel to three-channel RGB by copying the same single-channel image three times, using each copy as the R, G, and B channels respectively, to meet the input requirements of the segmentation model.
[0057] S13. Perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information.
[0058] Differential processing is key to eliminating static backgrounds and highlighting changing structures. In this step, a base differential image and a running differential image are first generated. Based on the generated base differential image and running differential image, a fusion process is performed to generate a multi-channel fused image that contains both base differential information and running differential information.
[0059] Specifically, for a target CME event, subtracting the coronal image from the image at the moment of eruption yields the Base Difference Image (BDI), which effectively reveals the overall enhancement of the CME relative to the quiescent corona. Subtracting the previous frame's coronal image from the current frame yields the Running Difference Image (RDI), which highlights the motion (propagation) information of the CME across consecutive moments.
[0060] To provide the model with richer feature information, the above-mentioned basic difference image and running difference image are fused together. A three-channel pseudo-RGB image is constructed by using the basic difference image for the first and second channels and the running difference image for the third channel, or by using the running difference image for the first and second channels and the basic difference image for the third channel, as the model input, to obtain a multi-channel fused image containing basic difference information and running difference information.
[0061] S2. Input the multi-channel fused image into the low-rank adapted visual basic segmentation model for segmentation to obtain the segmentation mask of the coronal mass ejection; wherein, the visual basic segmentation model uses a low-rank matrix factorization method to fine-tune the parameters of the pre-trained image encoder using a small sample dataset containing coronal mass ejection annotations; during the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder.
[0062] S21. Construct a visual basic segmentation model with low-rank adaptation.
[0063] In this step, the Visual Basic Segmentation (SAM) model consists of an image encoder, a cue encoder, and a lightweight mask encoder. The image encoder generates a dense feature representation of the input image; the cue encoder maps user cues (points, boxes, existing masks, text, etc.) to the feature space; and the mask decoder combines image features and cue features to generate the final segmentation mask. Since SAM is primarily trained on natural images, its zero-shot transfer performance on astronomical datasets is limited, and it performs poorly on CME images. Therefore, this embodiment employs a low-rank adaptation technique (LoRA) to efficiently adapt the SAM model to the CME segmentation domain.
[0064] Without adaptation, the original SAM model often provides a mask that approximates the entire CME image, indicating its inability to understand the spatial structure of astronomical images. For example... Figure 2 As shown in (a), when using the traditional full-parameter fine-tuning method to adapt the basic model to the CME analysis task, all parameters of the model need to be updated, which has two major drawbacks: (1) the computation and storage costs are high, which contradicts the limited computing resources often faced in the astronomical field; (2) it is easy to overfit on small-scale professional datasets, destroying the valuable prior knowledge learned by the model on general data. Therefore, in this embodiment, a trainable low-rank adaptation layer is inserted into the image encoder, which has the largest computational load and is responsible for aggregating spatial structure information, to improve the CME localization performance. Specifically, in the image encoder part of SAM, the query projection matrix is directed to the multi-head attention module in each Transformer layer. Sum projection matrix A low-rank adaptation layer is inserted to enhance the model's feature capture capability on CME images. Specifically, a low-rank adaptation module (LoRA_qkv) is first constructed. This module maintains the original linear projection layer (qkv) structure and weights in the attention mechanism while adding two sets of low-rank decomposition matrices in parallel. and A set of data used for querying projections, including low-rank matrices. and Another set is used for value projection, including low-rank matrices. and Among them, matrix The dimension is ,matrix The dimension is , The dimension is much smaller than the original model. The preset low-rank value. During the forward propagation of the model, the input features are transformed by the original qkv layer, and also processed by... and , and The cascaded low-rank transform generates the corresponding incremental signal. and Subsequently, and These are respectively superimposed onto the corresponding positions of the original query and value projection outputs, thereby achieving targeted adjustment of attention features without updating the original weights.
[0065] like Figure 2 As shown in (b), for an original weight matrix Introducing rank as The update method for a trainable low-rank matrix is expressed as follows:
[0066]
[0067] In the formula, and Represents a trainable factorization matrix, matrix The dimension is ,matrix The dimension is , The dimension is much smaller than the original model. The default low-rank value. The rank of LoRA. It is the dimension / capacity of low-rank updates, which updates the entire matrix that would otherwise need to be trained. Approximate as the product of two matrices ,in yes B is ,in, ,so The rank is at most . Smaller parameters mean fewer parameters and less training effort, but weaker expressive power. Larger sizes have stronger adaptability but consume more resources and are more prone to overfitting.
[0068] For the original query projection matrix of the multi-head attention module in each Transformer layer Sum projection matrix Introduce a rank of The trainable low-rank matrix factorization is expressed as:
[0069]
[0070]
[0071] In the formula, This represents the query projection matrix after low-rank adaptation; This represents the value projection matrix after low-rank adaptation.
[0072] Due to rank The size of trainable parameters can be controlled: a smaller r results in fewer parameters and faster computation, but limited expressive power; a larger r enhances fit but increases the risk of overfitting. Therefore, a trade-off needs to be struck between model capability and complexity. In determining low-rank fit, the rank... When considering the size of the rank r, this embodiment employs an optimization selection method based on experimental evaluation. Specifically, a set of candidate rank r values is first preset, for example... =16, 32, 64, 128, 256, 512. For each candidate rank, the SAM model is fine-tuned for low-rank adaptation using the same training dataset and configuration to obtain the corresponding adapted model. Subsequently, the segmentation performance of each model is evaluated on a unified test dataset, using the Dice loss function as the evaluation metric. By comparing the average Dice loss of the models under different rank values, the rank with the lowest loss is selected as the final adapted rank. This method, verified through systematic experiments, can ensure that the model achieves an optimal performance balance on the target task while controlling the number of parameters, avoiding insufficient feature representation due to too small a rank or overfitting and computational redundancy due to too large a rank.
[0073] Furthermore, the aforementioned low-rank adaptation module is systematically integrated into the SAM image encoder through a model wrapper class (LoRA_sam). This process first freezes all the original parameters of the image encoder, then iterates through each Transformer block in the encoder. For the multi-head attention module in each target block, its original qkv linear layer is extracted and replaced with the aforementioned constructed low-rank adaptation module. All newly introduced low-rank matrices... All are initialized using a Kaiming uniform distribution, low-rank matrix It is initialized to a zero matrix to ensure that the output of the adaptation layer is zero during the initial training phase, maintaining the original behavior of the model. This method supports adaptation of only a portion of specified Transformer layers and provides the function of saving and loading low-rank weights, facilitating efficient deployment and transfer of the model.
[0074] S22. Construct a small sample dataset.
[0075] To train the low-rank adapted visual basic segmentation model, this step constructs a small, high-quality, hand-annotated dataset.
[0076] Since there is currently no publicly available CME semantic segmentation annotation dataset, this step involves constructing and manually annotating a new dataset based on observations from the LASCO C2 coronagraph. Specifically, images from a specific time period are selected as the training and testing sets. Event annotations are based on the LASCO CME Catalog maintained by the CDAW data center, with each CME event archived in a separate event catalog.
[0077] CME types are classified into four categories according to CDAW's classification standards: Narrow (angle width less than 30°), Limb (angle width between 30° and 120°), Partial Halo (angle width between 120° and 360°), and Halo (angle width of 360°). The annotated images cover the entire corona field of view, from near the edge of the solar disk to the outer edge of the field of view. The image quality varies from sharp to blurry, and the scene complexity ranges from a single CME to multiple CMEs coexisting.
[0078] This embodiment sets up two experimental scenarios: (i) only CME (32 training images, 100 test images); (ii) a mixture of CME and non-CME (40 training images, 125 test images). To ensure sampling fairness and test the model's robustness and generalization ability, each data subset covers almost every month across the entire time range, rather than being concentrated in a few time periods, with each image corresponding to an independent CME event. The ratio of the four types of CMEs in the training set and test set is approximately 1:1:1:1, while the ratio for experiments containing non-CME images is set to 1:1:1:1:1. All images have a spatial resolution of 1024×1024 pixels, and the acquisition time interval is the instrument's nominal sampling rate of 12 minutes.
[0079] Because different CME morphologies exhibit clearer leading-edge structures in different representations, annotators simultaneously referenced the original white-light corona image and the difference image, using the LabelMe tool to manually annotate and generate a 0 / 1 binary PNG mask. During annotation, information such as angular width and position angle from the LASCO CME Catalog was used as a reference to accurately delineate the CME boundaries. Specifically, based on the SOHO CME catalog, multiple difference-enhanced images were cross-referenced. During annotation, the sharp boundaries of the moving leading edge were prioritized based on the running difference image, and then the base difference image was used to determine the contours of the main body and diffuse structures. Simultaneously, the spatial angle parameters of the delineated mask were compared and verified in real-time with the values recorded in the catalog to ensure that each annotation result possessed both physical realism and geometric accuracy.
[0080] S23. Train the visual basic segmentation model that has undergone low-rank adaptation.
[0081] In this embodiment, training is performed cyclically within a set number of rounds. In each training batch, the input image first has features extracted by the SAM image encoder, a process that integrates the frozen original weights with the incremental computation introduced by the trainable low-rank adaptation layer. The model outputs the predicted segmentation results through a lightweight mask decoder, using full-image bounding boxes as cues. Subsequently, the loss between the predicted mask and the ground truth labeled mask is calculated, employing a joint loss function combining Dice loss and cross-entropy loss to simultaneously optimize the overlap of segmented regions and pixel classification accuracy. The optimizer (such as Adam) is configured to update only the parameters of the low-rank adaptation layer in the model, calculating gradients and performing parameter updates via backpropagation, while the remaining large parameters of the SAM model remain frozen, thereby achieving extremely high training efficiency and effectively avoiding overfitting on small datasets.
[0082] To monitor training progress and ensure model generalization ability, a periodic performance evaluation mechanism is embedded during training. Specifically, after a certain number of training epochs (e.g., 3 epochs), the performance of the current model is evaluated on an independent test dataset. The evaluation metrics focus on Mean Intersection over Union (MIoU) and Mean Pixel Accuracy (MPA), which more sensitively reflect the model's segmentation quality of complex CME boundaries. The system continuously tracks the best MIoU score achieved on the test set, and automatically saves the weight parameters of all low-rank adapter layers when this performance reaches a preset excellent threshold or reaches a new high. Finally, the training process produces a fine-tuned model composed of the original frozen SAM weights and the trained low-rank adapter weights. This model can efficiently and accurately complete the automatic segmentation of CME structures with minimal labeled samples.
[0083] S24. Perform segmentation based on the trained visual basic segmentation model to obtain the segmentation mask of the coronal mass ejection.
[0084] The segmentation inference process implemented in this embodiment is an end-to-end automated process. Its core lies in utilizing a pre-trained SAM-LoRA model to perform batch CME structure extraction on one or more input corona images, without any manual interaction. In specific implementation, such as... Figure 3 As shown, the basic SAM model (sam_vit_b) is first loaded and the low-rank adapter weights obtained from training are loaded to form the final fine-tuned model. Then the model is set to evaluation mode and deployed to the computing device.
[0085] For each input image to be processed, the inference process performs the following operations in sequence:
[0086] (1) The image is read from the PIL library and converted into an array, which is then input into the image encoding and feature extraction component;
[0087] (2) The image encoding and feature extraction component automatically calls the image encoder that integrates the LoRA adaptation layer to normalize and extract features from the image, and caches the generated feature embeddings.
[0088] (3) To achieve fully automatic CME segmentation without interaction, a full-image bounding box circumscribed with the image is generated by default as a cue signal. This cue signal and the cached features are input into a lightweight mask decoder, and the decoder outputs the final segmentation logic mask. After binarization (foreground value is 255, background value is 0), the mask is saved as a PNG binary image file by the PIL library.
[0089] S3. Determine image-level labels based on segmentation masks for coronal mass ejections.
[0090] In this step, image-level labels are determined based on the coronal mass ejection (CME) segmentation mask output by the visual basic segmentation model. If the CME segmentation mask is empty, it is determined to be non-CME; otherwise, it is determined to be CME and the CME segmentation mask is output as the image-level label.
[0091] This embodiment uses four metrics to evaluate CME segmentation performance: pixel accuracy (PA), average pixel accuracy (MPA), intersection-over-union ratio (IoU), and average intersection-over-union ratio (MIoU). For segmentation of images containing CME data, using only 32 training images, the segmentation performance results are shown in Table 1:
[0092] Table 1. Segmentation results of CME images
[0093]
[0094] As shown in Table 1, the visual basic segmentation model with low-rank adaptation (SAM-LoRA) achieved an IoU of 0.81 and an MIoU of 0.89 on the test set, indicating that the model can effectively capture the complete structure of CME.
[0095] This embodiment also included an ablation experiment with input channel selection, and the experimental results are shown in Table 2:
[0096] Table 2 Ablation Experiment Results of Input Channel Selection
[0097]
[0098] As shown in Table 2, the pseudo-RGB input (e.g., [BDI, BDI, RDI]) combining the Basic Difference Map (BDI) and the Running Difference Map (RDI) significantly outperforms a single BDI or RDI input. This multi-channel scheme achieves higher performance in PA, MPA, IoU, and MIoU, effectively enhancing the model's sensitivity to changes in CME morphology and position.
[0099] The model performance systematically varies with the CME angular width. Narrow-angle CMEs (<30°) show the lowest IoU (0.73) and MIoU (0.86), mainly due to their small spatial range, where even small boundary errors occupy a large relative area. In larger angular width ranges (30°-120° and 120°-360°), the IoU stabilizes at approximately 0.84, while the MIoU improves to 0.91-0.92, indicating that the model recovers the CME envelope better. For halo-shaped CMEs (360°), the IoU (0.83) remains stable, demonstrating the model's adaptability to complex backgrounds.
[0100] like Figure 4The visualization results show that SAM-LoRA can recover continuous and complete CME structures, especially excelling in recognizing faint white CME envelopes. In examples containing multiple structured CME extensions, the low-rank adapted visual basic segmentation model avoids fragmenting the CME into multiple pieces, maintaining structural integrity, which is crucial for subsequent physical parameter inversion.
[0101] For segmentation of mixed CME and non-CME images, on a mixed dataset containing 40 training images (including 8 non-CME images) and 125 test images, the low-rank adapted visual basic segmentation model achieved near-perfect performance in the binary classification task of CME and non-CME images, with an accuracy of 0.98 and an F1 score of 0.97. It maintained robust segmentation ability on extended datasets and could reliably determine the presence of CME, demonstrating the potential of this method in practical applications.
[0102] Example 2
[0103] In a typical embodiment of the present invention, this embodiment discloses a few-sample coronal mass ejection segmentation system based on low-rank adaptation, comprising:
[0104] The image acquisition module is configured to: acquire a sequence of white light coronagraph images and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information;
[0105] The image segmentation module is configured to: input a multi-channel fused image into a low-rank adapted visual basic segmentation model for segmentation to obtain a segmentation mask for coronal mass ejections; wherein, the visual basic segmentation model fine-tunes the parameters of a pre-trained image encoder using a few-sample dataset containing coronal mass ejection annotations through low-rank matrix factorization; during the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder;
[0106] The output module is configured to determine image-level labels based on a segmentation mask for coronal mass ejections.
[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A few-sample coronal mass ejection segmentation method based on low-rank adaptation, characterized in that, include: Acquire white light coronagraph image sequences and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information; The multi-channel fused image is input into a low-rank adapted visual basic segmentation model for segmentation to obtain a segmentation mask for coronal mass ejections. The visual basic segmentation model uses a low-rank matrix factorization method to fine-tune the parameters of the pre-trained image encoder using a few-sample dataset containing coronal mass ejection annotations. During the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder. Image-level labeling is determined using segmentation masks based on coronal mass ejections; Inserting trainable low-rank adaptation layers into the image encoder specifically includes: inserting low-rank adaptation layers into the query projection matrix and value projection matrix of the multi-head attention module in each Transformer layer in the image encoder part of the visual base segmentation model. The specific process of parameter fine-tuning is as follows: During fine-tuning training, all original parameters of the image encoder of the visual base segmentation model and all parameters of the cue encoder are frozen, and only the parameters of the newly inserted trainable low-rank weight matrix and the lightweight mask decoder are trained. The fine-tuning training process specifically includes: first, freezing all the original parameters of the image encoder and cue encoder of the visual base segmentation model; then, traversing each Transformer block in the image encoder; injecting a trainable low-rank weight matrix into the query projection matrix and value projection matrix of the multi-head attention module of each Transformer block, and supervising the training of the low-rank weight matrix with a few-sample training set and a set loss function; finally, producing a fine-tuned model composed of the original frozen visual base segmentation model weights and the trained low-rank adapter weights; wherein, a periodic performance evaluation mechanism is embedded in the training process, and the performance of the current model is evaluated on an independent test dataset after a certain number of training rounds; the evaluation metrics include average intersection-union ratio and average pixel accuracy; continuously tracking the best average intersection-union score obtained on the test set, and automatically saving the weight parameters of all low-rank adapter layers when the performance reaches a preset excellent threshold or reaches a new high; To achieve fully automatic CME segmentation without interaction, a full-image bounding box circumscribed in the image is generated by default as a cue signal, and a lightweight... of The mask decoder outputs the predicted segmentation result.
2. The method for segmenting few-sample coronal mass ejections based on low-rank adaptation as described in claim 1, characterized in that, Differential fusion processing includes: For a target coronal mass ejection event, the coronal image at the moment of eruption is subtracted from the coronal image at the moment before the eruption to obtain the basic difference map. Subtract the previous frame's corona image from the current frame's corona image to obtain the running difference map; The base difference map and the running difference map are fused to construct a three-channel pseudo-RGB image, resulting in a multi-channel fused image containing both base difference information and running difference information.
3. The method for segmenting few-sample coronal mass ejections based on low-rank adaptation as described in claim 2, characterized in that, A three-channel pseudo-RGB image is constructed by using a base difference image for the first and second channels and a running difference image for the third channel; Alternatively, a three-channel pseudo-RGB image can be constructed by using a running differential image for the first and second channels and a basic differential image for the third channel.
4. The method for segmenting few-sample coronal mass ejections based on low-rank adaptation as described in claim 1, characterized in that, For each original query projection matrix Sum projection matrix Introduce a rank of The trainable low-rank matrix factorization is expressed as: In the formula, This represents the query projection matrix after low-rank adaptation; This represents the value projection matrix after low-rank fitting; and This represents trainable low-rank weights.
5. The method for segmenting few-sample coronal mass ejections based on low-rank adaptation as described in claim 1, characterized in that, Methods for constructing small sample datasets include: Acquire historical observation images from the second camera module of the Large Angle Spectroradiometer on the Solar and Heliospheric Observatory satellite; Based on the cataloging of coronal mass ejection events, images of events containing different types of coronal mass ejections were selected; The selected images were manually annotated at the pixel level to generate binary segmentation masks, which were then divided into training and testing sets containing only coronal mass ejection (CME) scenes and training and testing sets containing mixed CME and non-CME scenes.
6. The method for segmenting few-sample coronal mass ejections based on low-rank adaptation as described in claim 1, characterized in that, Image-level labels are determined based on the segmentation mask of the coronal mass ejection output by the visual basic segmentation model. If the segmentation mask of the coronal mass ejection is empty, it is determined to be a non-coronal mass ejection; otherwise, it is determined to be a coronal mass ejection.
7. A few-sample coronal mass ejection segmentation system based on low-rank adaptation, characterized in that, include: The image acquisition module is configured to: acquire a sequence of white light coronagraph images and perform differential fusion processing to generate a multi-channel fused image containing basic differential information and running differential information; The image segmentation module is configured to: input a multi-channel fused image into a low-rank adapted visual basic segmentation model for segmentation to obtain a segmentation mask for coronal mass ejections; wherein, the visual basic segmentation model fine-tunes the parameters of a pre-trained image encoder using a few-sample dataset containing coronal mass ejection annotations through low-rank matrix factorization; during the fine-tuning process, the original weights of the image encoder and the cue encoder are frozen, and a trainable low-rank adaptation layer is inserted into the image encoder; The output module is configured to: determine image-level labels based on a segmentation mask for coronal mass ejections; Inserting trainable low-rank adaptation layers into the image encoder specifically includes: inserting low-rank adaptation layers into the query projection matrix and value projection matrix of the multi-head attention module in each Transformer layer in the image encoder part of the visual base segmentation model. The specific process of parameter fine-tuning is as follows: During fine-tuning training, all original parameters of the image encoder of the visual base segmentation model and all parameters of the cue encoder are frozen, and only the parameters of the newly inserted trainable low-rank weight matrix and the lightweight mask decoder are trained. The fine-tuning training process specifically includes: first, freezing all the original parameters of the image encoder and cue encoder of the visual base segmentation model; then, traversing each Transformer block in the image encoder; injecting a trainable low-rank weight matrix into the query projection matrix and value projection matrix of the multi-head attention module of each Transformer block, and supervising the training of the low-rank weight matrix with a few-sample training set and a set loss function; finally, producing a fine-tuned model composed of the original frozen visual base segmentation model weights and the trained low-rank adapter weights; wherein, a periodic performance evaluation mechanism is embedded in the training process, and the performance of the current model is evaluated on an independent test dataset after a certain number of training rounds; the evaluation metrics include average intersection-union ratio and average pixel accuracy; continuously tracking the best average intersection-union score obtained on the test set, and automatically saving the weight parameters of all low-rank adapter layers when the performance reaches a preset excellent threshold or reaches a new high; To achieve fully automatic CME segmentation without interaction, a full-image bounding box circumscribed in the image is generated by default as a cue signal, and a lightweight... of The mask decoder outputs the predicted segmentation result.