A method and system for interpreting transverse aeolian ridges on Mars based on a SAM model with few samples

Abstract

The application discloses a Mars transverse wind ridge few-sample remote sensing interpretation method and system based on a SAM model. The method performs contrast stretching preprocessing on original Mars images, constructs a double-branch feature fusion framework based on VIT and CNN to extract and fuse image features, introduces a position coding generator to support input of any size, adopts a fine-tuning SAM strategy of selectively freezing an encoder and a trainable mask decoder, combines LoRA low-rank adaptive optimization calculation, uses a double-branch prompt generation module to fuse labeled and unlabeled data to generate high-precision prompts, finally outputs interpretation results through the mask decoder and performs connected component analysis to realize instance-level labeling. The system comprises image preprocessing, double-branch feature extraction and fusion, image embedding generation, model fine-tuning, prompt embedding generation and interpretation modules. The application effectively improves the recognition accuracy and robustness of Mars transverse wind ridges under few-sample conditions.

Description

Technical Field

[0001] This invention relates to the field of aerospace photogrammetry data processing technology, specifically to a method and system for interpreting small-sample remote sensing data of Martian transverse wind ridges based on the SAM model. Background Technology

[0002] Studying the morphological features of transverse aeolian ridges can help reveal the interaction between Mars' thin atmosphere and its surface, including recent climate change, modern deposition, and erosion processes. However, because transverse aeolian ridges are typically small and scattered, large-scale extraction from remote sensing images using visual interpretation would be extremely time-consuming and labor-intensive.

[0003] Inspired by the tremendous success of deep learning in image processing, an increasing number of studies are applying convolutional neural networks to the automatic identification of Martian aeolian landforms such as transverse wind ridges. However, most existing methods typically rely on a large dataset with meticulous pre-annotation by humans, and the discrete distribution and diverse morphologies of transverse wind ridges further exacerbate the difficulty of constructing such datasets. To address this challenge, some studies have proposed zero-shot transverse wind ridge extraction methods based on the visual foundation model SAM (Segment Anything Model). However, this method directly utilizes the original SAM pre-trained model for classless segmentation and relies on multiple filtering operations involving fixed thresholds, limiting its universality. Furthermore, because the original SAM uses fixed-position encoding, this method necessitates scaling the image to a fixed size (1024×1024), which destroys the detailed information of the original data. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for interpreting Mars transverse wind ridges with few samples based on the SAM model, in order to solve the problems of interference from complex regional features, reduced segmentation accuracy, and the fact that the SAM model can only accept inputs of fixed sizes.

[0005] To achieve the above objectives, the present invention provides a method for few-sample remote sensing interpretation of Martian transverse wind ridges based on the SAM model, comprising the following steps: S1. Perform contrast linear stretching preprocessing on the original digital orthophotos of Mars in the study area to obtain experimental image data, and annotate the transverse wind ridges of Mars in the experimental images to obtain annotation data. S2. Construct a dual-branch image feature fusion framework based on VIT and CNN, and extract the initial image features of the VIT branch from the experimental image data respectively. and initial image features of CNN branches ; S3. Process the initial image features of the acquired CNN branches respectively. and VIT branch initial image features Preprocessing is performed to extract the image features from the preprocessed CNN branches. and VIT branch image features Addition and fusion to obtain initial fusion features This is then fused with the location code to obtain the location code fusion feature. ; S4. Establish a fine-tuned SAM model. The fine-tuned SAM model is obtained by selectively freezing the image encoder, completely freezing the cue encoder, and keeping the mask decoder trained. A low-rank matrix is ​​added between the Transformer blocks of the SAM model using the LoRA fine-tuning strategy. The selectively frozen image encoder refers to taking arbitrary-size input through the unfreeze position encoding generator, extracting multi-scale spatial features through the unfreeze spatial multi-scale adapter, and freezing the rest of the image encoder at the same time. S5. Use Transformer blocks to process dual-stream input features simultaneously, and fuse features by positional encoding. As VIT branch image features, compared with CNN branch image features The dual-stream input features are combined, and the VIT branch image features processed by the Transformer block of the image encoder are then processed by the gating module. and CNN branch image features Dynamic fusion yields image embedding features. ; S6. The dual-branch feature generation module processes the prompt information from the labeled and unlabeled data, and fuses it in the SAM model prompt encoder to obtain the prompt embedding features. ; S7. Use the obtained image embedding features With hints and embedded features The unfrozen mask decoder in the SAM model is trained to obtain the trained model; the Martian transverse wind ridge interpretation results of the study area are output by inputting experimental image data. S8. Perform connected component analysis on the interpretation results of Martian transverse wind ridges in the study area to mark all independent transverse wind ridge objects.

[0006] To optimize the above technical solution, the specific measures also include: In step S2, the initial image features of the VIT branch are extracted from the experimental image data respectively. and initial image features of CNN branches The specific steps include: S2.1 In the dual-branch image feature fusion framework, the VIT branch uses the PatchEmbed0 module to segment the input image into blocks and project them into embedding vectors through convolutional layers, thereby obtaining the initial image features of the VIT branch. ; In the S2.2 dual-branch image feature fusion framework, the CNN branch uses a pre-trained ConvNeXt backbone network to extract multi-scale features through a hierarchical downsampling architecture. After adaptive pooling, channel alignment, and upsampling, the initial image features of the CNN branch are output. .

[0007] Furthermore, in step S3, the initial image features of the acquired CNN branch are... and VIT branch initial image features Preprocessing is performed, and the specific steps include: Initial image features of the CNN branch Upsampling and dimensional ordering are performed so that their spatial size and dimensional order align with the VIT branch image features. Consistent, obtaining the preprocessed CNN branch image features The initial image features of the VIT branch are processed through the Adapter module. Obtain the preprocessed VIT branch image features .

[0008] In step S4, the fine-tuning of the SAM model also employs a LoRA fine-tuning strategy to add a low-rank matrix to the Transformer block of the SAM model. Specific steps include: The location-encoded fusion feature The VIT branch image features are normalized as part of the VIT branch image features. QKV projection is performed, and low-rank adaptive enhancement is applied through LoRA layers. Adaptive average pooling compression is applied to the key K and value V. Scaled dot product attention of query Q and key K is calculated. The attention weights are multiplied by the compressed value V and multi-head feature fusion is performed. After residual connection and MLP processing, the updated VIT branch image features are output. .

[0009] The specific steps of step S5 include: The location encoding fusion feature With the preprocessed CNN branch image features The first Transformer block is input and iteratively processed, and the updated VIT branch image features are output. ; The final VIT branch image features after processing by all Transformer blocks Image features of the final CNN branch Perform splicing to obtain splicing features ; regarding the splicing features Perform convolution operations to compress dimensions and extract interactive features, resulting in compressed features. Based on the compression features A spatially adaptive gated weight map G is generated using an activation function; the gated weight map G is then used to analyze the final branch image features of the VIT. and the CNN branch features Dynamic weighted fusion is performed to obtain image embedding features. .

[0010] In step S6, the dual-branch feature generation module includes: The labeled data branch is used to perform image enhancement and intensity adjustment processing on the input raw image and labeled data; and to generate positive sample cue points and positive sample cue box information based on the enhanced labeled data. The unlabeled data branch is used to generate a superpixel segmentation map from the input image using the SLIC superpixel segmentation module; the superpixel segmentation map is processed using the GMM clustering analysis module to generate a binary classification image; the binary classification image is subjected to morphological post-processing to generate a binary pseudomask; and positive sample cue points and positive sample cue box information are generated based on the binary pseudomask.

[0011] The specific steps of step S5 include: The initial fusion feature is obtained by fusing it with the location code. With the preprocessed CNN branch image features Input a series of Transformer blocks and iteratively process them; output the updated VIT branch image features. ; The final VIT branch image features after processing by all Transformer blocks Image features of the final CNN branch Perform splicing to obtain splicing features The concatenated features are subjected to convolution operations to compress dimensionality and extract interactive features, resulting in compressed features. Based on the compressed features, a spatially adaptive gated weight map G is generated using an activation function; the gated weight map G is then used to analyze the final branch image features of the VIT. and the CNN branch features Dynamic weighted fusion is performed to obtain image embedding features. .

[0012] Furthermore, in step S4, the fine-tuning of the SAM model also employs a LoRA fine-tuning strategy to add a low-rank matrix to the Transformer block of the SAM model, specifically including: The VIT branch image features are normalized, and the normalized VIT branch image features are then processed. QKV projection is performed, and low-rank adaptive enhancement is applied through LoRA layers. Adaptive average pooling compression is applied to the key K and value V. Scaled dot product attention of query Q and key K is calculated. The attention weights are multiplied by the compressed value V and multi-head feature fusion is performed. After residual connection and MLP processing, the updated VIT branch image features are output. .

[0013] The dual-branch feature generation module in step S6 includes: The labeled data branch is used to perform image enhancement and intensity adjustment processing on the input raw image and binary labeled data; and to generate positive sample cue points and positive sample cue box information based on the enhanced labeled data. The unlabeled data branch is used to generate a superpixel segmentation map from the input image using the SLIC superpixel segmentation module; the superpixel segmentation map is processed using the GMM clustering analysis module to generate a binary classification image; the binary classification image is subjected to morphological post-processing to generate a binary pseudomask; and positive sample cue points and positive sample cue box information are generated based on the binary pseudomask.

[0014] The labeled data branch inputs the original labeled data, including the digital orthophoto of the Martian transverse wind ridge and its corresponding binary labeled data; the unlabeled data branch inputs only the digital orthophoto of the Martian transverse wind ridge. Further, in step S6, the prompt information from the labeled and unlabeled data is processed and fused in the SAM model prompt encoder to obtain prompt embedding features. The specific steps include: The cue point information is encoded using sine-cosine positional encoding; the bounding box information is encoded using vertex mean pooling; a type embedding module assigns type identifiers to the processed cue point and bounding box features; and a feature fusion module concatenates the two types of features along the channel dimension and performs a linear projection transformation to obtain the final cue embedding features. .

[0015] As another important technical solution, this invention also provides a few-sample remote sensing interpretation system for Mars transverse wind ridges based on the SAM model, comprising: The image preprocessing module is used to perform contrast linear stretching preprocessing on the original digital orthophotos of Mars in the study area to obtain experimental image data, and to annotate the transverse wind ridges of Mars in the experimental images to obtain annotation data. The dual-branch feature extraction and fusion module is used to build a dual-branch image feature fusion framework based on VIT and CNN, extracting initial image features of the VIT branch from the experimental image data respectively. and initial image features of CNN branches ; respectively, the initial image features of the acquired CNN branches and VIT branch initial image features Preprocessing is performed to extract the image features from the preprocessed CNN branches. and VIT branch image features Addition and fusion to obtain initial fusion features This is then fused with the location code to obtain the location code fusion feature. ; The image embedding feature generation and fusion module is used to process dual-stream input features simultaneously using the Transformer block, and to fuse features by position encoding. As VIT branch image features, compared with CNN branch image features The dual-stream input features are combined, and the VIT branch image features processed by the Transformer block of the image encoder are then processed by the gating module. and CNN branch image features Dynamic fusion yields image embedding features. ; The cue embedding feature generation and fusion module processes the cue information from labeled and unlabeled data using the dual-branch feature generation module, and then fuses it in the SAM model cue encoder to obtain cue embedding features. ; The model fine-tuning and fusion module is used to build a fine-tuned SAM model. This fine-tuned SAM model is obtained through selectively freezing the image encoder, completely freezing the cue encoder, and maintaining the mask decoder during training. A LoRA fine-tuning strategy is employed to add low-rank matrices between the Transformer blocks of the SAM model. Specifically, selectively freezing the image encoder involves taking an arbitrary-size input by unfreezing the positional encoder generator and extracting multi-scale spatial features through an unfrozen spatial multi-scale adapter, while simultaneously freezing the rest of the image encoder. The obtained image embedding features are then used... With hints and embedded features Train the unfrozen mask decoder in the SAM model to obtain the trained model; The model interpretation module is used to output the interpretation results of Martian transverse wind ridges in the study area by inputting experimental image data; and to perform connected component analysis on the interpretation results of Martian transverse wind ridges in the study area to mark all independent transverse wind ridge objects.

[0016] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a method for interpreting Mars transverse wind ridges with few samples based on the SAM model as described above.

[0017] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute a few-sample remote sensing interpretation method for Mars transverse wind ridges based on the SAM model as described above.

[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention proposes for the first time a few-sample fine-tuning interpretation method based on the SAM model for transverse wind ridges on Mars. By using a specific fine-tuning strategy to adjust the structure of the SAM model, it makes it suitable for few-sample remote sensing interpretation tasks of transverse wind ridges on Mars, significantly reducing the dependence on a large amount of labeled data.

[0019] 2. This invention employs a location encoding generator, a spatial multi-scale adapter, and a multi-stage feature fusion strategy, which can effectively improve the remote sensing interpretation accuracy of Martian transverse wind ridges under conditions of few samples.

[0020] 3. This invention combines VIT and CNN dual-branch feature fusion and gating mechanisms to enhance the model's ability to perceive and fuse multi-scale features.

[0021] 4. This invention employs a LoRA fine-tuning strategy to reduce computational overhead and improve training efficiency.

[0022] 5. This invention utilizes a dual-branch hint generation module to fully mine labeled and unlabeled data information, generating high-precision hints, and further improving segmentation accuracy and generalization ability. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall process framework of the method of the present invention.

[0024] Figure 2 This is a schematic diagram of multi-stage feature fusion in this invention.

[0025] Figure 3 This is a schematic diagram of the dual-stream prompt generation module in this invention.

[0026] Figure 4 This is a schematic diagram illustrating the position of the low-rank LORA matrix in this invention.

[0027] Figure 5 This is a schematic diagram of the position encoding generator in this invention.

[0028] Figure 6 This is a schematic diagram of the spatial multi-scale adapter in this invention. Detailed Implementation

[0029] The present invention will be further described in detail below through specific embodiments, but it should not be construed as limiting the scope of the subject matter of the present invention to the following embodiments. All technologies implemented based on the above content of the present invention fall within the scope of the present invention.

[0030] like Figure 1 As shown, this invention provides a method for interpreting Mars transverse wind ridges with few samples based on the SAM model, comprising the following steps: S1. Perform contrast linear stretching preprocessing on the original digital orthophotos of Mars in the study area to obtain experimental image data, and annotate the transverse wind ridges of Mars in the experimental images to obtain annotation data.

[0031] In some implementations, the original digital orthophoto of the study area is linearly stretched by 2%, as shown in the following formula:

[0032] Wherein, original_value represents the pixel value of the original image; min_val represents the threshold selected as the "darkest effective pixel" in the original image, serving as the reference zero point for contrast stretching; and max_val represents the threshold selected as the "brightest effective pixel" in the original image, serving as the reference for the maximum value of contrast stretching.

[0033] S2. Construct a dual-branch image feature fusion framework based on VIT and CNN, and extract the initial image features of the VIT branch from the experimental image data respectively. and initial image features of CNN branches .

[0034] In some implementations, step S2.1 involves the VIT branch processing the input image using the PatchEmbed0 module. A convolutional layer (kernel_size=16, stride=8) is used to segment the image into blocks, and each block is projected into the embedding space. Each block is projected as a 768-dimensional vector, thus obtaining the initial image features for the VIT branch. .

[0035] Step S2.2: The CNN branch uses a pre-trained ConvNeXt backbone network to extract multi-scale features through a four-stage hierarchical downsampling architecture. After adaptive pooling and channel alignment, it outputs a feature map that matches the dimension of the VIT branch, thus obtaining the initial image features of the CNN branch. It includes the following sub-steps: Step S2.2.1: First, use the initial downsampling layer of the ConvNeXt network to obtain the initial convolutional layer image features. .

[0036]

[0037] Where X represents the original Mars image, X , .

[0038] Step S2.2.2: Use the four-stage hierarchical downsampling architecture of the ConvNeXt branch to extract image features, including: first stage high-resolution feature extraction, second stage initial downsampling, third stage intermediate semantic extraction, and fourth stage deep feature extraction.

[0039] In the first stage: the input initial convolutional layer image features are processed using 3 ConvNeXt blocks. Each block contains 7×7 depthwise separable convolutions, layer normalization, and channel recalibration (two linear layers).

[0040]

[0041]

[0042]

[0043]

[0044] Where, when i=1, = When i=3, = The above operations are then performed to obtain the image features of the first stage. :

[0045] In the second stage, the image features obtained in the first stage are... After downsampling, the image features are obtained through three ConvNeXt blocks. :

[0046]

[0047] in, .

[0048] In the third stage, the image features acquired in the second stage are... After downsampling, the image features are obtained through nine ConvNeXt blocks. ;

[0049]

[0050] in, .

[0051] In the fourth stage, the image features acquired in the third stage are... After downsampling, the image features are obtained through three ConvNeXt blocks. :

[0052]

[0053] in, .

[0054] Step S2.2.3: To facilitate subsequent fusion with the VIT branch image features, adaptive average pooling is used to compress the 32×32 features to 8×8, extracting high-level semantic information. Then, a 1×1 convolution operation is used to reconstruct the feature space, maintaining the number of channels in the CNN branch image features consistent with the VIT branch. Finally, bilinear interpolation is used to restore the image resolution to match that of the VIT branch, resulting in... .

[0055]

[0056]

[0057]

[0058] in, .

[0059] S3. Process the initial image features of the acquired CNN branches respectively. and VIT branch initial image features Preprocessing is performed to extract the image features from the preprocessed CNN branches. and VIT branch image features Addition and fusion to obtain initial fusion features This is then fused with the location code to obtain the location code fusion feature. .

[0060] In some implementations, step S3.1 involves analyzing the features of the CNN branches. Upsampling is performed and the dimensional order of the CNN branches is adjusted so that the feature space size and dimensional order of the CNN branches are consistent with those of the VIT branches, thus obtaining the preprocessed CNN branch features. .

[0061] Step S3.2: Initial image features for the VIT branch The Adapter module processes the data and adjusts it to a representation space more suitable for the current image task, resulting in preprocessed VIT branch image features. :

[0062] Step S3.3: The preprocessed VIT branch image features... Features of the upsampled CNN branch images Add them together to obtain the initial fusion features. :

[0063] Step S3.4, as follows Figure 5 As shown, in the image encoder, the position code P is dynamically generated by unfreezing the position code generator (PEG) and using depthwise separable convolution. The single-layer structure contains only one depthwise convolution layer. The convolution kernel automatically matches the local positional relationship of the input size by sliding to capture the output size. It only needs to learn the spatial positional relationship and supports input of arbitrary resolution.

[0064]

[0065] Step S3.5 will use the initial fusion features This is then fused with the location code to obtain the location code fusion feature. :

[0066] S4. Establish a fine-tuned SAM model, which is obtained through a strategy of selectively freezing the image encoder, completely freezing the cue encoder, and keeping the mask decoder trainable. The frozen image encoder adds a position encoding generator to learn spatial positional relationships and support data input of arbitrary size. At the same time, the frozen image encoder uses a spatial multi-scale adapter to extract multi-scale spatial features required for semantic segmentation.

[0067] Step S4.1, as follows Figure 6As shown, in the image encoder, by unfreezing the Spatial Multi-Scale Adapter (SM-AdaptFormer), five types of receptive fields are obtained through two provided convolutional layers with kernel sizes of 1×1 and 3×3, as well as dilated convolutions (r=12, r=24, and r=32). To enable the model to understand the details and contextual information of the image, thereby gaining the ability to perceive Martian lateral wind-formed ridge structures of various sizes and making more accurate and robust predictions, the final multi-scale features are obtained by summing these receptive field features from small to large. The dilated convolutions are specifically as follows:

[0068] Where x is the input feature map with dimensions B×C×H×W; y i `r` is the value of the output feature map at position `i`, and `r` is the dilation rate of the dilated convolution, controlling the sampling interval of the convolution kernel. When `r` is small, dense sampling captures local details; when `r` is large, sparse sampling expands the receptive field and captures global context. k These are learnable convolutional kernel weights that aggregate features at different locations and are optimized through training to capture multi-scale features; i and k are both position indices, where i represents the coordinates of the output position and k represents the relative position offset within the convolutional kernel.

[0069] Step S4.2, as follows Figure 4 As shown, a LORA fine-tuning strategy is employed to add low-rank matrices between Transformer blocks of SAM to reduce the significant computational overhead when training on 1024×1024 data. This includes the following sub-steps: Step S4.2.1: Process the input features in LayerNorm1 of the Transformer block. (here) These are the input features of each Transformer Block, in the first Transformer Block. for In all subsequent Transformer blocks (All outputs from the previous block are normalized to obtain the normalized VIT features) :

[0070] Step S4.2.2: Normalize the VIT features An attention module enhanced with LORA is used, and QKV projection is employed to transform the input VIT features through a linear transformation. Simultaneously, the projection is into three sets of vectors: query (Q), key (K), and value (V).

[0071]

[0072]

[0073] Where X represents the input feature and W represents different parameter projection matrices.

[0074] Step S4.2.3: Apply low-rank adaptive enhancement to Q, K, and V respectively through the LoRA layer, that is, add the result of a low-rank transformation to obtain the low-rank enhancement vector.

[0075]

[0076]

[0077] Where adapter refers to low-rank adapter, and the dimensions of q, k, and v are (B*). H, W, ).

[0078] Step S4.2.4: Expand the spatial dimensions (H and W) of Q into sequence dimensions (L=HW), forming (B* H*W The shape of (B*) is chosen to facilitate attention computation. The features of K and V are adaptively average-pooled and compressed to a 3x3 spatial size, forming (B*) ,9, (The shape of)

[0079] Step S4.2.5: Calculate the similarity between the query vector (Q) and the key vector (K) by scaling the dot product correlation, while ensuring numerical stability through a scaling factor. The calculation formula is as follows:

[0080] Where Q and K are the query vector and key vector, respectively. The dimension representing the key vector is obtained. The shape is (B*) ,H*W,9).

[0081] Step S4.2.6: Softmax normalize the original attention score (similarity matrix) obtained by scaling dot product correlation calculation to finally obtain the attention weight matrix G of spatial perception probability distribution.

[0082]

[0083] The resulting shape of G is (B*) ,H*W,9).

[0084] Step S4.2.7: Multiply the normalized attention weight matrix P with the compressed value vector (V) to perform feature weighting and obtain the output of each head.

[0085]

[0086] Among them, the obtained The shape is (B*) H*W ).

[0087] Step S4.2.8: Combine the outputs of multiple attention heads Through multi-head feature fusion, the features are merged into the spatial form of the original input feature map, i.e., (B, H, W, C). First, the output of each head is reshaped into (B, H, W, C). L Since L=HW, it is actually first reshaped into (B, H, W, Next, we replace the dimensions, moving the attention head dimension before the channel dimension, resulting in (B, H, W). , Finally, the last two dimensions (i.e. and The merger forms a new channel dimension C= * Finally, the image features of (B, H, W, C) are obtained. .

[0088] Step S4.2.9: The image features obtained by fusing multi-head features through the first residual connection operation. Compared with the original features input to the Attention module The features are added together to preserve the original information and promote gradient flow, resulting in the output features. .

[0089]

[0090] Step S4.2.10: After the first residual connection, a second normalization (LayerNorm2) is performed, followed by MLP processing and another residual connection to obtain the output features of the Transformer block. :

[0091]

[0092] in, This represents the output feature of the j-th Transformer block finely tuned by LORA.

[0093] like Figure 2 As shown in Figure S5, the Transformer block is used to process the dual-stream input features simultaneously, and the features are fused by positional encoding. As VIT branch image features, compared with CNN branch image features The dual-stream input features are combined, and the VIT branch image features processed by the Transformer block of the image encoder are then processed by the gating module. and CNN branch image features Dynamic fusion yields image embedding features. .

[0094] In some implementations, during the intra-block feature fusion stage, each Transformer block processes dual-stream features simultaneously. For the first Transformer block, its input is... and .Right now:

[0095] This represents the output fused feature of the j-th block; each Transformer block receives two features and outputs the updated two features, and the output after passing through the j-th ParaBlock is as follows:

[0096] Obtain VIT branch image features .

[0097] In some implementations, such as Figure 2 As shown, the final fusion module uses an added gating module to dynamically fuse the VIT branch features and CNN branch features processed by the 12-layer Transformer block of the image encoder, which includes the following sub-steps: Step S5.1: Perform feature stitching to obtain the stitched feature image. :

[0098] in, .

[0099] Step S5.2: Perform convolutional feature compression to reduce the feature dimensionality and computational cost, while extracting interactive features to obtain compressed features. .

[0100]

[0101] in, .

[0102] Step S5.3: Based on compression features Gated weights G are generated through activation functions, resulting in a spatially adaptive weight heatmap. High-value regions prioritize VIT features, while low-value regions prioritize CNN features.

[0103]

[0104] Where σ is the Sigmoid function, with an output range of [0, 1].

[0105] Step S5.4, and Using gated weights G, the two features are dynamically weighted and fused. This automatically enhances the CNN's detailed features in shaded areas and strengthens ViT's global consistency in flat areas, resulting in the final image embedding features. .

[0106]

[0107] like Figure 3 As shown, S6 uses a dual-branch feature generation module to generate point hints and bounding box hints based on labeled data and unlabeled data respectively, forming high-precision hints. The two types of hints are then fused in the hint encoder to obtain hint embedding features. The dual-branch feature generation module includes a labeled data branch and an unlabeled data branch. The original labeled data is input into the labeled data branch, including the digital orthophoto of the Martian transverse wind ridge and its corresponding binary labeled data; the unlabeled data branch only contains the digital orthophoto of the Martian transverse wind ridge.

[0108] Step S6.1, Annotated Data Branch: First, the input image and binary annotation data are simultaneously processed by the image enhancement module for rotation (90°, 180°, 270°), mirroring (horizontal axis mirroring, vertical axis mirroring), and intensity adjustment, to obtain the output enhanced dataset (original image + 3 rotations + 2 mirrors + 3 intensity adjustments):

[0109] in Represents the original image and binary labeled data; This represents the enhanced image and the binary labeled data.

[0110] Preferably, the prompt generation module is used to... Generate point and polygon hints, that is, from Generate 1, 2, and 3 annotation points respectively as positive sample prompt points; and generate The smallest bounding rectangle of the positive sample is used as the positive sample prompt information.

[0111] Step S6.2: For unlabeled data branches, first process them using the SLIC superpixel segmentation module to initially generate a superpixel segmentation map:

[0112] in, X represents the generated superpixel segmentation map; X represents the input image. The principle of SLIC superpixel segmentation is as follows:

[0113] in, The core distance represents the superpixel clustering, and the metric comprehensively measures the color similarity and spatial density between pixels. The smaller the value, the higher the degree of matching between the pixel and the current cluster center; Represents the distance in the CIELAB color space, used to analyze the color similarity between pixels; Represents the spatial distance between pixel coordinates, used to analyze the spatial proximity between pixels; This represents the balancing weight.

[0114] CIELAB color space distance The calculation formula is as follows:

[0115] Where (l, a, b) represents the coordinates of the pixel in the CIELAB color space; i and j represent the two pixels to be compared.

[0116] Pixel coordinate space distance The calculation formula is as follows:

[0117] Where (X, Y) represents the coordinates of the pixel in the image plane.

[0118] Preferably, the GMM clustering analysis module is used to analyze the superpixel segmentation map initially generated by the SLIC superpixel segmentation module. Processing is performed to obtain a binary classification image. :

[0119] The parameters of the Gaussian Mixture Model (GMM) are defined as follows:

[0120] Where N=2 (two types: lateral wind ridges and background); Represents mixed weights; Represents the mean vector; This represents the covariance matrix.

[0121] Preferably, binary classification images Perform morphological closing operations to fill small holes; simultaneously remove isolated regions smaller than 100 saturation points to generate a binary pseudomask. .

[0122] Preferably, from a binary pseudomask Generate point and polygon hints, i.e., from The above generates 1, 2, and 3 annotation points as positive sample prompts respectively; and generates The smallest bounding rectangle of the positive sample is used as the positive sample prompt information.

[0123] Step S6.3: Input the cue point and cue box information generated by the dual branches into the cue encoder of SAM, and fuse the two types of cue information in the cue encoder to generate cue embedding features. .

[0124] First, the tooltip information and box information are encoded: The sine-cosine position encoding module encodes the location of the prompt point information, converting the two-dimensional coordinates into high-dimensional learnable features while preserving the spatial relative positional relationships. The formula is as follows:

[0125] in, Represents normalized coordinate values. Represents the feature dimension.

[0126] The vertex mean pooling module performs polygon encoding on the bounding box information, compressing the polygon vertex information into a single feature vector while preserving the overall shape features of the study area. The formula is as follows:

[0127] Where N represents the number of vertices in the polygon. This represents the coordinates of the k-th vertex.

[0128] Preferably, a type embedding module is used to assign corresponding types to the processed features to distinguish between point hints and region hints, enabling the model to understand that points represent key locations and boxes represent the overall area, and ultimately encode the features as... and

[0129]

[0130]

[0131] Preferably, the feature fusion module performs channel dimension concatenation and then uses linear projection transformation to align the dimensions of the two features, resulting in the final cue embedding feature. :

[0132]

[0133] S7. Using the obtained image embedding features and the obtained cue embedding features, train the unfrozen mask decoder in the SAM model to obtain the trained model and generate the interpretation results of the Martian transverse wind ridge in the study area. S8. Perform connected component analysis on the interpretation results of Martian transverse wind ridges in the study area, and mark all independent transverse wind ridge instances, including the following sub-steps: Step S8.1: Let I be a binary image of size H×W, I: Ω→(0,1). Where Ω is the image network. I(p)=1 represents a foreground pixel, and I(p)=0 represents a background pixel.

[0134] Neighborhood definition:

[0135] Step S8.2: Perform the first round of mapping (label mapping) using the following formula.

[0136] in, This represents the processed neighboring pixels, that is, the neighboring pixels that are located before p in the scan order; This represents a new label, and is usually the smallest positive integer that is not currently in use.

[0137] Step S8.3, let the disjoint-set data structure be... When multiple labels are found in the neighborhood, a merging operation is performed:

[0138] in, The temporary label representing the first mapping of q; Represents the tag value; This represents the unbounded set union operation.

[0139] Step S8.4: Use the following formula to unify the labels (second scan):

[0140] Step S8.5, set To remove duplicates and combine all the sets of root labels, and then sort r in order, define a remapping function:

[0141] The final label matrix is ​​updated as follows:

[0142] The final output is a set of connected components. ,in, .

[0143] In another embodiment of the present invention, a few-sample remote sensing interpretation system for Mars transverse wind ridges based on the SAM model is proposed, comprising: The image preprocessing module is used to perform contrast linear stretching preprocessing on the original digital orthophotos of Mars in the study area to obtain experimental image data, and to annotate the transverse wind ridges of Mars in the experimental images to obtain annotation data. The dual-branch feature extraction and fusion module is used to build a dual-branch image feature fusion framework based on VIT and CNN, extracting initial image features of the VIT branch from the experimental image data respectively. and initial image features of CNN branches ; respectively, the initial image features of the acquired CNN branches and VIT branch initial image features Preprocessing is performed to extract the image features from the preprocessed CNN branches. and VIT branch image features Addition and fusion to obtain initial fusion features This is then fused with the location code to obtain the location code fusion feature. ; The image embedding feature generation and fusion module is used to process dual-stream input features simultaneously using the Transformer block, and to fuse features by position encoding. As VIT branch image features, compared with CNN branch image features The dual-stream input features are combined, and the VIT branch image features processed by the Transformer block of the image encoder are then processed by the gating module. and CNN branch image features Dynamic fusion yields image embedding features. ; The cue embedding feature generation and fusion module processes the cue information from labeled and unlabeled data using the dual-branch feature generation module, and then fuses it in the SAM model cue encoder to obtain cue embedding features. ; The model fine-tuning and fusion module is used to build a fine-tuned SAM model. This fine-tuned SAM model is obtained through selectively freezing the image encoder, completely freezing the cue encoder, and maintaining the mask decoder during training. A LoRA fine-tuning strategy is employed to add low-rank matrices between the Transformer blocks of the SAM model. Specifically, selectively freezing the image encoder involves taking an arbitrary-size input by unfreezing the positional encoder generator and extracting multi-scale spatial features through an unfrozen spatial multi-scale adapter, while simultaneously freezing the rest of the image encoder. The obtained image embedding features are then used... With hints and embedded features Train the unfrozen mask decoder in the SAM model to obtain the trained model; The model interpretation module is used to output the interpretation results of Martian transverse wind ridges in the study area by inputting experimental image data; and to perform connected component analysis on the interpretation results of Martian transverse wind ridges in the study area to mark all independent transverse wind ridge objects.

[0144] In another embodiment of the present invention, an electronic device is proposed, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a method for interpreting Mars transverse wind ridges with few samples based on SAM model fine-tuning as described above.

[0145] In another embodiment of the present invention, a computer-readable storage medium is provided storing a computer program that causes a computer to execute a few-sample remote sensing interpretation method for Mars transverse wind ridges based on SAM model fine-tuning, as described above.

[0146] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0147] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent substitutions, and improvements made by those skilled in the art to the above embodiments without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for interpreting Mars transverse wind ridges with few samples based on the SAM model, characterized in that, Includes the following steps: S1. Perform contrast linear stretching preprocessing on the original digital orthophotos of Mars in the study area to obtain experimental image data, and annotate the transverse wind ridges of Mars in the experimental images to obtain annotation data. S2. Construct a dual-branch image feature fusion framework based on VIT and CNN, and extract the initial image features of the VIT branch from the experimental image data respectively. and initial image features of CNN branches ; S3. Process the initial image features of the acquired CNN branches respectively. and VIT branch initial image features Preprocessing is performed to extract the image features from the preprocessed CNN branches. and VIT branch image features Addition and fusion to obtain initial fusion features This is then fused with the location code to obtain the location code fusion feature. ; Among them, the initial image features of the CNN branch Upsampling and dimensional ordering are performed so that their spatial size and dimensional order align with the VIT branch image features. Consistent, obtaining the preprocessed CNN branch image features The initial image features of the VIT branch are processed through the Adapter module. Obtain the preprocessed VIT branch image features ; S4. Establish a fine-tuned SAM model. The fine-tuned SAM model is obtained by selectively freezing the image encoder, completely freezing the cue encoder, and keeping the mask decoder trained. A low-rank matrix is ​​added between the Transformer blocks of the SAM model using the LoRA fine-tuning strategy. The selectively frozen image encoder refers to taking arbitrary-size input through the unfreeze position encoding generator, extracting multi-scale spatial features through the unfreeze spatial multi-scale adapter, and freezing the rest of the image encoder at the same time. S5. Use Transformer blocks to process dual-stream input features simultaneously, and fuse features by positional encoding. As VIT branch image features, compared with CNN branch image features The dual-stream input features are combined, and the VIT branch image features processed by the Transformer block of the image encoder are then processed by the gating module. and CNN branch image features Dynamic fusion yields image embedding features. ; S6. The dual-branch feature generation module processes the prompt information from the labeled and unlabeled data, and fuses it in the SAM model prompt encoder to obtain the prompt embedding features. ; S7. Use the obtained image embedding features With hints and embedded features The unfrozen mask decoder in the SAM model is trained to obtain the trained model; the Martian transverse wind ridge interpretation results of the study area are output by inputting experimental image data. S8. Perform connected component analysis on the interpretation results of Martian transverse wind ridges in the study area to mark all independent transverse wind ridge objects.

2. The method for interpreting Mars transverse wind ridges with few samples based on the SAM model according to claim 1, characterized in that: In step S2, the initial image features of the VIT branch are extracted from the experimental image data respectively. and initial image features of CNN branches The specific steps include: S2.1 In the dual-branch image feature fusion framework, the VIT branch uses the PatchEmbed module to segment the input image into blocks through convolutional layers and project them into embedding vectors to obtain the initial image features of the VIT branch. ; In the S2.2 dual-branch image feature fusion framework, the CNN branch uses a pre-trained ConvNeXt backbone network to extract multi-scale features through a hierarchical downsampling architecture. After adaptive pooling, channel alignment, and upsampling, the initial image features of the CNN branch are output. .

3. The method for interpreting Mars transverse wind ridges with few samples based on the SAM model according to claim 1, characterized in that: In step S4, the fine-tuning of the SAM model also employs a LoRA fine-tuning strategy to add a low-rank matrix to the Transformer block of the SAM model. Specific steps include: The location-encoded fusion feature The VIT branch image features are normalized as part of the VIT branch image features. QKV projection is performed, and low-rank adaptive enhancement is applied through LoRA layers. Adaptive average pooling compression is applied to the key K and value V. Scaled dot product attention of query Q and key K is calculated. The attention weights are multiplied by the compressed value V and multi-head feature fusion is performed. After residual connection and MLP processing, the updated VIT branch image features are output. ;in, This represents the output feature of the j-th Transformer block finely tuned by LORA.

4. The method for interpreting Mars transverse wind ridges with few samples based on the SAM model according to claim 1, characterized in that: The specific steps of step S5 include: The location encoding fusion feature With the preprocessed CNN branch image features The first Transformer block is input and iteratively processed, and the updated VIT branch image features are output. ; The final VIT branch image features after processing by all Transformer blocks Image features of the final CNN branch Perform splicing to obtain splicing features ; regarding the splicing features Perform convolution operations to compress dimensions and extract interactive features, resulting in compressed features. Based on the compression features A spatially adaptive gated weight map G is generated using an activation function; the gated weight map G is then used to analyze the final branch image features of the VIT. and the CNN branch features Dynamic weighted fusion is performed to obtain image embedding features. .

5. A method for interpreting Mars transverse wind ridges with few samples based on the SAM model according to claim 1, characterized in that: In step S6, the dual-branch feature generation module includes: The labeled data branch is used to perform image enhancement and intensity adjustment processing on the input raw image and labeled data; and to generate positive sample cue points and positive sample cue box information based on the enhanced labeled data. The unlabeled data branch is used to generate a superpixel segmentation map from the input image using the SLIC superpixel segmentation module; the superpixel segmentation map is processed using the GMM clustering analysis module to generate a binary classification image; the binary classification image is subjected to morphological post-processing to generate a binary pseudomask; and positive sample cue points and positive sample cue box information are generated based on the binary pseudomask.

6. The method for interpreting Mars transverse wind ridges with few samples based on the SAM model according to claim 5, characterized in that: In step S6, the prompt information from the labeled and unlabeled data is processed and fused in the SAM model prompt encoder to obtain prompt embedding features. The specific steps include: The cue point information is encoded using sine-cosine positional encoding; the bounding box information is encoded using vertex mean pooling; a type embedding module assigns type identifiers to the processed cue point and bounding box features; and a feature fusion module concatenates the two types of features along the channel dimension and performs a linear projection transformation to obtain the final cue embedding features. .

7. A remote sensing interpretation system for Mars transverse wind ridges based on the SAM model with few samples, characterized in that, include: The image preprocessing module is used to perform contrast linear stretching preprocessing on the original digital orthophotos of Mars in the study area to obtain experimental image data, and to annotate the transverse wind ridges of Mars in the experimental images to obtain annotation data. The dual-branch feature extraction and fusion module is used to build a dual-branch image feature fusion framework based on VIT and CNN, extracting initial image features of the VIT branch from the experimental image data respectively. and initial image features of CNN branches ; respectively, the initial image features of the acquired CNN branches and VIT branch initial image features Preprocessing is performed to extract the image features from the preprocessed CNN branches. and VIT branch image features Addition and fusion to obtain initial fusion features This is then fused with the location code to obtain the location code fusion feature. ; The image embedding feature generation and fusion module is used to process dual-stream input features simultaneously using the Transformer block, and to fuse features by position encoding. As VIT branch image features, compared with CNN branch image features The dual-stream input features are combined, and the VIT branch image features processed by the Transformer block of the image encoder are then processed by the gating module. and CNN branch image features Dynamic fusion yields image embedding features. ; The cue embedding feature generation and fusion module processes the cue information from labeled and unlabeled data using the dual-branch feature generation module, and then fuses it in the SAM model cue encoder to obtain cue embedding features. ; The model fine-tuning and fusion module is used to build a fine-tuned SAM model. The fine-tuned SAM model is obtained by selectively freezing the image encoder, completely freezing the cue encoder, and keeping the mask decoder trained. The LoRA fine-tuning strategy is used to add low-rank matrices between the Transformer blocks of the SAM model. The selective freezing of the image encoder refers to taking arbitrary-size input by unfreezing the position encoder generator, extracting multi-scale spatial features by unfreezing the spatial multi-scale adapter, and freezing the rest of the image encoder. Use the obtained image embedding features With hints and embedded features Train the unfrozen mask decoder in the SAM model to obtain the trained model; The model interpretation module is used to output the interpretation results of Martian transverse wind ridges in the study area by inputting experimental image data; and to perform connected component analysis on the interpretation results of Martian transverse wind ridges in the study area to mark all independent transverse wind ridge objects.

8. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a method for interpreting Mars transverse wind ridges with few samples based on SAM model fine-tuning as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program causes the computer to execute a method for interpreting Mars transverse wind ridges with few samples, based on SAM model fine-tuning, as described in any one of claims 1 to 6.

Images