Text-guided semi-supervised pseudo-labeling method for ocean remote sensing coastline segmentation

Abstract

The present invention belongs to the field of smart ocean and remote sensing image processing technology, and discloses a method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling. The method comprises the following steps: collecting ocean remote sensing coastline image data, dividing the data into labeled and unlabeled data, and constructing text prompts; constructing a segmentation model containing a shared image encoder, a text encoder, two decoders, and a pseudo-label calibration module. These components collaboratively extract image and text features, generate pseudo-labels and prediction masks, and optimize the pseudo-labels through uncertainty calibration; training the model in supervised and unsupervised stages, summing the loss function values ​​of the two stages and back-propagating the optimization model; and finally, achieving accurate segmentation of ocean remote sensing coastline images based on the trained model. This method utilizes text guidance and semi-supervised pseudo-labeling to effectively improve the efficiency and accuracy of cross-regional ocean remote sensing coastline segmentation and reduce reliance on large amounts of labeled data.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart ocean and remote sensing image processing, and particularly relates to an ocean remote sensing coastline segmentation method based on text-guided semi-supervised pseudo-labeling. Background Art

[0002] Accurate coastline segmentation in remote sensing imagery is a core technical requirement in fields such as marine environmental monitoring, coastal resource management, and disaster early warning. Traditional coastline segmentation methods rely primarily on fully supervised deep learning algorithms, training segmentation networks with large amounts of manually annotated high-resolution remote sensing data to extract pixel-level coastline contours. However, the dynamic complexity of coastline morphology, influenced by multiple factors such as tidal fluctuations and human activity, and the presence of interfering factors such as cloud cover and uneven lighting in remote sensing images make manual labeling expensive and prone to inconsistent labeling.

[0003] The current mainstream solution fully utilizes the massive amount of unlabeled remote sensing data through a pseudo-label generation mechanism. This type of method usually uses two independent decoders to generate pseudo-labels from the labeled data, and gradually expands the training dataset through iterative optimization. However, existing methods have significant limitations in coastline segmentation scenarios. The fuzzy boundary between water and land in remote sensing images easily leads to the accumulation of pseudo-label noise. Secondly, traditional feature segmentation methods are difficult to effectively capture the multi-scale problem characteristics of the intertidal zone, resulting in insufficient adaptability of the model to the image. The existing technology has the following shortcomings:

[0004] (1) The problem of pseudo-label noise accumulation in existing semi-supervised segmentation techniques

[0005] Traditional semi-supervised methods lack effective noise suppression mechanisms for the fuzzy water-land boundaries in remote sensing images when iteratively generating pseudo-labels. This results in the continuous accumulation of pseudo-label errors during training, significantly reducing the model's segmentation accuracy for complex landforms.

[0006] (2) Insufficient multi-scale feature representation in existing semi-supervised segmentation techniques

[0007] Existing feature alignment methods do not fully consider the multi-scale characteristics of coastlines caused by tidal dynamics, making it difficult to simultaneously capture macroscopic contours and microscopic detail features, resulting in segmentation errors and distortions in low-resolution or occluded images.

[0008] (3) Limited cross-regional generalization capabilities

[0009] The current method does not design an adaptive mechanism for the differences in spectral characteristics of coastlines in different geographical environments, resulting in a significant performance degradation when the model is migrated to a new area due to domain shift problems.

[0010] Therefore, how to build a robust pseudo-labeling mechanism and improve the model's ability to identify complex coastline features has become a key challenge in improving the accuracy of remote sensing coastline segmentation. Summary of the Invention

[0011] In response to the existing technical defects in the background technology, the present invention proposes a method for marine remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling. This method divides image data and constructs text prompts, builds a segmentation model containing modules such as a shared image encoder and a text encoder, trains the model through supervised and unsupervised learning, optimizes the connection weights, and finally achieves accurate segmentation of marine remote sensing coastlines based on the trained model, effectively utilizes unlabeled data, and improves segmentation efficiency and accuracy.

[0012] The technical solution adopted by the present invention is:

[0013] A method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling, the specific steps include:

[0014] Step S1), collect ocean remote sensing coastline image data, randomly divide labeled data and unlabeled data in proportion, construct an ocean remote sensing coastline image dataset, and construct text prompts according to their categories;

[0015] Step S2) constructs a semi-supervised pseudo-labeling approach for ocean remote sensing coastline segmentation. The model structure consists of four parts: a shared image encoder, a text encoder, two independent text-image cross-modal decoders, and an uncertainty-driven pseudo-label calibration module.

[0016] (1) The shared image encoder extracts multi-scale remote sensing image features ;

[0017] (2) The text encoder extracts text features of specific task prompts ;

[0018] (3) The two independent text-image cross-modal decoders are a pseudo-label decoder and a prediction decoder, which generate initial pseudo-labels respectively. and the predicted mask ;

[0019] (4) The uncertainty-driven pseudo-label calibration module is used to calibrate the multi-scale remote sensing image features of the input pseudo-label decoder. Perform perturbation to make it uncertain, perform Monte Carlo sampling, calculate its uncertainty index and mean, use the mean as the pseudo label, and obtain the final pseudo label through uncertainty calibration ;

[0020] Step S3), training the ocean remote sensing coastline segmentation model, the training process includes two stages: supervised learning and unsupervised learning;

[0021] Supervised learning phase: Input the labeled data in the dataset into the ocean remote sensing coastline segmentation model to calculate the final pseudo-label and the supervised loss function value between the true label;

[0022] Unsupervised learning stage: Input the unlabeled data in the dataset into the ocean remote sensing coastline segmentation model to calculate the final pseudo label and the predicted mask The unsupervised loss function value between ;

[0023] The supervised loss function value and the unsupervised loss function value are added together to obtain the total loss function value, and backpropagation is performed. The connection weights are optimized using the selected optimizer and corresponding parameters. After multiple rounds of training, the final ocean remote sensing coastline segmentation model is obtained.

[0024] Step S4) Based on the trained ocean remote sensing coastline segmentation model, the ocean remote sensing coastline image data to be segmented is input, and the coastline segmentation image related thereto is output.

[0025] Preferably, in step S2 (1), the shared image encoder extracts multi-scale remote sensing image features The process is as follows:

[0026] The number of blocks contained in the four layers of the ConvNeXt network is defined as 3, 3, 9, and 3 respectively, with a total of 18 blocks. The labeled data and unlabeled data are input into the ConvNeXt network. In the first block, the labeled data and unlabeled data are processed to obtain image feature data. Then, the image feature data is downsampled by a two-dimensional convolution with a convolution kernel size of 2 and a step size of 2. The downsampled image feature data is used as the input of the next block, and the next block continues to repeat the above data processing operation until the last block is skipped and the multi-scale remote sensing image features are extracted by downsampling. ;

[0027] The specific process of performing data processing operations in Block is as follows:

[0028] Sa1) Extract image features from data through preprocessing , for image features Perform a two-dimensional convolution operation with a convolution kernel size of 7×7 and padding of 3 to obtain the convolution image features ;

[0029] Sa2) Convolutional image features Perform layer normalization operation;

[0030] Sa3) Convolutional image features after layer normalization operation Perform a 2D convolution operation with a convolution kernel size of 1×1 to increase its dimension by 4 times;

[0031] Sa4) Convolution image features after dimensionality increase Perform GELU activation function processing;

[0032] Sa5) Convolution image features after GELU activation function processing Perform a two-dimensional convolution dimensionality reduction operation with a convolution kernel size of 1×1 to reduce its dimension to the original value;

[0033] Sa6) Convolution image features after dimensionality reduction and image features Perform skip connections to obtain image feature data.

[0034] Preferably, in step S2 (2), the text encoder extracts text features of specific task prompts The specific process includes:

[0035] The text prompt is converted into a fixed length by the text encoder The initial text vector is then input into the CXR-BERT model to extract text features. The specific process is as follows:

[0036] Sb1) Use the WordPiece tokenizer to split the text prompt into subword units and add special tags: [CLS] and [SEP] at both ends of the sequence to obtain the token sequence and corresponding token IDs;

[0037] Sb2) Based on the token IDs, the word vector of the corresponding subword is obtained by looking up the table and obtaining the token embedding;

[0038] Sb3) Learn a position vector for each position in the token sequence, so that the model can perceive the order of tokens in the sequence and obtain the position embedding;

[0039] Sb4) Add the obtained Token Embedding and Position Embedding to get the initial text vector ;

[0040] Sb5) Processed by the multi-layer Transformer Encoder in the CXR-BERT model, which includes multi-head self-attention, residual connections, layer normalization, a feed-forward network, a second residual connection and layer normalization;

[0041] Sb6) The output of the last layer Transformer Encoder The text vector at the [CLS] position is regarded as the summary representation of the whole sentence, and the text vectors at other positions are regarded as the context representation of each token sequence, that is, the text features. .

[0042] Further preferably, the Transformer Encoder processes the initial text vector The specific process is as follows:

[0043] Sc1) By initial text vector Perform Linear transformation to divide the subspace and generate Attention heads , the calculation formula is as follows:

[0044]

[0045] in, ,n is the sequence length, is the model dimension, is the number of attention heads, For the A linear transformation, that is, the initial text vector Projected to dimension subspace of ;

[0046] Sc2) Generate queries through Linear transformation ,key ,value The matrix is ​​then calculated through the attention mechanism to obtain the attention weight, focusing on key information. The calculation formula is as follows:

[0047]

[0048] in, is a scaling factor to prevent the dot product from being too large, is the normalization operation, Yes The transpose operation, It is The attention weight of each attention head;

[0049] Sc3) Merge the multi-head self-attention weights of each subspace, that is, splice the attention results of all heads, and use the projection matrix Transform and add , get the total attention weight , the calculation formula is as follows:

[0050]

[0051] in, is the output projection matrix, Randomly set some activation values ​​to zero to prevent overfitting. For splicing operation;

[0052] Sc4) The total attention weight With initial text vector Perform residual connection and layer normalization to obtain normalized text vector , the calculation formula is as follows:

[0053]

[0054] Among them, LayerNorm is layer normalization;

[0055] Sc5) Feedforward neural network processing, that is, Perform two-layer full connection transformation and use Activate and add , and get the feedforward network output , the calculation formula is as follows:

[0056]

[0057] in, , are all fully connected weights, ReLU is a nonlinear activation function, is the bias vector of the first fully connected layer, is the bias vector of the second fully connected layer;

[0058] Sc6) Secondary residual connection and layer normalization, and Residual connection, and then layer normalization to get the final output , the calculation formula is as follows:

[0059]

[0060] Preferably, in step S2 (3), the pseudo-label decoder generates an initial pseudo-label The specific process includes:

[0061] The pseudo-label decoder consists of 4 blocks. The specific process of each block is as follows:

[0062] Sd11) Text features Through a fully connected layer, it is mapped to , whose dimension is aligned with the image feature dimension, and the calculation formula is as follows:

[0063]

[0064] Among them, Conv is a one-dimensional convolution with a convolution kernel size of 1. is the learnable weight matrix, is the text feature after the fully connected layer;

[0065] Sd12) Multi-scale remote sensing image features Input to the self-attention layer, where , , for , the calculation formula is as follows:

[0066]

[0067] in, is layer normalization, is multi-head self-attention, is the image feature after the attention layer;

[0068] Sd13) The image features after the attention layer And the text features after the fully connected layer Input cross-modal attention layer, where Q is , K and V are , the calculation formula is as follows:

[0069]

[0070] in, represents the weight parameter value, is the fused image feature after cross-modal fusion;

[0071] Sd14) will Multi-scale remote sensing image features with the previous input Input the Unetr module for skip connection fusion. The calculation formula is as follows:

[0072]

[0073] in, is the multi-scale remote sensing image feature of the previous input, is the number of blocks, Output features after skip connection fusion;

[0074] Sd15) Output features of the last Block Perform a two-dimensional convolution with a convolution kernel size of 1 to reduce its dimension to 1 and obtain the prediction result , the calculation formula is as follows:

[0075]

[0076] Sd16) through the pseudo-label decoder Generate multiple pseudo labels, calculate uncertainty indicators based on these pseudo labels, and take the average of these pseudo labels as the initial pseudo label , the calculation formula is as follows:

[0077]

[0078] in, represents the number of forward passes, is the total number of pseudo labels, For Start to The summation operation ends, is the time step.

[0079] Preferably, in step S2 (3), the prediction decoder generates a prediction mask The specific process is as follows:

[0080] The prediction decoder also consists of 4 blocks. The specific process of each block is as follows:

[0081] Sd21) Text features Through a fully connected layer, it is mapped to , whose dimension is aligned with the image feature dimension, and the calculation formula is as follows:

[0082]

[0083] Among them, Conv is a one-dimensional convolution with a convolution kernel size of 1. is the learnable weight matrix, is the text feature after the fully connected layer;

[0084] Sd22) Multi-scale remote sensing image features Input to the self-attention layer, where , , for , the calculation formula is as follows:

[0085]

[0086] in, is layer normalization, is multi-head self-attention, is the image feature after the attention layer;

[0087] Sd23) The image features after the attention layer And the text features after the fully connected layer Input cross-modal attention layer, where Q is , K and V are , the calculation formula is as follows:

[0088]

[0089] in, represents the weight parameter value, is the fused image feature after cross-modal fusion;

[0090] Sd24) will Multi-scale remote sensing image features with the previous input Input the Unetr module for skip connection fusion. The calculation formula is as follows:

[0091]

[0092] in, is the multi-scale remote sensing image feature of the previous input, is the number of blocks, Output features after skip connection fusion;

[0093] Sd25) output features of the last Block Perform a two-dimensional convolution with a convolution kernel size of 1 to reduce its dimension to 1 and obtain the prediction result , the calculation formula is as follows:

[0094]

[0095] in, It is a two-dimensional convolution operation with a convolution kernel size of 1;

[0096] Sd26) through the predictive decoder Perform a forward pass and then pass The final value obtained by the function is the prediction mask , the calculation formula is as follows:

[0097]

[0098] in, To convert into probability form, The output of is transformed to make it conform to the probability distribution and output the probability value.

[0099] Preferably, in step S2 (4), the final pseudo label is obtained by the uncertainty calibration module The process is as follows:

[0100] Se1) Multi-scale remote sensing image features for input pseudo-label decoder Perform Dropout random perturbation to introduce uncertainty. The calculation formula is as follows:

[0101]

[0102] Se2) forward pass through the pseudo-label decoder to obtain Different pseudo labels , the calculation formula is as follows:

[0103]

[0104] in, represents the number of forward passes;

[0105] Se3) Different pseudo labels Calculate variance pixel by pixel and mean , the calculation formula is as follows:

[0106] ,

[0107] in, is the variance, which represents the average estimate of pixel values, is the mean, which measures the uncertainty of the pixel;

[0108] Se4) Use variance as uncertainty indicator to calibrate pseudo labels and obtain the final pseudo labels , the calculation formula is as follows:

[0109]

[0110] in, is element-wise multiplication, is the index.

[0111] Preferably, in step S3, the model training process is as follows:

[0112] Using binary cross entropy and The sum of the loss function values ​​is used to calculate the total loss function value of the model. The specific process is as follows:

[0113] Sf1) supervised learning stage, through the final pseudo label Calculate the supervised loss function value with the true label , the calculation formula is as follows:

[0114]

[0115] in, is the true label, is the binary cross entropy, is the Dice loss function value;

[0116] Sf2) unsupervised learning stage, through the final pseudo label With the prediction mask Calculate the unsupervised loss function value , the calculation formula is as follows:

[0117]

[0118] Sf3) Calculate the total loss function value of the model , the calculation formula is as follows:

[0119]

[0120] in, and is the weight parameter value.

[0121] Compared with the existing technology, the present invention proposes a method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling, which has the following beneficial effects:

[0122] (1) The segmentation model is constructed based on a semi-supervised pseudo-labeling approach, using a combined architecture of a shared image encoder, a text encoder, two dual-cross-modal decoders, and an uncertainty-driven pseudo-label calibration module. The deep fusion of multi-scale image features and task-specific text features enables the model to capture the characteristic information of the coastline from different dimensions, effectively improving the perception and segmentation accuracy of coastline details in complex marine remote sensing images. The two decoders generate pseudo-labels and prediction masks respectively, with a clear division of labor, further optimizing the model's processing capabilities for coastline segmentation tasks.

[0123] (2) Accurate pseudo-label calibration to improve segmentation reliability. By perturbing visual features and Monte Carlo sampling, the uncertainty of the data is quantified and utilized, and the mean is used as the pseudo-label to make the generated pseudo-label more reliable. This calibration mechanism effectively reduces the noise of pseudo-labels, avoids the negative impact of incorrect pseudo-labels on model training, enhances the stability and accuracy of model prediction, and ensures the high quality of coastline segmentation results.

[0124] (3) The training strategy that combines supervised learning with unsupervised learning enables the model to learn more universal feature representations with the help of unlabeled data while using labeled data for precise learning. This training method helps the model adapt to marine remote sensing images in different regions and under different imaging conditions, significantly improving the generalization ability of the model, enabling it to perform well in cross-regional coastline segmentation tasks and effectively cope with the complex and changing challenges of the marine environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] Figure 1 A flow chart of the method proposed in the present invention;

[0126] Figure 2 This is the framework diagram of the pseudo-label decoder and prediction decoder proposed in the present invention;

[0127] Figure 3 This is a structural block diagram of the uncertainty-driven pseudo-label calibration module proposed in this invention. DETAILED DESCRIPTION

[0128] The following will be combined with the drawings in the embodiments of this application to further clearly and completely describe the technical solutions in the embodiments of this application. It should be noted that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making any creative work shall fall within the scope of protection of this application.

[0129] In order to make the invention objectives, technical solutions and advantages of this application clearer, the embodiments of this application are further described in detail in conjunction with the drawings in the specification: In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the advantages of the present invention will be further illustrated by comparing the embodiments in conjunction with the drawings and specific implementation methods.

[0130] This paper proposes a method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling. The flowchart of this method is as follows: Figure 1 The steps of this method are described in detail as shown below:

[0131] Step S1) collects ocean remote sensing coastline image data, randomly divides labeled data and unlabeled data into three ratios of 1 / 16: 1 / 8: 1 / 4, constructs an ocean remote sensing coastline image dataset, and constructs text prompts based on their categories. The text prompt template is "This image depicts a {cls} region.", replacing cls with the region type;

[0132] Step S2) constructs a semi-supervised pseudo-labeling approach for ocean remote sensing coastline segmentation. The model structure consists of four parts: a shared image encoder, a text encoder, two independent text-image cross-modal decoders, and an uncertainty-driven pseudo-label calibration module.

[0133] (1) The shared image encoder extracts multi-scale remote sensing image features ;

[0134] Specifically, in step S2 (1), the shared image encoder extracts multi-scale remote sensing image features The specific process is as follows:

[0135] The number of blocks contained in the four layers of the ConvNeXt network is defined as 3, 3, 9, and 3 respectively, with a total of 18 blocks. The labeled data and unlabeled data are input into the ConvNeXt network. In the first block, the labeled data and unlabeled data are processed to obtain image feature data. Then, the image feature data is downsampled by a two-dimensional convolution with a convolution kernel size of 2 and a step size of 2. The downsampled image feature data is used as the input of the next block, and the next block continues to repeat the above data processing operation until the last block is skipped and the multi-scale remote sensing image features are extracted by downsampling. ;

[0136] The specific process of performing data processing operations in Block is as follows:

[0137] Sa1) Extract image features from data through preprocessing , for image features Perform a two-dimensional convolution operation with a convolution kernel size of 7×7 and padding of 3 to obtain image features The local feature information in the image is extracted through the convolution operation. The padding of 3 is to keep the size of the feature map unchanged during the convolution process so that the edge information of the image can also be effectively processed.

[0138] Sa2) Convolutional image features Perform layer normalization operations; layer normalization operations help speed up model convergence and alleviate gradient vanishing or gradient exploding problems;

[0139] Sa3) Convolutional image features after layer normalization operation Perform a 2D convolution operation with a convolution kernel size of 1×1 to increase its dimension by 4 times. 1×1 convolution can adjust the number of channels of the feature map without changing the spatial size (width and height) of the image. Increasing the number of channels by 4 times can increase the expressive power of features.

[0140] Sa4) Convolution image features after dimensionality increase Perform GELU activation function processing; activation function is used to introduce nonlinear factors into the neural network;

[0141] Sa5) Convolution image features after GELU activation function processing Perform a 2D convolution with a kernel size of 1×1 to reduce the dimension to its original value. After the previous dimensionality increase operation expands the number of channels and enriches the information, the dimensionality reduction operation restores the number of channels to the original level, removes any redundant information, and further integrates the features.

[0142] Sa6) Convolution image features after dimensionality reduction With the original image features Perform skip connections to obtain image feature data.

[0143] (2) The text encoder extracts text features of specific task prompts ;

[0144] Specifically, in step S2 (2), the text encoder extracts text features of specific task prompts The specific process includes:

[0145] The text prompt is converted into a fixed length by the text encoder The initial text vector is then input into the CXR-BERT model to extract text features. The specific process is as follows:

[0146] Sb1) Use the WordPiece tokenizer to split the text prompt into subword units and add special tags at both ends of the sequence: [CLS] (classification tag) and [SEP] (separation tag), respectively, to obtain a token sequence and corresponding token IDs (subword numbers). The WordPiece tokenizer effectively handles unregistered words (such as spelling errors or new vocabulary), improving the accuracy of the text. [CLS] is used to aggregate the semantics of the entire sentence (such as classification tasks), and [SEP] is used to distinguish sentence pairs (if any).

[0147] Sb2) Based on the Token IDs, the table is looked up to obtain the word vector of the corresponding subword and obtain the TokenEmbedding;

[0148] Sb3) Learn a position vector for each position in the token sequence, so that the model can perceive the order of tokens in the sequence and obtain the position embedding;

[0149] Sb4) Add the obtained Token Embedding and Position Embedding to get the initial text vector ;

[0150] Sb5) is processed by a multi-layer Transformer Encoder, which includes multi-head self-attention (to capture inter-word dependencies), residual connections (to prevent gradient vanishing), layer normalization (to stabilize training), a feedforward network (non-linear transformation), a second residual connection and layer normalization;

[0151] Sb6) The output of the last layer Transformer Encoder The vector at the [CLS] position is regarded as the summary representation of the whole sentence (or sentence pair), and the vectors at other positions can be used as the context representation of each token, that is, the text feature .

[0152] More specifically, the specific process of the Transformer Encoder processing text vectors is as follows:

[0153] Sc1) By initial text vector Perform Linear transformation to divide the subspace and generate Attention heads , which is used to capture different feature representations and enhance the model's ability to capture text information. The calculation formula is as follows:

[0154]

[0155] in, ,n is the sequence length, is the model dimension, h is the number of attention heads, For the A linear transformation, that is, the initial text vector Projected to dimension subspace of ;

[0156] Sc2) Generate queries through Linear transformation ,key ,value The matrix is ​​then calculated through the attention mechanism to obtain the attention weight, focusing on key information. The calculation formula is as follows:

[0157]

[0158] in, is a scaling factor to prevent the dot product from being too large, is the normalization operation, Yes The transpose operation, It is The attention weight of each attention head;

[0159] Sc3) Merge the multi-head self-attention weights of each subspace, that is, splice the attention results of all heads, and use the projection matrix Transform and add , get the total attention weight , integrate multi-head information, restore model dimension, enhance generalization ability, the calculation formula is as follows:

[0160]

[0161] in, is the output projection matrix, Randomly set some activation values ​​to zero to prevent overfitting. For splicing operation;

[0162] Sc4) Attention weight Perform residual connection and layer normalization with the text vector T to obtain the normalized text vector , residual connection alleviates gradient disappearance, layer normalization improves training stability, and the calculation formula is as follows:

[0163]

[0164] Among them, LayerNorm is layer normalization;

[0165] Sc5) Feedforward neural network processing, that is, Perform two-layer full connection transformation and use Activate and add , and get the feedforward network output , the calculation formula is as follows:

[0166]

[0167] in, , are all fully connected weights, ReLU is a nonlinear activation function, is the bias vector of the first fully connected layer, is the bias vector of the second fully connected layer;

[0168] Sc6) Secondary residual connection and layer normalization, and Residual connection, then layer normalization, to get the final output , the calculation formula is as follows:

[0169]

[0170] (3) The two independent text-image cross-modal decoders are a pseudo-label decoder and a prediction decoder, which generate initial pseudo-labels respectively. and the predicted mask ;

[0171] Specifically, in step S2 (3), if Figure 2 As shown, the pseudo-label decoder generates the initial pseudo-label The specific process includes:

[0172] The pseudo-label decoder consists of 4 blocks. The specific process of each block is as follows:

[0173] Sd11) Text features Through a fully connected layer, it is mapped to , whose dimension is aligned with the image feature dimension, and the calculation formula is as follows:

[0174]

[0175] Among them, Conv is a one-dimensional convolution with a convolution kernel size of 1. is the learnable weight matrix, is the text feature after the fully connected layer;

[0176] Sd12) Multi-scale remote sensing image features Input to the self-attention layer, where for , the internal dependencies of image features are captured through the self-attention mechanism to enhance the expressiveness of image features. The calculation formula is as follows:

[0177]

[0178] in, is layer normalization, is multi-head self-attention, is the image feature after the attention layer;

[0179] Sd13) The image features after the attention layer And the text features after the fully connected layer Input cross-modal attention layer, where Q is , K and V are , realizing cross-modal interaction between image features and text features, integrating image features with text information, and further enriching feature expression. The calculation formula is as follows:

[0180]

[0181] in, represents the weight parameter value, is the fused image feature after cross-modal fusion;

[0182] Sd14) will Multi-scale remote sensing image features with the previous input Input the Unetr module for skip connection fusion. The calculation formula is as follows:

[0183]

[0184] in, is the multi-scale remote sensing image feature of the previous input, is the number of blocks, Output features after skip connection fusion;

[0185] Sd15) output features of the last layer of Block Perform a two-dimensional convolution with a convolution kernel size of 1 to reduce its dimension to 1 and obtain the prediction result , the calculation formula is as follows:

[0186]

[0187] Sd16) through the pseudo-label decoder Generate multiple pseudo labels, calculate uncertainty indicators based on these pseudo labels, and take the average of these pseudo labels as the initial pseudo label , the calculation formula is as follows:

[0188]

[0189] in, represents the number of forward passes, is the total number of pseudo labels, For Start to The summation operation ends, is the time step.

[0190] Specifically, in step S2 (3), the prediction decoder generates a prediction mask The specific process is as follows:

[0191] The prediction decoder also consists of 4 blocks. The specific process of each block is as follows:

[0192] Sd21) Text features Through a fully connected layer, it is mapped to , whose dimension is aligned with the image feature dimension, and the calculation formula is as follows:

[0193]

[0194] Among them, Conv is a one-dimensional convolution with a convolution kernel size of 1. is the learnable weight matrix, is the text feature after the fully connected layer;

[0195] Sd22) Multi-scale remote sensing image features Input to the self-attention layer, where for , the internal dependencies of image features are captured through the self-attention mechanism to enhance the expressiveness of image features. The calculation formula is as follows:

[0196]

[0197] in, is layer normalization, is multi-head self-attention, is the image feature after the attention layer;

[0198] Sd23) The image features after the attention layer And the text features after the fully connected layer Input cross-modal attention layer, where Q is , K and V are , realizing cross-modal interaction between image features and text features, integrating image features with text information, and further enriching feature expression. The calculation formula is as follows:

[0199]

[0200] in, represents the weight parameter value, is the fused image feature after cross-modal fusion;

[0201] Sd24) will Multi-scale remote sensing image features with the previous input Input the Unetr module for skip connection fusion. The calculation formula is as follows:

[0202]

[0203] in, is the multi-scale remote sensing image feature of the previous input, is the number of blocks, Output features after skip connection fusion;

[0204] Sd25) output features of the last layer of Block Perform a two-dimensional convolution with a convolution kernel size of 1 to reduce its dimension to 1 and obtain the prediction result , the calculation formula is as follows:

[0205]

[0206] Sd26) through the predictive decoder Perform a forward pass and then pass The final value obtained by the function is the prediction mask , the calculation formula is as follows:

[0207]

[0208] in, To convert into probability form, The output of is transformed to make it conform to the probability distribution and output the probability value.

[0209] (4) The uncertainty-driven pseudo-label calibration module is used to calibrate the multi-scale remote sensing image features of the input pseudo-label decoder. Perform perturbation to make it uncertain, perform Monte Carlo sampling, calculate its uncertainty index and mean, use the mean as the pseudo label, and obtain the final pseudo label through uncertainty calibration ;

[0210] Specifically, in step S2 (4), if Figure 3 As shown, the final pseudo label is obtained by uncertainty calibration The process is as follows:

[0211] Se1) Multi-scale remote sensing image features for input pseudo-label decoder Perform Dropout random perturbation to introduce uncertainty. The calculation formula is as follows:

[0212]

[0213] Se2) forward pass through the pseudo-label decoder to obtain Different pseudo labels , the calculation formula is as follows:

[0214]

[0215] in, represents the number of forward passes;

[0216] Se3) Different pseudo labels Calculate variance pixel by pixel and mean , the calculation formula is as follows:

[0217] ,

[0218] in, is the variance, which represents the average estimate of pixel values, is the mean, which measures the uncertainty of the pixel (the larger the variance, the higher the uncertainty);

[0219] Se4) Use variance as uncertainty indicator to calibrate pseudo labels and obtain the final pseudo labels , the calculation formula is as follows:

[0220]

[0221] in, is element-wise multiplication, is the index.

[0222] Step S3), training the ocean remote sensing coastline segmentation model, the training process includes two stages: supervised learning and unsupervised learning;

[0223] Supervised learning phase: Input the labeled data in the dataset into the ocean remote sensing coastline segmentation model to calculate the final pseudo label and the supervised loss function value between the true label;

[0224] Unsupervised learning stage: Input the unlabeled data in the dataset into the ocean remote sensing coastline segmentation model to calculate the final pseudo label and the predicted mask The unsupervised loss function value between ;

[0225] The supervised loss function value and the unsupervised loss function value are added together to obtain the total loss function value, and backpropagation is performed. The connection weights are optimized using the selected optimizer and corresponding parameters. After multiple rounds of training, the final ocean remote sensing coastline segmentation model is obtained.

[0226] Specifically, in step S3, the process of training the model is as follows:

[0227] The model training process is as follows:

[0228] Using binary cross entropy and The sum of the loss function values ​​is used to calculate the total loss function value of the model. The specific process is as follows:

[0229] Sf1) supervised learning stage, through the final pseudo label Calculate the supervised loss function value with the true label , the calculation formula is as follows:

[0230]

[0231] in, is the true label, is the binary cross entropy, which measures the difference in probability distribution between pseudo labels and true labels, focusing on the correctness of classification. The Dice loss function value measures the overlap between pseudo labels and true labels. It is more robust to class imbalance and focuses on segmentation accuracy.

[0232] Sf2) unsupervised learning stage, through the final pseudo label With the prediction mask Calculate the unsupervised loss function value , the calculation formula is as follows:

[0233]

[0234] Sf3) Calculate the total loss function value of the model , the calculation formula is as follows:

[0235]

[0236] in, and is the weight parameter value, which is used to adjust the value of the supervision loss function And the unsupervised loss function value Contribution to total losses.

[0237] Step S4) Based on the trained ocean remote sensing coastline segmentation model, the ocean remote sensing coastline image data to be segmented is input, and the coastline segmentation image related thereto is output.

[0238] This paper proposes a core innovative mechanism, a text-guided decoding architecture, in the field of cross-domain coastline semantic segmentation under a semi-supervised pseudo-labeling approach. Compared with the limitations of traditional semi-supervised coastline segmentation methods that rely on a single visual representation, this architecture innovatively constructs a regionally adaptive text-visual collaborative learning paradigm: at the feature reconstruction stage of each decoder level, semantic cue vectors strongly correlated with geographical partitions are injected, and the decoupling and enhancement of fine-grained regional features are achieved through the cross-modal fusion of hierarchical text semantic embedding and multi-scale visual features. This design enables the model to dynamically establish semantic alignment between the text description space and the pixel-level visual feature space during cross-regional parallel training, effectively improving the joint representation learning ability of heterogeneous coastline elements in multiple regions, thereby breaking through the regional feature confusion bottleneck of traditional methods in cross-domain generalization, and significantly enhancing the model's adaptability to the topological structure of unlabeled areas and the edge segmentation accuracy.

[0239] At the same time, in order to improve the quality of pseudo-labels generated by the pseudo-label decoder, an uncertainty-driven pseudo-label calibration module is further introduced to optimize the output reliability of the pseudo-label decoder in the semi-supervised framework; by performing Monte Carlo sampling on the decoder, a set of pseudo-labels with differentiated probability distributions is generated, and an uncertainty heat map is constructed based on the pixel-level prediction variance. By minimizing the inter-class information entropy in high-uncertainty areas, the confidence threshold of the pseudo-label is dynamically adjusted to suppress noise propagation; this joint optimization strategy not only realizes the probabilistic evaluation of the pseudo-label quality, but also alleviates the random deviation of single inference through the integrated learning paradigm, significantly improving the topological consistency and boundary robustness of pseudo-labels in cross-domain scenarios.

[0240] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0241] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling, characterized by: include: Step S1), collect ocean remote sensing coastline image data, randomly divide labeled data and unlabeled data in proportion, construct an ocean remote sensing coastline image dataset, and construct text prompts according to their categories; Step S2) constructs a semi-supervised pseudo-labeling approach for ocean remote sensing coastline segmentation. The model structure consists of four parts: a shared image encoder, a text encoder, two independent text-image cross-modal decoders, and an uncertainty-driven pseudo-label calibration module. (1) The shared image encoder extracts multi-scale remote sensing image features ; Shared image encoder for extracting multi-scale remote sensing image features The process is as follows: The number of blocks contained in the four layers of the ConvNeXt network is defined as 3, 3, 9, and 3 respectively, with a total of 18 blocks. The labeled data and unlabeled data are input into the ConvNeXt network. In the first block, the labeled data and unlabeled data are processed to obtain image feature data. Then, the image feature data is downsampled by a two-dimensional convolution with a convolution kernel size of 2 and a step size of 2. The downsampled image feature data is used as the input of the next block, and the next block continues to repeat the above data processing operation until the last block is skipped and the multi-scale remote sensing image features are extracted by downsampling. ; The specific process of performing data processing operations in the Block is as follows: Sa1) Extract image features from data through preprocessing , for image features Perform a two-dimensional convolution operation with a convolution kernel size of 7×7 and padding of 3 to obtain the convolution image features ; Sa2) Convolutional image features Perform layer normalization operation; Sa3) Convolutional image features after layer normalization operation Perform a 2D convolution operation with a convolution kernel size of 1×1 to increase its dimension by 4 times; Sa4) Convolution image features after dimensionality increase Perform GELU activation function processing; Sa5) Convolution image features after GELU activation function processing Perform a two-dimensional convolution dimensionality reduction operation with a convolution kernel size of 1×1 to reduce its dimension to the original value; Sa6) Convolution image features after dimensionality reduction and image features Perform skip connection to obtain image feature data; (2) The text encoder extracts text features of specific task prompts ; (3) The two independent text-image cross-modal decoders are a pseudo-label decoder and a prediction decoder, which generate initial pseudo-labels respectively. and the predicted mask ; (4) The uncertainty-driven pseudo-label calibration module is used to calibrate the multi-scale remote sensing image features input to the pseudo-label decoder. Perform perturbation to make it uncertain, perform Monte Carlo sampling, calculate its uncertainty index and mean, use the mean as the pseudo label, and obtain the final pseudo label through uncertainty calibration ; Step S3), training the ocean remote sensing coastline segmentation model; Step S4) Based on the trained ocean remote sensing coastline segmentation model, the ocean remote sensing coastline image data to be segmented is input, and the coastline segmentation image related thereto is output.

2. The method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling according to claim 1 is characterized in that: In step S2 (2), the text encoder extracts text features of specific task prompts The specific process includes: The text prompt is converted into a fixed length by the text encoder The initial text vector is then input into the CXR-BERT model to extract text features. The specific process is as follows: Sb1) Use the WordPiece tokenizer to split the text prompt into subword units and add special tags: [CLS] and [SEP] at both ends of the sequence to obtain the token sequence and corresponding token IDs; Sb2) Based on the token IDs, the word vector of the corresponding subword is obtained by looking up the table and obtaining the token embedding; Sb3) Learn a position vector for each position in the token sequence, so that the model can perceive the order of tokens in the sequence and obtain the position embedding; Sb4) Add the obtained Token Embedding and Position Embedding to get the initial text vector ; Sb5) Processed by the multi-layer Transformer Encoder in the CXR-BERT model, which includes multi-head self-attention, residual connections, layer normalization, a feed-forward network, a second residual connection and layer normalization; Sb6) The output of the last layer Transformer Encoder The text vector at the [CLS] position is regarded as the summary representation of the whole sentence, and the text vectors at other positions are regarded as the context representation of each token sequence, that is, the text features. .

3. The method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling according to claim 2 is characterized in that: The Transformer Encoder processes the initial text vector The specific process is as follows: Sc1) By initial text vector Perform Linear transformation to divide the subspace and generate Attention heads , the calculation formula is as follows: ； in, ,n is the sequence length, is the model dimension, is the number of attention heads, For the A linear transformation, that is, the initial text vector Projected to dimension subspace of ; Sc2) Generate queries through Linear transformation ,key ,value The matrix is ​​then calculated through the attention mechanism to obtain the attention weight, focusing on key information. The calculation formula is as follows: ; in, is a scaling factor to prevent the dot product from being too large, is the normalization operation, Yes The transpose operation, It is The attention weight of each attention head; Sc3) Merge the multi-head self-attention weights of each subspace, that is, splice the attention results of all heads, and use the projection matrix Transform and add , get the total attention weight , the calculation formula is as follows: ; in, is the output projection matrix, Randomly set some activation values ​​to zero to prevent overfitting. For splicing operation; Sc4) The total attention weight With initial text vector Perform residual connection and layer normalization to obtain normalized text vector , the calculation formula is as follows: ; Among them, LayerNorm is layer normalization; Sc5) Feedforward neural network processing, that is, Perform two-layer full connection transformation and use Activate and add , and get the feedforward network output , the calculation formula is as follows: ; in, , are all fully connected weights, ReLU is a nonlinear activation function, is the bias vector of the first fully connected layer, is the bias vector of the second fully connected layer; Sc6) Secondary residual connection and layer normalization, and Residual connection, and then layer normalization to get the final output , the calculation formula is as follows: 。 4. The method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling according to claim 1, characterized in that: In step S2 (3), the pseudo-label decoder generates an initial pseudo-label The specific process includes: The pseudo-label decoder consists of 4 blocks. The specific process of each block is as follows: Sd11) Text features Through a fully connected layer, it is mapped to , whose dimension is aligned with the image feature dimension, and the calculation formula is as follows: ； Among them, Conv is a one-dimensional convolution with a convolution kernel size of 1. is the learnable weight matrix, is the text feature after the fully connected layer; Sd12) Multi-scale remote sensing image features Input to the self-attention layer, where , , for , the calculation formula is as follows: ； in, is layer normalization, is multi-head self-attention, is the image feature after the attention layer; Sd13) The image features after the attention layer And the text features after the fully connected layer Input cross-modal attention layer, where Q is , K and V are , the calculation formula is as follows: ; in, represents the weight parameter value, is the fused image feature after cross-modal fusion; Sd14) will Multi-scale remote sensing image features with the previous input Input the Unetr module for skip connection fusion. The calculation formula is as follows: ； in, is the multi-scale remote sensing image feature of the previous input, is the number of blocks, Output features after skip connection fusion; Sd15) Output features of the last Block Perform a two-dimensional convolution with a convolution kernel size of 1 to reduce its dimension to 1 and obtain the prediction result , the calculation formula is as follows: ； Sd16) through the pseudo-label decoder Generate multiple pseudo labels, calculate uncertainty indicators based on these pseudo labels, and take the average of these pseudo labels as the initial pseudo label , the calculation formula is as follows: ; in, represents the number of forward passes, is the total number of pseudo labels, For Start to The summation operation ends, is the time step.

5. The method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling according to claim 1, characterized in that: In step S2 (3), the prediction decoder generates a prediction mask The specific process is as follows: The prediction decoder also consists of 4 blocks. The specific process of each block is as follows: Sd21) Text features Through a fully connected layer, it is mapped to , whose dimension is aligned with the image feature dimension, and the calculation formula is as follows: ； Among them, Conv is a one-dimensional convolution with a convolution kernel size of 1. is the learnable weight matrix, is the text feature after the fully connected layer; Sd22) Multi-scale remote sensing image features Input to the self-attention layer, where , , for , the calculation formula is as follows: ； in, is layer normalization, is multi-head self-attention, is the image feature after the attention layer; Sd23) The image features after the attention layer And the text features after the fully connected layer Input cross-modal attention layer, where Q is , K and V are , the calculation formula is as follows: ； in, represents the weight parameter value, is the fused image feature after cross-modal fusion; Sd24) will Multi-scale remote sensing image features with the previous input Input the Unetr module for skip connection fusion. The calculation formula is as follows: ； in, is the multi-scale remote sensing image feature of the previous input, is the number of blocks, Output features after skip connection fusion; Sd25) output features of the last Block Perform a two-dimensional convolution with a convolution kernel size of 1 to reduce its dimension to 1 and obtain the prediction result , the calculation formula is as follows: ; in, It is a two-dimensional convolution operation with a convolution kernel size of 1; Sd26) through the predictive decoder Perform a forward pass and then pass The final value obtained by the function is the prediction mask , the calculation formula is as follows: ; in, To convert into probability form, The output of is transformed to make it conform to the probability distribution and output the probability value.

6. The method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling according to claim 1, characterized in that: In step S2 (4), the final pseudo label is obtained through the uncertainty calibration module The process is as follows: Se1) Multi-scale remote sensing image features for input pseudo-label decoder Perform Dropout random perturbation to introduce uncertainty. The calculation formula is as follows: ； Se2) forward pass through the pseudo-label decoder to obtain Different pseudo labels , the calculation formula is as follows: ; in, represents the number of forward passes; Se3) Different pseudo labels Calculate variance pixel by pixel and mean , the calculation formula is as follows: ； in, is the variance, which represents the average estimate of pixel values, is the mean, which measures the uncertainty of the pixel; Se4) Use variance as uncertainty indicator to calibrate pseudo labels and obtain the final pseudo labels , the calculation formula is as follows: ; in, is element-wise multiplication, is the index.

7. The method for ocean remote sensing coastline segmentation based on text-guided semi-supervised pseudo-labeling according to claim 1, characterized in that: In step S3, the model training process is as follows: The training process includes two stages: supervised learning and unsupervised learning; Supervised learning phase: Input the labeled data in the dataset into the ocean remote sensing coastline segmentation model to calculate the final pseudo label and the supervised loss function value between the true label; Unsupervised learning stage: Input the unlabeled data in the dataset into the ocean remote sensing coastline segmentation model to calculate the final pseudo label and the predicted mask The unsupervised loss function value between ; The supervised loss function value and the unsupervised loss function value are added together to obtain the total loss function value. Back propagation is performed and the connection weights are optimized using the selected optimizer and corresponding parameters. After multiple rounds of training, the final ocean remote sensing coastline segmentation model is obtained. Using binary cross entropy and The sum of the loss function values ​​is used to calculate the total loss function value of the model. The specific process is as follows: Sf1) supervised learning stage, through the final pseudo label Calculate the supervised loss function value with the true label , the calculation formula is as follows: ； in, is the true label, is the binary cross entropy, is the Dice loss function value; Sf2) unsupervised learning stage, through the final pseudo label With the prediction mask Calculate the unsupervised loss function value , the calculation formula is as follows: ; Sf3) Calculate the total loss function value of the model , the calculation formula is as follows: ; in, and is the weight parameter value.

Images