A class prototype-based remote sensing image semantic segmentation method

Abstract

The application relates to a remote sensing image semantic segmentation method based on a category prototype and belongs to the technical field of remote sensing image processing. The method comprises the following steps: pre-processing an original remote sensing image to obtain a labeled data set and an unlabeled data set; performing weak enhancement and strong enhancement based on the unlabeled data set, generating double masks, thereby generating a multi-domain hybrid strong enhancement sample set and a weak enhancement sample, and obtaining a prototype consistency loss according to a category prototype; generating a reliable region mask based on the weak enhancement sample, filtering out pseudo-label noise, and obtaining an unlabeled data noise filtering loss; constructing a total loss and iteratively training a network, updating model parameters and the category prototype, and outputting a semantic segmentation model; and inputting a remote sensing image to be semantically segmented into the semantic segmentation model to obtain a remote sensing image semantic segmentation result. The application aims to solve the technical problems that the prior art has a limited perturbation space, single consistency constraint and difficulty in filtering out pseudo-label noise.

Description

Technical Field

[0001] This invention relates to a semantic segmentation method for remote sensing images based on category prototypes, belonging to the field of remote sensing image processing technology. Background Technology

[0002] With the development of remote sensing observation technology, the ability to acquire high-resolution remote sensing images has been significantly improved. Remote sensing image semantic segmentation, as a core technology for ground feature identification and accurate segmentation, plays a crucial role in multiple fields. In recent years, remote sensing image semantic segmentation combined with deep learning has achieved remarkable results in disaster assessment, urban planning, and other fields. Although deep learning-driven semantic segmentation methods can extract high-level features and possess strong robustness, they rely on a large amount of high-quality pixel-level labeled data—the labeling process is not only time-consuming and labor-intensive but also requires expertise in the remote sensing field, resulting in extremely high labeling costs and severely limiting its practical application.

[0003] To address the problem of scarce labeled data, semi-supervised learning (SSL) has become a mainstream approach. Its core principle is to improve model performance by utilizing a small amount of labeled data alongside a large amount of unlabeled data. In remote sensing semantic segmentation, existing SSL methods are mainly divided into two categories: Consistency Regularization (CR): By applying input-level / feature-level perturbations to the same image, the model output is forced to remain consistent (such as the dual-stream perturbation of UniMatch and the feature perturbation of CCT). However, the single strong or weak consistency constraint limits the exploration of the perturbation space and makes it difficult to make full use of the rich ground cover information in remote sensing images. Self-training (ST): High-quality pseudo-labels are selected to expand the training set through confidence thresholds (such as linear sampling of LSST and pseudo-label optimization of ST++). However, fixed thresholds cannot adapt to model iteration changes, the pseudo-label noise filtering effect is poor, and it is easy to cause model optimization deviation.

[0004] Furthermore, remote sensing images exhibit complex distributions and significant class differences in ground features, and existing methods lack refined optimization tailored to class characteristics, further limiting segmentation accuracy. Therefore, designing a semi-supervised segmentation method that can expand the perturbation space, supplement consistency constraints, and finely filter out noise has become crucial to overcoming existing bottlenecks. Summary of the Invention

[0005] The purpose of this invention is to provide a remote sensing image semantic segmentation method based on category prototypes, which aims to solve the technical problems of limited perturbation space, single consistency constraint, and difficulty in filtering out pseudo-label noise in existing technologies.

[0006] To achieve the above objectives, the technical solution of this invention is: a remote sensing image semantic segmentation method based on category prototypes. This method achieves efficient utilization of labeled and unlabeled data through the synergy of three modules: a multi-domain hybrid CutMix strong enhancement strategy, a prototype-based prediction result consistency method, and a category adaptive entropy mapping method. The specific steps are as follows: Step 1: Preprocess the original remote sensing images to obtain a preprocessed dataset; wherein, the dataset includes a labeled dataset and an unlabeled dataset; Step 2: Perform weak and strong augmentation on the unlabeled dataset and generate a double mask to generate a multi-domain mixed strong augmentation sample set and weak augmentation samples; wherein, the double mask is a variable rectangular mask and a confidence mask; Step 3: Based on the multi-domain hybrid strongly enhanced sample set and weakly enhanced samples, obtain the prototype consistency loss according to the category prototype; Step 4: Based on the weakly enhanced samples, generate a reliable region mask, filter out false label noise, and obtain the loss after filtering out noise in the unlabeled data; Step 5: Construct the total loss, iteratively train the network based on the total loss, update the model parameters and category prototypes, and output the semantic segmentation model; wherein, the total loss includes labeled data cross-entropy loss, unlabeled data noise filtering loss, prototype consistency loss, feature-prototype comparison loss, and similar prototype compactness loss; Step 6: Input the remote sensing image to be semantically segmented into the semantic segmentation model to obtain the semantic segmentation result of the remote sensing image.

[0007] Optionally, Step 2 specifically includes: Step 2.1: For each unlabeled remote sensing image Perform weak and strong enhancements, generating several strongly enhanced samples, forming a batch with a size of [size missing]. sample set ,in, This represents the b-th strongly augmented sample within the batch; Step 2.2: Follow the marked scale Generate a variable rectangular mask with training round t Initial quantity Number of rounds in the current round ,in, Total number of rounds; Each rectangle is assigned a random area ratio Aspect Ratio Calculate width ,high ,in, Indicates uniform distribution. It is the minimum allowed side length. This indicates the number of pixels in the vertical direction of the image. This represents the number of pixels in the horizontal direction of the image, with the top-left corner coordinates randomly selected. ; A confidence mask is generated by predicting the confidence matrix C from a weakly enhanced unlabeled remote sensing image. Iterate through each pixel in C, and set the mask value to 1 if the confidence level is less than the preset threshold, otherwise set it to 0. Step 2.3: Merge the two masks to obtain ,in, For the final mask region, the operator This indicates that two matrices of the same shape are operated on; based on the final mask region, samples are mixed across batches, and if the b-th sample in the batch is even, it is processed according to... Mixed, in which, This indicates a circular index within the batch. If the b-th sample in the batch is odd, then it is indexed by... This process involves mixing to generate a multi-domain hybrid strongly enhanced sample set. .

[0008] Optionally, Step 3 specifically includes: Step 3.1: Create K prototypes for each category c. ,in, The feature dimension is represented by an initial prototype randomly selected from the output features of the labeled data encoder; after each training round, the strongly enhanced feature z of the unlabeled samples is executed. Normalize, calculate its cosine similarity with all prototypes and assign it to the prototype category with the highest similarity, according to... Update, in which, For the updated prototype, The momentum coefficient, It is the original feature vector go through The normalized mean vector is then used to assign the vector to the prototypes using online clustering, wherein the online clustering is performed by calculating... The similarity to each prototype will Assign the model to the prototype with the highest similarity and update the model using an exponential moving average; Step 3.2: Enhance the features of strongly enhanced samples Calculate the cosine similarity between it and K prototypes of each category, take the maximum value for each category, and generate the prototype prediction probability, expressed as: in, For a certain pixel to belong to the first The probability of a class It is a function that calculates the cosine similarity between two vectors. An index of the most similar prototype in each category. For temperature parameters; Step 3.3: Label the dataset by Calculate the loss, where, Indicates the number of labeled data. For CE loss, For accurate labeling, This represents the non-parametric semantic segmentation result of the labeled data stream; For unlabeled datasets, first... Generate pseudo-tags, where, This indicates weak enhancement of execution. It is the first An unlabeled input image, This represents a neural network model used for semantic segmentation. These are the trainable parameters of the model. This indicates taking the category corresponding to the highest probability, and then pressing... Calculate the loss, where, Indicates the number of unlabeled data. It is a general representation of unlabeled data, referring to any single unlabeled data point. This represents the non-parametric semantic segmentation result of the unlabeled data stream, thus yielding the prototype consistency loss as... .

[0009] Optionally, Step 4 specifically includes: Step 4.1: Calculate the weak enhancement entropy graph ,in, express The Middle Predicted probability of class This represents the segmentation result predicted by the weak enhancement model; Step 4.2: Generate a category mask pixel by pixel, the expression is: in, This indicates that the confidence matrix predicted by the model is used to generate the category. The mask, extract Composed of pixel entropy values ​​of 1 , Indicate category The set of entropy values; Step 4.3: If ,Pick The 40th percentile value , Represents the category entropy tensor, otherwise ;according to Generate an entropy mapping graph, where, Represents an entropy mapping diagram. The confidence mask is then used to generate the reliable region mask. The expression is: Finally, the loss after filtering out noise from the unlabeled data was calculated. ,in, For the first Unlabeled samples Pseudo-labels generated by model prediction.

[0010] Optionally, the expression for the total loss is: in, For the total loss, For the cross-entropy loss of labeled data, As weight, This refers to the loss after filtering out noise from unlabeled data. For prototype consistency loss, For feature-prototype contrastive loss, It is a compact loss of the same prototype.

[0011] Compared with existing semi-supervised remote sensing semantic segmentation methods, the advantages of this invention are: First, by using dynamic double masking and cross-batch mixing, the diversity of perturbed samples is significantly improved, making the model more robust in complex terrain scenes; second, by supplementing non-parametric prototype constraints, the class discriminativeness can still be enhanced under low labeling rates, resulting in better model generalization ability; third, by dynamically generating entropy thresholds for each class, the accuracy of pseudo-labels is improved, avoiding the limitations of global thresholds. Furthermore, the method has demonstrated generalization ability on multiple remote sensing datasets, providing a feasible solution for low-cost applications of remote sensing image semantic segmentation. Attached Figure Description

[0012] Figure 1 This is a flowchart of the present invention; Figure 2 This is a framework description diagram of the present invention; Figure 3 This invention is in different and Comparison of remote sensing image segmentation performance under different parameter combinations; Figure 4 This invention is applied to the Vaihingen dataset with different parameters. Performance chart for the combined configuration. Detailed Implementation

[0013] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. These embodiments are merely illustrative of the technical solutions of the present invention and do not constitute any limitation on their scope.

[0014] Example 1: As Figure 1 and Figure 2 As shown, this embodiment proposes a remote sensing image semantic segmentation method based on category prototypes, including the following steps: Step 1: Preprocess the original remote sensing images to obtain a preprocessed dataset; wherein, the dataset includes a labeled dataset and an unlabeled dataset; Optionally, the preprocessing involves uniformly cropping the original remote sensing images to 320×320 pixels to match the model input size, aligning the number of spectral bands (near-infrared, red, and green bands) of all images, and removing images with invalid pixels at the edges; then, the labeled dataset is divided into two parts according to the labeling rate of 1 / 8 or 1 / 4. Compared with unlabeled datasets ,in, For the first A labeled input image, It is the first An unlabeled input image, To correspond to its pixel-level true annotation, This indicates the number of pixels in the vertical direction of the image. This indicates the number of pixels in the horizontal direction of the image. Indicates the number of categories. Determined by the annotation rate, This represents the amount of unlabeled data.

[0015] Optionally, after the preprocessing operation, the segmentation network is initialized. In this embodiment, the DeepLabV3+ segmentation network with ResNet-101 as the backbone is initialized and ImageNet pre-trained weights are loaded. Step 2: Perform weak and strong augmentation on the unlabeled dataset and generate a double mask to generate a multi-domain mixed strong augmentation sample set and weak augmentation samples; wherein, the double mask is a variable rectangular mask and a confidence mask; Step 2.1: For each unlabeled remote sensing image Perform weak and strong enhancements, generating several strongly enhanced samples, forming a batch with a size of [size missing]. sample set ,in, This represents the b-th strongly augmented sample within the batch; Optionally, the weak enhancement includes random cropping to 320×320, random horizontal / vertical flipping probability of 0.5, and random factor of 0.8-1.2; the strong enhancement includes color jitter ±0.2, edge enhancement, grayscale conversion with a probability of 0.3, and 3×3 kernel Gaussian blur (σ=0-1). In this embodiment, two strong enhancement samples are generated for each image, forming a batch size. B =8 sample set The sample includes 4 labeled samples and 4 unlabeled samples, forming a strongly enhanced sample.

[0016] Step 2.2: Follow the marked scale Generate a variable rectangular mask with training round t For a 1 / 8 labeling rate, the initial quantity When training reaches 50 rounds ,in, The total number of rounds is 100 in this embodiment. Each rectangle is assigned a random area ratio Aspect Ratio Calculate width ,high ,in, Indicates uniform distribution. It is the minimum allowed side length, ensuring that a large base region can be preserved even with a low annotation ratio to maintain semantic continuity. This indicates the number of pixels in the vertical direction of the image. This represents the number of pixels in the horizontal direction of the image, with the top-left corner coordinates randomly selected. ; A confidence mask is generated by predicting the confidence matrix C from a weakly enhanced unlabeled remote sensing image. For each pixel in C, the mask value is set to 1 if the confidence level is less than a preset threshold, and 0 otherwise. In this embodiment, the preset threshold is 0.25. Step 2.3: Merge the two masks to obtain ,in, For the final mask region, the operator This indicates that two matrices of the same shape are operated on; based on the final mask region, samples are mixed across batches, and if the b-th sample in the batch is even, it is processed according to... Mixed, in which, This indicates a circular index within the batch. If the b-th sample in the batch is odd, then it is indexed by... This process involves mixing to generate a multi-domain hybrid strongly enhanced sample set. .

[0017] Step 3: Based on the multi-domain hybrid strongly enhanced sample set and weakly enhanced samples, obtain the prototype consistency loss according to the category prototype; Step 3.1: Initialize and update category prototypes: Set K prototypes for each category c. ,in, The feature dimension is represented by an initial prototype randomly selected from the output features of the labeled data encoder; after each training round, the strongly enhanced feature z of the unlabeled samples is executed. Normalize, calculate its cosine similarity with all prototypes and assign it to the prototype category with the highest similarity, according to... Update, in which, For the updated prototype, The momentum coefficient, It is the original feature vector go through The normalized mean vector is then used to assign the vector to the prototypes using online clustering, wherein the online clustering is performed by calculating... The similarity to each prototype will Assign the model to the prototype with the highest similarity and update the model using an exponential moving average; Step 3.2: Calculate the prototype prediction probability: for strongly enhanced sample features Calculate the cosine similarity between it and the K=3 prototypes in each category, take the maximum value for each category, and generate the prototype prediction probability, expressed as: in, For a certain pixel to belong to the first The probability of a class It is a function that calculates the cosine similarity between two vectors. An index of the most similar prototype in each category. For temperature parameters; Step 3.3: Calculate prototype consistency loss: Label the dataset according to... Calculate the loss, where, Indicates the number of labeled data. For CE loss, For accurate labeling, This represents the non-parametric semantic segmentation result of the labeled data stream; For unlabeled datasets, first... Generate pseudo-tags, where, This indicates weak enhancement of execution. It is the first An unlabeled input image, This represents a neural network model used for semantic segmentation. These are the trainable parameters of the model. This indicates taking the category corresponding to the highest probability, and then pressing... Calculate the loss, where, Indicates the number of unlabeled data. It is a general representation of unlabeled data, referring to any single unlabeled data point. This represents the non-parametric semantic segmentation result of the unlabeled data stream, thus yielding the prototype consistency loss as... .

[0018] Step 4: Based on the weakly enhanced samples, generate a reliable region mask, filter out false label noise, and obtain the loss after filtering out noise in the unlabeled data; Step 4.1: Calculate the weak enhancement entropy graph ,in, express The Middle Predicted probability of class This represents the segmentation result predicted by the weak enhancement model; Step 4.2: Generate a category mask pixel by pixel, the expression is: in, This indicates that the confidence matrix predicted by the model is used to generate the category. The mask, extract Composed of pixel entropy values ​​of 1 , Indicate category The set of entropy values; Step 4.3: Set the category entropy threshold: If ,Pick The 40th percentile value , Represents the category entropy tensor, otherwise ;according to Generate an entropy mapping graph, where, Represents an entropy mapping diagram. The confidence mask is then used to generate the reliable region mask. The expression is: Finally, the loss after filtering out noise from the unlabeled data was calculated. ,in, For the first Unlabeled samples Pseudo-labels generated by model prediction.

[0019] Step 5: Construct the total loss, iteratively train the network based on the total loss, update the model parameters and category prototypes, and output the semantic segmentation model; wherein, the total loss includes labeled data cross-entropy loss, unlabeled data noise filtering loss, prototype consistency loss, feature-prototype comparison loss, and similar prototype compactness loss; Optionally, the total loss is constructed. In this embodiment, the SGD optimizer is used with an initial learning rate of 0.001, weight decay of 1e-4, and momentum of 0.9 to minimize the total loss. The expression for the total loss is: in, For the total loss, For the cross-entropy loss of labeled data, As weight, This refers to the loss after filtering out noise from unlabeled data. For prototype consistency loss, For feature-prototype contrastive loss, In this embodiment, a compact loss mechanism similar to that of the prototype is used. We set it to 0.5 to ensure a balance between supervised and unsupervised losses.

[0020] Specifically, the cross-entropy loss of the labeled data is achieved through... The feature-prototype contrastive loss was obtained through... The compact loss of the same prototype was obtained through We obtained, among which, Represents the query vector. An index of the most similar prototype in each category. For temperature parameters, Representation and query vector The prototype vector with the highest similarity.

[0021] Optionally, the iterative training network loads 8 samples (4 labeled and 4 unlabeled) in batches for each round, performs Step 2-Step 4 to calculate each loss, updates the segmentation network parameters through backpropagation, updates the category prototypes, adopts a poly learning rate decay strategy, stops training after 100 rounds, and outputs a semantic segmentation model.

[0022] Step 6: Input the remote sensing image to be semantically segmented into the semantic segmentation model to obtain the semantic segmentation result of the remote sensing image.

[0023] Based on the specific implementation details, the effectiveness of the technical solution of the present invention will be demonstrated through experiments.

[0024] The dataset used in this embodiment is the Vaihingen dataset, which contains 33 high-resolution aerial images with a spatial resolution of 9 cm, spectral bands of near-infrared, red, and green channels, and single image sizes ranging from 1996×1995 to 3816×2550 pixels; semantic categories are 5: Impervious Surfaces (SF), Buildings (BD), Low Vegetation (LV), Trees (Tree), and Cars (Car), and pixel-level ground truth values ​​are provided for model evaluation.

[0025] Specifically, the class prototype-based remote sensing image semantic segmentation method (CPDM) of this invention is compared with other state-of-the-art SSL methods on the Vaihingen dataset under different label ratios. The results are shown in Table 1, with the best results displayed in bold.

[0026] Table 1. Comparison of CPDM with five state-of-the-art methods on the Vaihingen dataset at different labeling rates.

[0027] As shown in the table, at label rates of 1 / 8 and 1 / 4, CPDM achieves improvements of 8.02% and 4.76% respectively compared to the supervised training baseline. Furthermore, compared to other state-of-the-art methods, CPDM boasts the highest mIoU (mean Intersection over Union) across all classes. Moreover, CPDM achieves improvements of 16.94% and 9.72% on the Car class compared to the supervised baseline at label rates of 1 / 8 and 1 / 4. These results demonstrate that while the effective information gain learned from unlabeled data decreases with increased labeled data, this invention can still effectively utilize limited labeled data and improve network performance.

[0028] Furthermore, Figure 3 and Figure 4 This visually demonstrates the application of the CPDM method in different... and Performance of remote sensing image segmentation under parameter combinations, where, It is the confidence level value. It is the percentage of category entropy, such as Figure 3 As shown, when , At that time, CPDM achieved state-of-the-art performance on the iSAID and DFC22 datasets, with mIoU of 64.22% and 40.58%, respectively. On the Vaihingen dataset, CPDM showed better performance with varying parameter combinations. , It achieved optimal performance, with an mIoU of 70.98%. However, when the parameters were adjusted to... , The model's performance decreased by only 0.26%, and mIoU remained at 70.72%. However, when , At that time, CPDM's mIoU decreased by 4.09% and 2.77% on iSAID and DFC22, respectively. Figure 4 The segmentation performance of CPDM under different parameters is clearly visible in the visualizations of the various datasets. The experimental data above fully demonstrates that the category prototype-based semantic segmentation method for remote sensing images can solve the bottleneck problems faced in semantic segmentation tasks, efficiently utilize unlabeled data to improve segmentation accuracy and robustness, and reduce annotation costs.

[0029] In summary, this invention addresses the challenges of high annotation costs, limited perturbation space in existing semi-supervised methods, and difficulty in filtering out pseudo-label noise in remote sensing image semantic segmentation. It achieves performance improvement through a three-layer core module: First, multi-domain CutMix hybrid strong enhancement (MCS) dynamically expands the perturbation space based on a dual mask (variable rectangular mask + confidence mask) to retain high-confidence regions. Second, prototype-based prediction consistency (PBC) maintains class prototypes to align parametric and non-parametric predictions, reducing reliance on single strong / weak consistency methods. Finally, class adaptive entropy mapping (CAEM) dynamically generates entropy thresholds for each class, finely filtering out pseudo-label noise. Through layer-by-layer optimization and multi-module collaboration, this method significantly outperforms existing state-of-the-art methods on the mainstream remote sensing dataset Vaihingen, efficiently utilizing unlabeled data to improve segmentation accuracy and robustness while reducing annotation costs.

[0030] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A semantic segmentation method for remote sensing images based on category prototypes, characterized in that, The method includes the following steps: Step 1: Preprocess the original remote sensing image to obtain a preprocessed dataset; wherein, the dataset includes a labeled dataset and an unlabeled dataset; Step 2: Perform weak and strong augmentation on the unlabeled dataset and generate a double mask to generate a multi-domain mixed strong augmentation sample set and weak augmentation samples; wherein, the double mask is a variable rectangular mask and a confidence mask; Step 3: Based on the multi-domain hybrid strongly enhanced sample set and weakly enhanced samples, obtain the prototype consistency loss according to the category prototype; Step 4: Based on the weakly enhanced samples, generate a reliable region mask, filter out false label noise, and obtain the loss after filtering out noise in the unlabeled data; Step 5: Construct the total loss, iteratively train the network based on the total loss, update the model parameters and category prototypes, and output the semantic segmentation model; wherein, the total loss includes labeled data cross-entropy loss, unlabeled data noise filtering loss, prototype consistency loss, feature-prototype comparison loss, and similar prototype compactness loss; Step 6: Input the remote sensing image to be semantically segmented into the semantic segmentation model to obtain the semantic segmentation result of the remote sensing image; Step 2 specifically includes: Step 2.1: For each unlabeled remote sensing image Perform weak and strong enhancements, generating several strongly enhanced samples, forming a batch with a size of [size missing]. sample set ,in, This represents the b-th strongly augmented sample within the batch; Step 2.2: Follow the marked scale Generate a variable rectangular mask with training round t Initial quantity Number of rounds in the current round ,in, Total number of rounds; Each rectangle is assigned a random area ratio Aspect Ratio Calculate width ,high ,in, Indicates uniform distribution. It is the minimum allowed side length. This indicates the number of pixels in the vertical direction of the image. This represents the number of pixels in the horizontal direction of the image, with the top-left corner coordinates randomly selected. ; A confidence mask is generated by predicting the confidence matrix C from a weakly enhanced unlabeled remote sensing image. Iterate through each pixel in C, and set the mask value to 1 if the confidence level is less than the preset threshold, otherwise set it to 0. Step 2.3: Merge the two masks to obtain ,in, M final For the final mask region, the operator This indicates that two matrices of the same shape are operated on; based on the final mask region, samples are mixed across batches, and if the b-th sample in the batch is even, it is processed according to... Mixed, in which, This indicates a circular index within the batch. If the b-th sample in the batch is odd, then it is indexed by... This process involves mixing to generate a multi-domain hybrid strongly enhanced sample set. ; Step 3 specifically refers to: Set K prototypes for each category c ,in, The feature dimension is represented by an initial prototype randomly selected from the output features of the labeled data encoder; after each training round, the strongly enhanced feature z of the unlabeled samples is executed. Normalize, calculate its cosine similarity with all prototypes and assign it to the prototype category with the highest similarity, according to... Update, in which, For the updated prototype, The momentum coefficient, It is the original feature vector go through The normalized mean vector is then used to assign the vector to the prototypes using online clustering, wherein the online clustering is performed by calculating... The similarity to each prototype will Assign it to the prototype with the highest similarity and update that prototype using an exponential moving average.

2. The remote sensing image semantic segmentation method based on category prototypes according to claim 1, characterized in that, Step 3 also includes: Step 3.2: Enhance the features of strongly enhanced samples Calculate the cosine similarity between it and K prototypes of each category, take the maximum value for each category, and generate the prototype prediction probability, expressed as: ； in, For a certain pixel to belong to the first The probability of a class It is a function that calculates the cosine similarity between two vectors. An index of the most similar prototype in each category. For temperature parameters; Step 3.3: Label the dataset by Calculate the loss, where, Indicates the number of labeled data. For CE loss, For accurate labeling, This represents the non-parametric semantic segmentation result of the labeled data stream; For unlabeled datasets, first... Generate pseudo-tags, where, This indicates weak enhancement of execution. It is the first An unlabeled input image, This represents a neural network model used for semantic segmentation. These are the trainable parameters of the model. This indicates taking the category corresponding to the highest probability, and then pressing... Calculate the loss, where, Indicates the number of unlabeled data. It is a general representation of unlabeled data, referring to any single unlabeled data point. This represents the non-parametric semantic segmentation result of the unlabeled data stream, thus yielding the prototype consistency loss as... .

3. The remote sensing image semantic segmentation method based on category prototypes according to claim 2, characterized in that, Step 4 specifically refers to: Step 4.1: Calculate the weak enhancement entropy graph ,in, y i Indicates the first i The predicted probability of a class; Step 4.2: Generate a category mask pixel by pixel, the expression is: ； in, This indicates that the confidence matrix predicted by the model is used to generate the category. The mask, extract Composed of pixel entropy values ​​of 1 , Indicate category The set of entropy values; Step 4.3: If ,Pick The 40th percentile value , Represents the category entropy tensor, otherwise ;according to Generate an entropy mapping graph, where, T map Represents an entropy mapping diagram. The confidence mask is then used to generate the reliable region mask. The expression is: ； Finally, the loss after filtering out noise from the unlabeled data was calculated. ,in, For the first Unlabeled samples Pseudo-labels generated by model prediction.

4. The remote sensing image semantic segmentation method based on category prototypes according to claim 3, characterized in that, The expression for the total loss is: ； in, For the total loss, For the cross-entropy loss of labeled data, As weight, This refers to the loss after filtering out noise from unlabeled data. For prototype consistency loss, For feature-prototype contrastive loss, It is a compact loss model of the same type.

Images