Remote sensing image semantic segmentation method and device based on frequency enhancement and uncertainty-aware contrast refinement

Abstract

The application discloses a remote sensing image semantic segmentation method and device based on frequency enhancement and uncertainty perception contrast refinement, and the method comprises the following steps: extracting features from an input remote sensing image to obtain a patch token sequence; performing processing by using a multi-anchor frequency enhancement module, including: converting the features to a frequency domain, applying a low-pass filter to retain low-frequency components, fusing the features converted back to the spatial domain with original features, and performing anchor point interaction by using a plurality of class corresponding cls tokens to generate enhanced feature representation; processing the enhanced feature representation by using an uncertainty perception contrast module, including: providing prior knowledge by using a pre-trained teacher model, calculating uncertainty weights based on prediction entropy, constructing a confidence weighted contrast learning loss to refine the features, and generating a semantic segmentation result. The method can inhibit noise by introducing a frequency enhancement mechanism, can enhance class distinction by combining multi-anchor interaction, and can optimize feature representation by using uncertainty perception contrast learning, so that more accurate semantic segmentation can be finally realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of pattern recognition and machine learning, in particular to the field of weakly supervised technology, and more particularly to a remote sensing image semantic segmentation method and device based on frequency enhancement and uncertainty perception contrast refinement. BACKGROUND

[0002] With the rapid development of remote sensing technology, it has become increasingly convenient to obtain high-resolution remote sensing images. However, due to the wide coverage of remote sensing images, the complex ground feature elements, and the diverse target scales, pixel-level fine labeling requires extremely high professional manpower and time cost. To reduce the labeling cost, researchers turn to weakly supervised learning, in which image-level labels have become a research hotspot due to their low labeling cost.

[0003] In recent years, the visual Transformer (ViT) architecture has been introduced into the field of weakly supervised semantic segmentation, and its strong global context modeling capability provides new possibilities for generating more refined class activation maps. In the prior art, there are related documents based on visual transformer (ViT) that directly synthesize pseudo segmentation masks by using multiple class-specific cls labels to generate self-attention maps; there are also technologies that exploit the potential of CLIP in patch-text alignment, use large language models to enhance text semantics and visual calibration, and achieve good results in weakly supervised semantic segmentation tasks with lower training cost. There are also related documents that disclose that by introducing multiple prototypes for each class and using an optimal transport mechanism for similarity perception to allocate features, the quality of pseudo labels is improved.

[0004] However, the above prior art still has the following disadvantages:

[0005] (1) High-frequency noise interference problem: Remote sensing images differ from natural images in that they contain a large amount of high-frequency details and complex textures. The self-attention mechanism of existing methods lacks sufficient inductive bias and effective regularization, and when calculating global dependencies, it is easily disturbed by high-frequency noise and irrelevant background clutter in the image, resulting in poor quality of the generated class activation map, and the problems of fuzzy response area and noise scatter points are common.

[0006] (2) Semantic confusion and boundary blurring: In complex remote sensing scenes, different classes may share similar high-frequency textures, and the high-frequency information of target boundary regions is interwoven, so the pseudo labels generated by existing methods have low confidence and insufficient distinguishability at object boundaries, resulting in serious semantic blurring and fragmentation of the final segmentation results. SUMMARY

[0007] The application aims to provide a remote sensing image semantic segmentation method and device based on frequency enhancement and uncertainty-aware contrast refinement, and aims to solve the problems of high-frequency noise sensitivity, boundary blurring and semantic confusion in the existing weakly supervised remote sensing image segmentation method. By introducing a frequency enhancement mechanism to suppress noise, combining multi-anchor interaction to enhance class distinction, and using uncertainty-aware contrast learning to optimize feature representation, more accurate semantic segmentation is ultimately achieved.

[0008] In order to achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0009] In a first aspect, the application provides a remote sensing image semantic segmentation method based on frequency enhancement and uncertainty-aware contrast refinement, comprising the following steps:

[0010] Extracting features from the input remote sensing image to obtain a patch token sequence;

[0011] Processing the patch token sequence through a multi-anchor frequency enhancement module, including: converting the features to the frequency domain, applying a low-pass filter to retain low-frequency components, fusing the features back to the spatial domain after inverse transformation, and performing anchor interaction using multiple class corresponding cls tokens to generate enhanced feature representation;

[0012] Processing the enhanced feature representation through an uncertainty-aware contrast module, including: using a pre-trained teacher model to provide prior knowledge, calculating uncertainty weights based on prediction entropy, and constructing a confidence weighted contrast learning loss to refine the features;

[0013] Generating a semantic segmentation result based on the refined features.

[0014] In one embodiment, the features are converted to the frequency domain, a low-pass filter is applied to retain low-frequency components, and the features are fused back to the spatial domain after inverse transformation; specifically including:

[0015] Projecting the patch token sequence to obtain: ; The feature map after projecting the patch token sequence output by the last layer of ViT is represented as ; R represents the set of real numbers, D represents the number of feature channels, and represents the length of the feature vector at each spatial position; H and W represent the height and width of the feature map, respectively;

[0016] Applying Fourier transform to obtain frequency domain representation: ; The coefficient of each position in the frequency domain is calculated as follows:

[0017]

[0018] h, w represent spatial position indexes, h e (0, H-1), w e (0, W-1); d represents the index of the feature channel;

[0019] A circular low-pass filter mask is designed to filter out high-frequency components and retain low-frequency regions centered on the frequency domain;

[0020]

[0021] wherein, represents the ratio for controlling the low-frequency range, (H c ,W c )=(H / 2,W / 2) represents the center of the frequency domain;

[0022] Inverse Fourier transform is performed on the frequency domain signal after the mask to obtain a low-frequency enhanced feature:

[0023]

[0024] wherein, represents element-wise multiplication; the original feature is fused with the low-frequency feature to obtain a stronger patch token representation:

[0025]

[0026] wherein, represents a hyperparameter for controlling the low-frequency enhancement strength.

[0027] In one embodiment, the anchor point interaction includes:

[0028] The cosine similarity of each cls token and patch token representation P is calculated, and the K patch tokens with the highest similarity are selected to form a neighbor set;

[0029] Through a soft fusion strategy and a nonlinear gating mechanism, an anchor point representation is generated, and a multi-classification regularization loss is applied to improve the inter-class discrimination.

[0030] In one embodiment, the function expression of the multi-classification regularization loss is as follows:

[0031]

[0032] wherein, is a balance factor, is a multi-label classification loss; represents a regularization loss, ;

[0033] wherein, B is the batch size, CE is the cross-entropy loss function; S a represents the cosine similarity between two anchors, To regularize the label.

[0034] In one embodiment, the processing procedure in the uncertainty-aware contrastive module includes:

[0035] Using the DINO model as the teacher model, the feature similarity is extracted;

[0036] Generating initial semantic labels according to the class activation map, and combining the teacher model feature similarity to correct the positive and negative sample pairs;

[0037] Based on the uncertainty weight, the contribution degree of the sample pairs in the contrastive loss is dynamically adjusted;

[0038] Using DINOv3 as a fixed teacher model, the feature representation of the input remote sensing image is extracted;

[0039] In the student network, the intermediate layer features are extracted to obtain the corresponding class activation map, and the initial semantic labels are generated by threshold processing to preliminarily define the relationship between positive and negative sample pairs;

[0040] The cosine similarity between the features of the teacher model is calculated, and the threshold is dynamically adjusted based on the batch average value; if the initial semantic labels of two patch tokens are the same, or the feature similarity is higher than the batch average value, it is determined as a positive sample pair, otherwise as a negative sample pair;

[0041] Combined with the uncertainty estimation based on the predictive entropy, a confidence-weighted contrastive learning loss is constructed.

[0042] In one embodiment, the function of the contrastive learning loss is as follows:

[0043]

[0044] represents the cosine similarity between each pair of features, P and N represent positive and negative sample pairs respectively; is a stability constant, and then converted into confidence; represents the confidence weight matrix of the pair.

[0045] In one embodiment, the method comprises a regularization loss of a multi-anchor frequency enhancement module and a contrastive loss of an uncertainty-aware contrastive module during training.

[0046] In a second aspect, the embodiments of the present application also provide a remote sensing image semantic segmentation device based on frequency enhancement and uncertainty-aware contrastive refinement, comprising:

[0047] The feature extraction module is used to extract features from the input remote sensing image to obtain a patch token sequence.

[0048] The multi-anchor frequency enhancement module is used for processing the patch token sequence, and includes: converting features to a frequency domain, applying a low-pass filter to reserve low-frequency components, fusing the features with original features after inverse transformation back to a spatial domain, and generating enhanced feature representations by anchor interaction with a plurality of class corresponding cls tokens;

[0049] The uncertainty perception contrast module is used for processing the enhanced feature representations, and includes: providing prior knowledge by using a pre-trained teacher model, calculating uncertainty weights based on prediction entropy, and constructing a confidence weighted contrast learning loss to refine features.

[0050] The segmentation output module is used for generating a semantic segmentation result based on the refined features.

[0051] In a third aspect, an electronic device is provided, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of the embodiments of the first aspect.

[0052] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the method according to any one of the embodiments of the first aspect.

[0053] According to the above technical solutions, compared with the prior art, the present application has the following technical advantages:

[0054] The multi-anchor frequency enhancement module is proposed to suppress high-frequency noise and enhance semantic consistency, the uncertainty perception contrast module is designed to optimize feature learning by using prior knowledge of a teacher model, and the overall method performs excellently on multiple data sets and has strong practicality and promotional value. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0056] Figure 1 A weakly supervised remote sensing image semantic segmentation method flowchart provided in the embodiments of the present application;

[0057] Figure 2 An overall algorithm framework diagram provided in the embodiments of the present application;

[0058] Figure 3 A Potsdam data set segmentation result diagram provided in the embodiments of the present application;

[0059] Figure 4 Fig. 1 is an iSAID dataset segmentation result diagram provided in an embodiment of the present application;

[0060] Figure 5 Fig. 2 is a weakly supervised remote sensing image semantic segmentation device block diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0062] The embodiments of the present application disclose a remote sensing image semantic segmentation method based on frequency enhancement and uncertainty perception contrast refinement, as shown in Fig. 1, which comprises the following steps: Figure 1

[0063] S1, extracting features from an input remote sensing image to obtain a patch token sequence; patch token refers to a small block or segment obtained by dividing input data such as images or texts into basic units for model processing.

[0064] S2, processing the patch token sequence through a multi-anchor frequency enhancement module, including: converting the features to the frequency domain, applying a low-pass filter to retain low-frequency components, fusing with the original features after inverse transformation back to the spatial domain, and utilizing multiple class corresponding cls tokens for anchor interaction to generate enhanced feature representation; wherein cls token is a design in the ViT model, mainly used to aggregate global information of the entire input sequence as the core basis for classification or task output.

[0065] S3, processing the enhanced feature representation through an uncertainty perception contrast module, including: utilizing a pre-trained teacher model to provide prior knowledge, calculating uncertainty weight based on prediction entropy, and constructing confidence weighted contrast learning loss to refine the features;

[0066] S4, generating a semantic segmentation result based on the refined features.

[0067] Referring to Fig. 2, the weakly supervised remote sensing image semantic segmentation device provided in an embodiment of the present application comprises a feature extraction module 1, a multi-anchor frequency enhancement module 2, an uncertainty perception contrast module 3, and a semantic segmentation result generation module 4. Figure 2 ​As shown, the overall framework diagram of the algorithm is shown, the visual Transformer is used as the backbone network to extract the patch token sequence of the input remote sensing image; first, the input image is input to the ViT encoder, the final layer token of the ViT is filtered in the frequency domain, the Fourier forward and inverse transformation and the LPF low-pass filter are passed through in it to suppress high-frequency noise, then, the multi-anchor frequency enhancement module is entered, the anchor points are constructed by utilizing the relationship between the patch and the cls token, and is applied to the loss function To achieve regularization. Secondly, the DINO self-supervised encoder and the features from the intermediate layers of the ViT are used to generate class activation maps, after which the uncertainty-aware contrastive module evaluates the discriminative confidence of each patch pair and assigns a lower weight to the token pair with lower confidence in the contrastive loss During the training process And .

[0068] The two important modules and technical effects of the application will be described in detail as follows:

[0069] 1. Multi-anchor frequency enhancement module

[0070] In order to alleviate the problem that the ViT is easily disturbed by high-frequency noise in complex remote sensing scenes, the multi-anchor frequency enhancement module is proposed in the embodiment. The method enhances the semantic consistency of the feature by extracting the low-frequency component, and improves the inter-class discriminability by using multi-anchor interaction.

[0071] Given the patch token sequence output by the last layer of the ViT, it is projected to obtain , R represents the set of real numbers, D represents the number of feature channels, and represents the length of the feature vector at each spatial position; H and W represent the height and width of the feature map respectively. First, it is converted to the frequency domain by applying the Fourier transform:

[0072]

[0073] Specifically, the coefficient of each position in the frequency domain is calculated as follows:

[0074]

[0075] Wherein, represents the height and width in the frequency domain, d represents the index of the feature channel, and is selected from the set (0, 1 / H,..., H-1 / H) (0, 1 / W,..., W-1 / W). In order to highlight the low-frequency semantic information, we move the zero-frequency component to the center and design a low-pass filter mask to filter out high-frequency components and only keep low-frequency components near the center.

[0076]

[0077] where, is the ratio for controlling the low-frequency range, (H c ,W c )=(H / 2,W / 2) represents the center of the frequency domain, and the inverse transform is performed on the masked frequency domain signal to obtain the enhanced spatial domain feature:

[0078]

[0079] where, represents element-wise multiplication. Finally, the original feature and the low-frequency enhanced feature are fused to obtain a patch token representation with stronger robustness:

[0080]

[0081] where, is a hyperparameter that controls the low-frequency enhancement intensity.

[0082] To further improve the inter-class discrimination ability, there are C class-specific cls tokens. First, calculate the cosine similarity between each projected cls token and all P , find the most relevant P , and let the aggregation process only focus on the P with the most semantic association:

[0083]

[0084] where, is the projected cls token, represents the jth patch token, and P is the patch token fused with the low-frequency feature described above. For each , select the top K P with the highest similarity to form the set , and then use a soft fusion strategy to fuse the information of the neighbors with high confidence. When the neighbor confidence is high, more reliance is placed on the neighbors, otherwise the own information is retained:

[0085]

[0086] Add the fused information to the original cls token and pass it through the tanh function:

[0087]

[0088] where, tanh represents the activation function tanh, By the above calculation, is the projected cls token. Subsequently, a weight factor H is calculated using a nonlinear gating vector, so as to more finely determine which neighbors ultimately contribute more. In this way, a preliminary Anchor representation is obtained:

[0089]

[0090]

[0091] Finally, through the combination of addition and multiplication, both linear combination information and high-order interaction information are retained to obtain the final Anchor representation:

[0092]

[0093] wherein, represents a learnable projection matrix, represents a LeakyReLU activation function, and different subscripts represent two different variables; is the projected cls token, is the preliminary Anchor calculated by the above formula.

[0094] Then we apply a multi-classification regularization loss function to each anchor A i， After averaging in the feature dimension, the cosine similarity between each pair of anchors is calculated,

[0095]

[0096] wherein τ is a temperature coefficient, and the regularization loss function is defined as:

[0097]

[0098] wherein, is a regularization label, B represents a batch size, and CE represents a cross-entropy loss function, which is used to guide each anchor to be most similar to itself and as different as possible from other anchors. The total loss function is composed of the regularization loss and the multi-label classification loss:

[0099]

[0100] wherein, is a balance factor, is the multi-label classification loss.

[0101] 2. Uncertainty-aware contrastive module

[0102] To improve the accuracy and robustness of the initial class activation map generated pseudo labels, we design an uncertainty-aware contrastive module. This module provides prior knowledge rich in semantic structure without human annotation by introducing a pre-trained DINO self-supervised encoder as a teacher model, and combines uncertainty estimation based on predictive entropy to construct a confidence-weighted contrastive learning framework to refine the patch token feature representation output by the backbone network.

[0103] We use DINOv3 as a fixed teacher model. Given the same input remote sensing image I, it is input into the teacher network and the student network (i.e., the main segmentation network) respectively.

[0104] In the student network, the intermediate layer features are extracted, and then the corresponding class activation map is obtained. We use the class activation map to generate the initial semantic prediction by thresholding. If the two labels C are the same, they are considered as positive sample pairs.

[0105] If there is no common semantic label C in the semantic label, it may be that the intermediate layer features are still uncertain. We further use the features D output by the teacher network to calculate a batch average of feature similarity T , and then calculate the direct feature similarity between each pair of patch tokens. If the similarity is greater than the average T , they are still considered as positive pairs. As follows:

[0106]

[0107] where, Y equal means that the patch tokens form a positive pair, otherwise a negative pair, sim represents similarity calculation, D represents the patch token feature output by the self-supervised encoder, C represents the semantic label, T represents the batch feature average, as follows:

[0108]

[0109] Here, V represents the set of all possible patch pairs. This mechanism ensures that even if the initial class activation map CAM prediction has uncertainty, the positive and negative sample relationships can still be corrected by the feature similarity of the teacher network.

[0110] With the positive and negative sample pairs, we combine the uncertainty estimation based on predictive entropy to construct a confidence-weighted contrastive learning loss. This loss function dynamically adjusts the contribution of each sample in the contrastive learning process using uncertainty measures, allowing the model to focus on learning discriminative features in high-confidence regions while suppressing the negative impact of noisy samples.

[0111] First, the intermediate layer activation map Probability normalization is performed:

[0112]

[0113] For each spatial position, calculate its information entropy:

[0114]

[0115] Where C represents the total number of classes, c represents a specific class, p From the above formula, is a stability constant, and then converted to confidence,

[0116]

[0117] In each batch, based on individual confidence, we construct a pair-wise confidence weight matrix:

[0118]

[0119] Where con is obtained from the above formula, and T represents the transpose. Based on the constructed positive and negative sample relationships and confidence weights, we propose an uncertainty-aware contrastive loss function:

[0120]

[0121] Where, represents the cosine similarity between each pair of features, P and N represent the positive and negative sample pairs respectively. After multiplying the confidence map with the similarity of positive and negative sample features, the differential contribution mechanism of the loss function for different confidence sample pairs is realized. High-confidence sample pairs obtain a larger weight and are focused on optimization during the training process, thereby accelerating feature learning in the determined area. Confidence sample pairs obtain a smaller weight, and their contribution is significantly inhibited, effectively preventing unreliable similarity constraints from interfering with model optimization and improving the robustness of the algorithm in semantic boundaries and complex scenarios.

[0122] 3. Technical effects:

[0123] We designed a weakly supervised semantic segmentation model for remote sensing images based on frequency enhancement and uncertainty-aware contrast refinement, which contains a multi-anchor frequency enhancement module and an uncertainty-aware contrast module. The total loss function in training is composed of the regularization loss of the multi-anchor frequency enhancement module and the contrast loss of the uncertainty-aware contrast module.

[0124] In the experimental part, two common remote sensing classification datasets, Potsdam dataset and iSAID dataset, were used in this embodiment. Among them:

[0125] Potsdam: The dataset is derived from high-resolution aerial images of the city of Potsdam, Germany, a total of 38, with a spatial resolution of 5 cm, each with a size of 6000*6000 pixels, covering six types of ground objects: impervious surface, building, low vegetation, tree, vehicle and miscellaneous. According to the official classification standard, 23 are used for training, 14 for verification, and another 7-10 are excluded due to labeling problems. For ease of training, we crop the original image into 256*256 non-overlapping subgraphs, finally generating 13,248 training samples and 8,064 verification samples.

[0126] iSAID dataset: iSAID dataset is a large-scale aerial image annotation-intensive dataset, including 2806 high-resolution images, 655451 object instances and a total of 16 categories, including background (BG), airplane (PL), baseball field (BD), ship (SH), track and field (GTF), basketball court (BC), football field (SBF), roundabout (RA), oil tank (ST), bridge (BR), large vehicle (LV), tennis court (TC), helicopter (HC), port (HA), small vehicle (SV) and swimming pool (SP). Similarly, crop to 512*512 size. Finally, 7500 training images are obtained, 1653 validation images and 1315 test images are obtained.

[0127] Among them, the segmentation performance of the present invention and the existing method on the Potsdam dataset and the iSAID dataset, respectively, as shown in Table 1 and Table 2.

[0128] Among them, the existing method introduction: Dupl published in 2024 CVPR, proposes a progressive learning dual student network framework, which supervises each other through multi-level features of parallel networks, and adaptively introduces more high-confidence pseudo labels to participate in supervision.

[0129] CTFA published in 2024 TGRS, proposes token contrast learning and clstoken contrast learning, which promotes the cam to generate a more complete target area.

[0130] WeCLIP+ published in 2025 TPAMI, aligns visual-text features through contrastive learning, and introduces dynamic prompt learning to optimize the text encoder, uses cross-modal alignment to enhance regional semantic localization, and reduces the dependence on manual annotation.

[0131] Excel published in 2025 CVPR, by mining the potential of CLIP in patch-text alignment, uses large language models to enhance text semantics and visual calibration, and achieves good results on weakly supervised semantic segmentation tasks with lower training cost.

[0132] MoRe published in 2025 AAAI, by introducing graph class representation and positioning guided regularization, effectively solves the artifact problem caused by insufficient attention modeling of ViT in weakly supervised semantic segmentation

[0133] Table 1 Comparison of segmentation performance on Potsdam dataset

[0134]

[0135] Table 2 Comparison of segmentation performance on iSAID dataset

[0136]

[0137] The comparison of segmentation performance on Potsdam dataset is shown in Table 1. The proposed method is better than the current optimal method in mIoU. As shown in Figure 3 , in large-scale scenarios, the segmentation results of the proposed method are more complete; in the left local area, the More method misjudges the low-lying vegetation class around the vehicle, while the proposed method effectively handles high-frequency details to achieve accurate segmentation of the vehicle target without introducing additional noise. In the right local area, due to the similarity of low-lying vegetation and tree features, the More method misclassifies part of the trees as low-lying vegetation, while the proposed method benefits from the frequency enhancement and uncertainty perception mechanism to accurately distinguish between the two, thus completely segmenting the tree target from the low-lying vegetation. The experimental results show that the proposed method has good robustness and generalization ability in complex remote sensing scenarios.

[0138] The comparison of segmentation performance on iSAID dataset is shown in Table 2. The proposed method is better than the current optimal method in mIoU. The visualization results are shown in Figure 4 , for large targets: the tennis court target in the first column, the More method has boundary adhesion phenomenon, resulting in multiple independent instances being incorrectly connected as a whole, while the proposed method has enhanced boundary recognition ability, completely and clearly segmenting all four instances with intact contours. The ship target in the second column is misclassified by the More method as other classes, while the proposed method accurately identifies and does not produce class confusion.

[0139] For small targets: In the fourth column, the More method has a problem of missed segmentation for very small targets, while the proposed method successfully segments all instances. In the fifth column, the More method incorrectly segments the helicopter target, while the proposed method achieves complete and accurate segmentation.

[0140] For morphologically special elongated targets: The port targets in the last column, the More method produces obvious artifact diffusion, leading to over-inflated segmentation regions and intrusion into the background. The proposed method, thanks to the effective constraint of high-frequency details by the frequency enhancement module and the precise feature discrimination brought by uncertainty-aware contrastive learning, clearly segments the three narrow targets, with accurate morphology and almost no background interference.

[0141] Therefore, on the Potsdam and iSAID remote sensing datasets, the proposed method outperforms the existing techniques in overall segmentation accuracy (mIoU). Our method shows excellent robustness and accuracy in segmentation tasks of different scales, morphologies, and categories. For example, in complex scenes, it can more accurately distinguish between trees and low-lying vegetation and reduce missed segmentation of small targets.

[0142] Based on the same inventive concept, the embodiments of the present application also provide a remote sensing image semantic segmentation device based on frequency enhancement and uncertainty-aware contrastive refinement. Since the principles of the problems solved by these devices are similar to the aforementioned weakly supervised remote sensing image semantic segmentation method, the implementation of the device can be referred to the implementation of the aforementioned method, and the repeated parts will not be described again.

[0143] Reference Figure 5 As shown in the figure, the remote sensing image semantic segmentation device based on frequency enhancement and uncertainty-aware contrastive refinement comprises:

[0144] A feature extraction module is configured to extract features from an input remote sensing image to obtain a patch token sequence.

[0145] A multi-anchor frequency enhancement module is configured to process the patch token sequence, including: converting the features to the frequency domain, applying a low-pass filter to retain low-frequency components, fusing the features back to the spatial domain with the original features, and performing anchor interaction using multiple class corresponding cls tokens to generate enhanced feature representations.

[0146] An uncertainty-aware contrastive module is configured to process the enhanced feature representations, including: using a pre-trained teacher model to provide prior knowledge, calculating uncertainty weights based on prediction entropy, and constructing a confidence weighted contrastive learning loss to refine the features.

[0147] A segmentation output module is configured to generate a semantic segmentation result based on the refined features.

[0148] The remote sensing image semantic segmentation device based on frequency enhancement and uncertainty perception contrast refinement provided by the embodiment of the present application realizes the hardware integration of the foregoing method through modular design, while maintaining the innovation advantages of the method, significantly improves the engineering application efficiency.

[0149] The present application also provides an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the weakly supervised remote sensing image semantic segmentation method as described above when executing the computer program.

[0150] The present application also provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the weakly supervised remote sensing image semantic segmentation method as described above.

[0151] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer usable program code.

[0152] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0153] These computer program instructions can also be stored in a computer readable memory that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product comprising instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0154] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1

[0155] The various embodiments in the specification are described in progressive order with reference to each embodiment, each embodiment emphasizing different aspects of the other embodiments, and the same reference numerals are used in different drawings for the same or similar elements. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts are referred to the method part.

[0156] The above description of disclosed embodiments enables a person skilled in the art to implement or use the invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.​​

Claims

1. A method for semantic segmentation of remote sensing images based on frequency enhancement and uncertainty-aware contrast refinement, characterized in that, The method comprises the following steps: S1, extracting features from an input remote sensing image to obtain a patch token sequence; S2, processing the patch token sequence through a multi-anchor frequency enhancement module, including: converting the features to the frequency domain, applying a low-pass filter to retain low-frequency components, fusing the original features after inverse transformation back to the spatial domain, and utilizing multiple class corresponding clstokens for anchor interaction to generate enhanced feature representation; S3, processing the enhanced feature representation through an uncertainty perception contrast module, including: utilizing a pre-trained teacher model to provide prior knowledge, calculating uncertainty weights based on prediction entropy, and constructing a confidence weighted contrast learning loss to refine the features; S4, generating a semantic segmentation result based on the refined features; In step S2, the features are converted to the frequency domain, a low-pass filter is applied to retain low-frequency components, and the original features are fused after inverse transformation back to the spatial domain; specifically including: The projection of the patch token sequence obtains: ; denotes the feature map after the projection of the patch token sequence of the last layer output of the ViT; R denotes the real set, D denotes the feature channel number, denotes the length of the feature vector of each spatial position; H and W respectively denote the height and width of the feature map. The Fourier transform is applied to obtain a frequency domain representation: The coefficient at each position in the frequency domain is calculated as follows: h and w represent spatial position indexes, h∈(0, H-1), w∈(0, W-1); d represents the index of a feature channel; A circular low-pass filter mask is designed to filter out high-frequency components and retain low-frequency regions centered on the frequency domain; wherein, represents a ratio for controlling the low frequency range, (H c ,W c )=(H / 2,W / 2) represents the frequency domain center; Inverse Fourier transform is performed on the masked frequency domain signal to obtain low-frequency enhanced features: wherein, denotes element-wise multiplication; and fusing the original features with the low-frequency features to get a stronger patch token representation: wherein, denotes a hyperparameter controlling the low-frequency enhancement strength; The processing process in the uncertainty perception contrast module in step S3 includes: Using a DINO model as a teacher model to extract feature similarity; Generating initial semantic labels according to class activation maps and correcting positive and negative sample pairs in combination with the feature similarity of the teacher model; Based on the uncertainty weight, the contribution degree of the sample pairs in the contrast loss is dynamically adjusted; Using DINOv3 as a fixed teacher model to extract feature representation of the input remote sensing image; Extracting intermediate layer features in the student network to obtain corresponding class activation maps, generating initial semantic labels through threshold processing, and defining the relationship between positive and negative sample pairs; Calculating the cosine similarity between the features of the teacher model, and dynamically adjusting the threshold value based on the batch average value; if the initial semantic labels of two patchtokens are the same, or the feature similarity is higher than the batch average value, it is determined as a positive sample pair, otherwise as a negative sample pair; Combined with the uncertainty estimation based on prediction entropy, a confidence weighted contrast learning loss is constructed.

2. The method of claim 1, wherein, The anchor interaction includes: Calculating the cosine similarity between each cls token and the patch token representation P, and selecting the K patchtokens with the highest similarity to form a neighbor set; Through a soft fusion strategy and a nonlinear gating mechanism, an anchor point representation is generated, and a multi-classification regularization loss is applied to improve the inter-class discrimination.

3. The method of claim 2, wherein, The function expression of the multi-classification regularization loss is as follows: wherein, is a balancing factor, is a multi-label classification loss; denotes a regularization loss, ; where B is batch size, CE is cross-entropy loss function; S a denotes the cosine similarity between two pairs of anchor points, is the regularization label.

4. The method of claim 3, wherein, The function of the contrast learning loss is as follows: denotes the cosine similarity between each pair of features, P and N denote the positive and negative sample pairs, respectively; is a stability constant, which is then converted into a confidence score; denotes the pair-wise confidence weight matrix.

5. The method of claim 1, wherein, The total loss function of the method during training is composed of the regularization loss of the multi-anchor frequency enhancement module and the contrast loss of the uncertainty perception contrast module.

6. The device for semantic segmentation of remote sensing images based on frequency enhancement and uncertainty-aware contrast refinement, characterized in that, The device comprises: A feature extraction module for extracting features from an input remote sensing image to obtain a patch token sequence; The multi-anchor frequency enhancement module is configured to process the patch token sequence, including: converting features to a frequency domain, applying a low-pass filter to reserve low-frequency components, fusing the features with original features after inverse transformation back to a spatial domain, and generating enhanced feature representations by anchor interaction with clstokens corresponding to multiple categories; The uncertainty-aware contrast module is configured to process the enhanced feature representations, including: providing prior knowledge by using a pre-trained teacher model, calculating uncertainty weights based on prediction entropy, and constructing a confidence-weighted contrast learning loss to refine features. The segmentation output module is configured to generate a semantic segmentation result based on the refined features.

7. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program, and the processor executes the computer program to implement the method of any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-5.

Images