Semantic segmentation method for weak supervision of point annotation

Abstract

The invention discloses a semantic segmentation method for weak supervision of point annotation. The semantic segmentation method comprises the following steps: S1, setting hyper-parameters of a model; s2, data preparation and normalization are carried out; s3, performing multi-scale feature coding based on the SGF-Encoder; s4, carrying out multi-level feature fusion decoding on the basis of HSA-Decoder; and S5, setting loss and training strategies. The invention provides a semantic segmentation method based on point annotation weak supervision, and the method comprises the steps: fusing the prior of edge / bilateral filtering, morphology and wavelet in the conventional image processing with a deep network structure, and forming a joint feature learning encoder (SGF-Ender) of "spectral local-global context-filtering prior". And a hierarchical structure perception decoder (HSA-Decoder) is matched to carry out multi-scale alignment and structure reconstruction, so that under the condition that only about 0.01% of pixel points are annotated, relatively high precision is obtained, and interpretability and robustness are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically to a point-annotated weakly supervised semantic segmentation method. Background Technology

[0002] Semantic segmentation of remote sensing images aims to assign a semantic category to each pixel, and is a crucial underlying technology for applications such as land cover monitoring, detailed urban mapping, ecological environment assessment, disaster emergency response, and extraction of transportation facility elements. With the diversification of acquisition methods and imaging carriers, high-resolution, multispectral, panchromatic, and SAR data are constantly being enriched, and image resolution is rapidly increasing from meter-level to sub-meter and centimeter-level. Scene details and texture complexity are significantly increasing, and traditional methods based on handcrafted features and shallow classifiers are no longer sufficient to meet practical needs in terms of inter-class separability, boundary delineation, and robustness. In recent years, although deep convolutional networks and Transformer structures have made significant progress under fully supervised conditions, their reliance on large-scale pixel-level accurate annotation leads to high costs, limited cross-regional generalization, and slow updates, making it difficult to adapt to rapidly changing surface environments and engineering deployment pace.

[0003] To reduce annotation costs, weakly supervised learning has gradually become a research hotspot. Weakly supervised labeling forms include image-level labels, block-level labels, coarse annotations (such as polygons and line segments), and a small number of keypoint / point annotations. Among these, point annotations have attracted attention due to their high annotation efficiency, ease of scene transfer, and suitability for crowdsourcing or semi-automatic generation. However, point annotations are naturally sparse, and the supervision signal is mainly concentrated in a small number of locations that are confirmed as positive samples. Unlabeled pixels lack direct constraints, and the model is prone to the following problems: (1) Incomplete boundaries: It is difficult for complex boundary areas such as thin lines, broken lines, and holes to propagate from sparse points to complete contours, resulting in artifacts such as "adhesion", "fracture" and "edge burrs"; (2) Internal inhomogeneity: Texture differences, shadows and sensor noise within the same type of area cause prediction to become patchy and porous; (3) Cross-class confusion: Spectral-texture similar categories such as roads / roofs, impermeable / bare ground, and low vegetation / farmland are prone to mutual erosion under point-based supervision; (4) Insufficient interpretability: The intermediate representations of end-to-end deep networks are difficult to intuitively correspond to geometric boundaries and spectral features, making engineering coordination and scene migration costs high.

[0004] Existing weakly supervised methods can be broadly categorized into three types. The first type is based on region growing or pseudo-label generation, using graph-based propagation, superpixel consistency, and CRF / dense conditional random field post-processing to extend point annotations into dense supervision. However, this type of method is sensitive to hyperparameters and prone to erroneous expansion in noisy, occluded, and complex structural scenes. The second type emphasizes network structure and loss design, introducing attention, boundary branches, prototype contrastive learning, confidence constraints, region consistency, and sparsity regularization into the backbone network to mitigate the drift caused by sparse supervision. However, without explicit boundaries and frequency domain priors, the network may still fall into local optima of "texture equivalence." The third category focuses on the fusion of multi-source information and prior knowledge, such as complementing optics and SAR, using multi-scale morphology, and introducing learnable filters or frequency domain transformations, in order to enhance structural perception and denoising capabilities in the absence of dense labels. However, existing solutions often simply connect prior knowledge and deep features in parallel or couple them in post-processing, failing to achieve integrated collaborative optimization of "prior knowledge-feature-decoding" within an end-to-end framework, resulting in the inability to fully release the value of prior knowledge.

[0005] At the level of specific challenges, the first is the difficulty of boundary preservation; high-resolution images contain a large number of slender features (roads, canals, roof edges, fences, etc.), and traditional downsampling will lose gradients and edge strengths in shallow layers; at the same time, point supervision is difficult to cover key boundary sites, resulting in insufficient information in the upsampling recovery stage. The second is noise and texture interference; shadows, vegetation textures, and repetitive roofs introduce high-frequency artifacts. Without explicit modeling and selective suppression of high / low frequencies, the network is prone to overfitting local textures, causing uneven predictions within regions. The third is the lack of global context; local convolution alone is insufficient to establish "shape-structure" consistency across building clusters and street blocks; while windowed attention can capture long-range dependencies, under sparse supervision, the learning of attention weights is easily influenced by noise, requiring the introduction of interpretable boundary / frequency domain constraints for "correction." Furthermore, sparse supervision and positive / negative imbalance bring challenges to loss design: how to reliably converge at point locations while applying gentle and stable consistency / prior constraints to unlabeled areas is key to the viability of weakly supervised methods. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application provides a semantic segmentation method with weak supervision based on point annotation. It integrates priors from traditional image processing, such as edge / bilateral filtering, morphology, and wavelet analysis, with deep network structures to form a joint feature learning encoder (SGF-Encoder) based on "spectral local—global context—filter prior". This encoder is then combined with a hierarchical perceptual decoder (HSA-Decoder) for multi-scale alignment and structural reconstruction. As a result, it achieves high accuracy and significantly improves interpretability and robustness with only about 0.01% pixel annotation.

[0007] A weakly supervised semantic segmentation method based on point annotations includes the following steps: S1. Model hyperparameter settings; S1-1: Basic Channel Construction and Phase Expansion; S1-2: Employs a self-attention structure; S1-3: Set up training and inference; S2, Data Preparation and Normalization; S2-1: Remote sensing image cropping and data partitioning; S2-2: Normalization and Alignment; S3. Multi-scale feature encoding based on SGF-Encoder; S3-1: Shallow embedding; S3-2: Three-branch joint feature learning using the SGF-FL module; S3-3: Wavelet-preserving downsampling is performed using the WPD module; S3-4: Multi-stage progressive coding; S3-5: Deep semantic extraction; S4. Multi-level feature fusion decoding based on HSA-Decoder; S4-1: Shallow edge enhancement and noise reduction; S4-2: Overlapping patch embedding is used for mid-to-deep features to achieve cross-stage semantic alignment and context modeling; S4-3, multi-scale stitching and structural perception fusion; S4-4: Upsampling reconstruction and classification output; S5. Setting the loss and training strategy.

[0008] In step S1-1, a multi-stage encoder is constructed starting with the shallowest feature channel number C=32. Each time the next stage is entered, the spatial resolution of the feature map is reduced by 1 / 2 and the number of channels is expanded by 2 times, thus forming a hierarchical pyramid of "resolution decrease and channel increase". In steps S1-2, in order to establish long-distance dependencies with lower complexity, a windowed multi-head self-attention structure is adopted in the Transformer feature extraction layer of the multi-stage encoder; wherein, each Transformer feature extraction layer is fixed with 8 attention heads and the window side length is 8. In steps S1-3, the Adam optimizer is used during the training phase with an initial learning rate of 0.001 and a weight decay of 1e-4. The learning rate is gradually reduced by combining a cosine annealing strategy to achieve smooth convergence. Cross-entropy loss is used as the method for calculating the model optimization loss. In step S2-1, in order to balance memory usage and scene integrity, the original remote sensing image is cropped into 256×256 patches, or when resources allow, the original remote sensing image is cropped into 512×512 patches. The patches are divided into three datasets, namely training set, validation set and test set, in a ratio of 6:2:2. The patches in each dataset are labeled using dot annotation. The point annotations are performed by randomly sampling within the valid regions corresponding to different categories, with the total labeled pixels accounting for 0.01%, and the number of categories (numclass) is set for each dataset. In order to reduce training bias, it is necessary to maintain the balance between different categories during the sampling process. In step S2-2, the input patch is normalized to zero mean and unit variance in the channel dimension to eliminate the overall brightness and contrast drift caused by differences in imaging conditions; if necessary, global brightness / contrast normalization or histogram matching can be further used to alleviate domain differences across scenes and time phases; finally, the training sample image X0∈R is obtained for model training. H×W×3 Where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively. The aforementioned steps can provide a numerically stable starting point for subsequent multi-scale feature encoding.

[0009] In step S3-1, multi-channel semantic features are extracted from the input training sample image features, and the spatial resolution is reduced. Specifically, shallow embedding uses two 3×3 convolutions combined with normalization to perform multi-channel semantic priming on the training sample images. The stride of the first convolution is set to 2, which not only aligns color and texture but also provides a smooth and noise-controlled base for the subsequent SGF-FL module. In this stage, the spatial resolution becomes half that of the input, and the output is F0∈R. (H / 2)×(W / 2)×32 , where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively; In step S3-2, at each encoding stage, the three feature extraction branches of the SGF-FL module are expanded in parallel to form a spectral branch, a global branch, and a filtering branch to obtain spectral information, global information, and filtering information from the input features. The spectral information, global information, and filtering information are concatenated in the channel dimension, and after being compressed back to the stage channel number by 1×1 convolution, they are added to the input residual to obtain the stage characterization, thereby realizing spatial-spectral complementarity and numerical steady state. In step S3-3, high-frequency and low-frequency information are extracted while downsampling the input image features using wavelet transform. Specifically, to preserve the edge and texture information required for reconstruction during downsampling, the WPD module first performs two-dimensional Haar wavelet decomposition on the output of the SGF-FL module to obtain four components: LL, LH, HL, and HH. Then, the three high-frequency components of LH, HL, and HH are fused, and the pooling features of the input features are obtained through max pooling. The fused high-frequency components are then merged with the pooling features, and finally, after lightweight mapping, they are superimposed with the low-frequency LL components through residual summation to form a downsampling result that reduces resolution while explicitly carrying high-frequency contributions. In steps S3-4, steps S3-2 and S3-3 are repeated 2-3 times in multiple stages to continuously deepen the semantics and gradually compress the spatial scale, while learning semantic features and continuously downsampling the features. In steps S3-5, the semantic features of deep features are extracted using the SGF-FL module, ultimately obtaining a multi-scale tensor set {F1∈R}. (H / 2)×(W / 2)×64 F2∈R (H / 4)×(W / 4)×128 F3∈R (H / 8)×(W / 8)×256}, where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively.

[0010] The spectral branching described in step S3-2 uses a combination of "3×3 standard convolution + 5×5 depth-separable convolution + 1×1 channel blending" to characterize local textures and detailed contours; The global branch uses a windowed multi-head self-attention structure to establish long-distance dependencies within the local window and enhances the nonlinear representation through a feedforward network. The filtering branch combines the Sobel convolution, which is fixed in the prior image processing, with a learnable bilateral filtering module. It uses the "dilation-erosion" differentiable approximation to achieve a closing operation, generating boundary buffer weights to selectively amplify true boundaries and suppress pseudo-high-frequency noise. The specific operation process is as follows: the input feature image is processed by Sobel convolution and a learnable bilateral filtering module to extract edge features and denoising features respectively. Then, the Tanh activation function is used to normalize the values ​​of the edge feature map. The "dilation-erosion" closing operation is then used to generate a boundary buffer for the boundary features. The boundary buffer is then multiplied with the input image features to achieve selective edge amplification. Finally, it is added to the denoising features. This process highlights edge features while improving the consistency of the ground features and suppressing the influence of noise through denoising.

[0011] The Sobel convolution is used to explicitly introduce classical image gradient operators to characterize directional edge information in the feature map. Its specific implementation is as follows: for the input feature map F∈R... H×W×C In terms of spatial dimensions, Sobel kernels K are used horizontally. x Sobel kernel K in the vertical direction yPerform convolution operations to obtain the corresponding gradient response G. x With G y ,in ; The edge magnitude map G is further calculated by performing a square root operation on a pixel-by-pixel basis. ; The Sobel convolution operator described above is predefined with fixed parameters and does not participate in backpropagation updates. Its role is to explicitly extract locations in the feature space where grayscale or feature response changes drastically, thereby obtaining direction-sensitive edge feature maps. The edge responses generated in this way provide reliable initial boundary cues for subsequent morphological closing operations, helping to reduce overfitting of the network to noisy gradients and improving the stability and interpretability of real-world object boundaries.

[0012] The bilateral filtering module is used to perform adaptive weighted smoothing of input features based on feature similarity. Its specific implementation process is carried out in three steps: similarity calculation, weight normalization, and weighted feature aggregation.

[0013] First, take the center feature F corresponding to any pixel position in the input feature map. input ∈R H×W×C For reference, it is adjusted to F∈R by scaling. C×H×W×1 Then, neighboring region features N are selected within its eight-neighbor range. Feature similarity is measured by calculating the mean absolute difference between the central feature and the neighboring features, and an exponential decay function is introduced to construct a similarity score Aff, which is calculated as follows: ; Where Aff is the similarity score, and Mean(.) represents the averaging operation on the feature channel dimension. The exponential decay form can effectively suppress neighboring pixels that are significantly different from the central feature, so that feature-similar regions can obtain higher response weights.

[0014] Subsequently, the similarity score Aff is normalized within the neighborhood to obtain the bilateral filter weight Weight, which is calculated as follows: ; Sum(.) represents the summation operation of all similarity scores in the neighborhood. This normalization operation ensures that the sum of the weights of each neighborhood is 1, thereby avoiding unstable amplification of the feature amplitude as the neighborhood changes.

[0015] Finally, using the normalized bilateral filter weights, the neighborhood features N are weighted and summed to obtain the output features BF of the bilateral filter module. output The calculation process is as follows: ; Through the above calculation method, the bilateral filtering module can achieve effective smoothing in regions with similar features, while adaptively reducing the cross-region feature propagation intensity at boundary locations with large feature differences. This suppresses noise interference while maintaining the integrity of the ground feature boundary structure, providing a stable and consistent denoised feature representation for subsequent fusion with the Sobel convolution enhancement branch.

[0016] In step S4-1, the decoder performs shallow feature structure perception on shallow feature F1 and mid-level feature F2 to complete detail correction. This process first adjusts the spatial scale information of shallow feature F1 to be the same as that of mid-level feature F2 through the CBAM module and WPD module. Then, the edge feature extraction method of SGF-FL model in the encoding stage is reused to selectively amplify the real edges of the adjusted shallow feature F1 and mid-level feature F2 and suppress false edges and noise. Finally, the two sets of features are combined by combining the channel dimensions to obtain shallow and mid-level features for subsequent feature fusion. The CBAM module is the attention module. In step S4-2, for the mid-level feature F2 and the deep feature F3, the decoder uses overlapping patch embedding, followed by normalization and activation to unify with the 1×1 projection to the same channel dimension, thus obtaining the mid-to-deep features. These mid-to-deep features take into account both cross-stage semantic alignment and spatial detail compensation, so that the features at each scale after upsampling have fusionability on the common resolution of H / 4×W / 4, realizing cross-stage semantic alignment and context modeling. In step S4-3, shallow and mid-level features are concatenated with mid-to-deep features at a uniform scale along the channel dimension. Then, they are recalibrated using a structure-aware module that combines channel attention and spatial attention to obtain fused features. Channel attention uses global averaging / max pooling and two fully connected layers to capture "which type of semantics is more important", while spatial attention uses 7×7 convolution to highlight "where is more critical" on the plane. Finally, the fused features are regularized into the decoding channel dimension using "3×3 convolution shaping + 1×1 compression" to obtain a fused representation that is structurally consistent, textured, and semantically distinct.

[0017] In step S4-4, the fused representation is gradually restored to the original resolution through two levels of bilinear upsampling, and then lightweight convolution and normalization are used to avoid checkerboard artifacts; a single-layer 1×1 convolution is used in the classification head to directly map to the numclass dimension, and a pixel-level probability map X of the same size as the input is generated through Softmax. output ∈R H×W×numclass Where R represents a vector, H and W are the number of pixels in the vertical and horizontal directions of the image, respectively, and numclass refers to the number of categories output by the model.

[0018] The overlapping patch embedding described in step S4-2 is specifically a linear projection using a 3×3 convolution in stride-overlap mode.

[0019] In step S5, cross-entropy loss is used for supervision at the point annotation location; Specifically, the cross-entropy loss of the supervision signal is calculated only at the set of point annotations, thereby driving end-to-end training with an extremely low annotation ratio of 0.01% pixels. Since edge and frequency domain priors are introduced in the encoding stage, and spatial consistency is constrained by structure-aware fusion in the decoding stage, the model can effectively suppress overfitting and maintain boundary clarity under sparse supervision. Combined with the learning rate scheduling of cosine annealing, the entire training process exhibits good convergence stability and cross-scene generalization ability. Here, the edge and frequency domain priors refer to the edge features extracted by Sobel convolution in the SGF-FL module and the wavelet transformation downsampling in the WPD module, respectively.

[0020] Compared with the prior art, the embodiments of this application have the following beneficial effects: This invention enables fine semantic segmentation of remote sensing images relying solely on point-level annotations. It ensures the basic learnability of the model under weak supervision and overcomes the problems of high cost and difficulty in scaling traditional pixel-level annotations. This allows for segmentation accuracy comparable to fully supervised methods while significantly improving annotation efficiency, thereby promoting the development and progress of large-scale remote sensing mapping technology.

[0021] Some of the additional features of this application can be further described in the following description. By examining the following description and corresponding drawings, or by understanding the operation and training process of the embodiments, those skilled in the art can clearly recognize some of the additional technical effects proposed in this application. The features disclosed in this application can be implemented and obtained through the practice of various construction methods, training processes, and combinations of different modules, strategies, and structures in specific embodiments. Attached Figure Description

[0022] The accompanying drawings, which are provided to further illustrate this application and constitute a part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute a limitation thereof. In the drawings, the same reference numerals denote the same components. Figure 1 This is a schematic diagram of the overall framework of the method of the present invention.

[0023] Figure 2 This is a schematic diagram of the SGF-FL three-branch module structure of the present invention.

[0024] Figure 3This is a schematic diagram of the Sobel convolution and bilateral filtering modules in the SGF-FL module of the present invention.

[0025] Figure 4 This is a schematic diagram of the WPD module structure of the present invention.

[0026] Figure 5 This is a schematic diagram of the HSA-Decoder structure of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] Example 1 like Figure 1 As shown, a weakly supervised semantic segmentation method based on point annotations includes the following steps: S1, Model hyperparameter settings; S1-1: Basic Channel Construction and Phase Expansion; A multi-stage encoder was constructed starting with the shallowest layer feature channel number C=32. In order to gradually enhance the feature representation capability while controlling the amount of computation, the spatial resolution of the feature map was reduced by 1 / 2 and the number of channels was expanded by 2 times each time it entered the next stage, thus forming a hierarchical pyramid of "resolution decrease and channel increase". For example, with an input size of H×W, the Stage-1 output is (H / 2)×(W / 2)×64, the Stage-2 output is (H / 4)×(W / 4)×128, and if a deeper layer is enabled, the output can reach (H / 8)×(W / 8)×256. This design can preserve details in shallow layers and enhance semantics in deeper layers, enabling subsequent decoders to simultaneously acquire both boundary and contextual information.

[0030] S1-2: Employs a self-attention structure; To establish long-distance dependencies with low complexity, a windowed multi-head self-attention structure is adopted in the Transformer feature extraction layer of the multi-stage encoder; wherein, each Transformer feature extraction layer is fixed with 8 attention heads and the window side length is 8; S1-3: Set up training and inference; During the training phase, the Adam optimizer is used with an initial learning rate of 0.001 and a weight decay of 1e-4. A cosine annealing strategy is combined to gradually reduce the learning rate to achieve smooth convergence. Cross-entropy loss is used as the model optimization loss calculation method. It is worth noting that only the pixels at the point annotation positions are calculated for loss, and the loss function used by the model output and the point annotation labels is the cross-entropy loss function. Unannotated pixels are not calculated for loss, thus fitting the scenario of extremely low annotation cost.

[0031] The cross-entropy loss function is a commonly used loss function in deep learning and is an existing technology in this field. It calculates the loss between the model output and the sparse point annotations. Those skilled in the art can perform its calculation without creative effort, so it will not be elaborated here.

[0032] S2, Data Preparation and Normalization; S2-1: Remote sensing image cropping and data partitioning; To balance memory usage and scene integrity, the original remote sensing image was cropped into 256×256 patches, or, if resources allowed, into 512×512 patches. The patches were divided into three datasets—training set, validation set, and test set—in a 6:2:2 ratio, and the patches in each dataset were labeled using dot annotation. The point annotations are performed by randomly sampling within the valid regions corresponding to different categories, with the total labeled pixels accounting for 0.01%, and the number of categories (numclass) is set for each dataset. In order to reduce training bias, it is necessary to maintain the balance between different categories during the sampling process. S2-2: Normalization and Alignment; The input patches are normalized to zero mean and unit variance in the channel dimension to eliminate overall brightness and contrast drift caused by differences in imaging conditions. If necessary, global brightness / contrast normalization or histogram matching can be further used to alleviate domain differences across scenes and time phases. Finally, training sample images X0∈R are obtained for model training. H×W×3 Where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively. The aforementioned steps can provide a numerically stable starting point for subsequent multi-scale feature encoding.

[0033] S3. Multi-scale feature encoding based on SGF-Encoder; S3-1: Shallow embedding; Multi-channel semantic feature extraction is performed on the input training sample image features, and the spatial resolution is reduced. Specifically, shallow embedding uses two 3×3 convolutional layers combined with normalization to perform multi-channel semantic priming on the training sample images. The stride of the first convolution is set to 2, which not only aligns color and texture but also provides a smooth and noise-controlled base for the subsequent SGF-FL module. In this stage, the spatial resolution becomes half that of the input, F0∈R (H / 2)×(W / 2)×32 , where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively; S3-2: Three-branch joint feature learning using the SGF-FL module; like Figure 2 As shown, in each encoding stage, the three feature extraction branches of the SGF-FL module are expanded in parallel to form a spectral branch, a global branch, and a filtering branch to obtain spectral information, global information, and filtering information from the input features. The spectral information, global information, and filtering information are concatenated in the channel dimension, and after being compressed back to the stage channel number by 1×1 convolution, they are added to the input residual to obtain the stage characterization, thereby realizing spatial-spectral complementarity and numerical steady state. The spectral branches use a combination of "3×3 standard convolution + 5×5 depth-separable convolution + 1×1 channel blending" to depict local textures and detailed contours; The global branch uses a windowed multi-head self-attention structure to establish long-distance dependencies within the local window and enhances the nonlinear representation through a feedforward network. like Figure 3 As shown, the filtering branch combines the Sobel convolution, which is fixed in the prior image processing, with a learnable bilateral filtering module. It then uses a differentiable approximation of "dilation-erosion" to achieve a closing operation, generating boundary buffer weights to selectively amplify true boundaries and suppress pseudo-high-frequency noise. The specific operation is as follows: the input feature image is processed by Sobel convolution and a learnable bilateral filtering module to extract edge features and denoising features respectively. Then, the Tanh activation function is used to normalize the edge feature map values. Next, a boundary buffer for the boundary features is generated through the "dilation-erosion" closing operation. The boundary buffer is then multiplied with the input image features to achieve selective edge amplification. Finally, it is added to the denoising features. This process highlights edge features while improving the consistency of the ground features through denoising, thus suppressing the influence of noise.

[0034] The Sobel convolution is used to explicitly introduce classical image gradient operators to characterize directional edge information in the feature map. Its specific implementation is as follows: for the input feature map F∈R... H×W×C In terms of spatial dimensions, Sobel kernels K are used horizontally. x Sobel kernel K in the vertical direction y Perform convolution operations to obtain the corresponding gradient response G. x With G y,in ; The edge magnitude map G is further calculated by performing a square root operation on a pixel-by-pixel basis. ; The Sobel convolution operator described above is predefined with fixed parameters and does not participate in backpropagation updates. Its role is to explicitly extract locations in the feature space where grayscale or feature response changes drastically, thereby obtaining direction-sensitive edge feature maps. The edge responses generated in this way provide reliable initial boundary cues for subsequent morphological closing operations, helping to reduce overfitting of the network to noisy gradients and improving the stability and interpretability of real-world object boundaries.

[0035] The bilateral filtering module is used to perform adaptive weighted smoothing of input features based on feature similarity. Its specific implementation process is carried out in three steps: similarity calculation, weight normalization, and weighted feature aggregation.

[0036] First, take the center feature F corresponding to any pixel position in the input feature map. input ∈R H×W×C For reference, it is adjusted to F∈R by scaling. C×H×W×1 Then, neighboring region features N are selected within its eight-neighbor range. Feature similarity is measured by calculating the mean absolute difference between the central feature and the neighboring features, and an exponential decay function is introduced to construct a similarity score Aff, which is calculated as follows: ; Mean(.) represents the averaging operation on the feature channel dimension. The exponential decay form can effectively suppress neighboring pixels that differ greatly from the central feature, so that feature-similar regions can obtain higher response weights.

[0037] Subsequently, the similarity score Aff is normalized within the neighborhood to obtain the bilateral filter weight, which is calculated as follows: ; Sum(.) represents the summation operation of all similarity scores in the neighborhood. This normalization operation ensures that the sum of the weights of each neighborhood is 1, thereby avoiding unstable amplification of the feature amplitude as the neighborhood changes.

[0038] Finally, using the normalized bilateral filter weights, the neighborhood features N are weighted and summed to obtain the output features BF of the bilateral filter module. output The calculation process is as follows: ; Through the above calculation method, the bilateral filtering module can achieve effective smoothing in regions with similar features, while adaptively reducing the cross-region feature propagation intensity at boundary locations with large feature differences. This suppresses noise interference while maintaining the integrity of the ground feature boundary structure, providing a stable and consistent denoised feature representation for subsequent fusion with the Sobel convolution enhancement branch.

[0039] S3-3: Wavelet-preserving downsampling is performed using the WPD module; like Figure 4 As shown, high-frequency and low-frequency information are extracted while downsampling the input image features using wavelet transform. Specifically, to preserve the edge and texture information required for reconstruction during downsampling, the WPD module first performs two-dimensional Haar wavelet decomposition on the output of the SGF-FL module to obtain four components: LL, LH, HL, and HH. Then, the three high-frequency components of LH, HL, and HH are fused, and the pooling features of the input features are obtained through max pooling. The fused high-frequency components are then merged with the pooling features, and finally, after lightweight mapping, they are superimposed with the low-frequency LL components through residual summation to form a downsampling result that reduces resolution while explicitly carrying high-frequency contributions. This mechanism can significantly reduce the boundary blurring and detail loss caused by using only pooling or straddle convolution, providing a guarantee for fine reconstruction in the decoding stage.

[0040] S3-4: Multi-stage progressive coding; Steps S3-2 and S3-3 are repeated 2-3 times in multiple stages to continuously deepen the semantics and gradually compress the spatial scale, while learning semantic features and continuously downsampling the features. Taking an input of H×W×3 as an example, after Stage-1 and Stage-2, feature maps with spatial resolutions of (H / 2)×(W / 2)×64 and (H / 4)×(W / 4)×128 are obtained respectively. If the scene is complex or stronger abstraction is required, a deeper feature of (H / 8)×(W / 8)×256 can be obtained to enhance global consistency and inter-class separability.

[0041] S3-5: Deep semantic extraction; The semantic features of deep features are extracted using the SGF-FL module, ultimately resulting in a multi-scale tensor set {F1∈R}. (H / 2)×(W / 2)×64 F2∈R (H / 4)×(W / 4)×128 F3∈R (H / 8)×(W / 8)×256}, where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively.

[0042] S4, based on such Figure 5 The HSA-Decoder shown demonstrates multi-level feature fusion decoding. S4-1: Shallow edge enhancement and noise reduction; The decoder performs shallow feature structure perception on shallow features F1 and mid-level features F2 to complete detail correction. This process first adjusts the spatial scale information of shallow feature F1 to be the same as that of mid-level feature F2 through the CBAM and WPD modules. Then, it reuses the edge feature extraction method of the SGF-FL model in the encoding stage to selectively amplify the real edges of the adjusted shallow features F1 and mid-level features F2 and suppress false edges and noise. Finally, it combines the two sets of features by combining the channel dimensions to obtain shallow and mid-level features for subsequent feature fusion. The CBAM module is the attention module. S4-2: Overlapping patch embedding is used for mid-to-deep features to achieve cross-stage semantic alignment and context modeling; The overlapping patch embedding is specifically a linear projection using a 3×3 convolution in stride-overlap mode.

[0043] For the mid-level feature F2 and the deep feature F3, the decoder uses overlapping patch embedding, followed by normalization and activation to unify with the 1×1 projection to the same channel dimension to obtain the mid-to-deep features. These mid-to-deep features take into account both cross-stage semantic alignment and spatial detail compensation, so that the features at each scale after upsampling have fusionability on the common resolution of H / 4×W / 4, realizing cross-stage semantic alignment and context modeling. S4-3, multi-scale stitching and structural perception fusion; At a unified scale, shallow and mid-level features are concatenated with mid-to-deep features along the channel dimension. Then, they are recalibrated using a structure perception module that combines channel attention and spatial attention to obtain fused features. Channel attention uses global averaging / max pooling and two fully connected layers to capture "which type of semantics is more important", while spatial attention uses 7×7 convolution to highlight "where is more critical" on the plane. Finally, the fused features are regularized into the decoding channel dimension using "3×3 convolution shaping + 1×1 compression" to obtain a fused representation that is structurally consistent, textured, and semantically distinct.

[0044] S4-4: Upsampling reconstruction and classification output; The fused representation is gradually restored to the original resolution through two levels of bilinear upsampling, and then lightweight convolution and normalization are used to avoid checkerboard artifacts. A single-layer 1×1 convolution is used in the classification head to directly map to the numclass dimension, and a pixel-level probability map X of the same size as the input is generated through Softmax. output ∈R H×W×numclass Where R represents a vector, H and W are the number of pixels in the vertical and horizontal directions of the image, respectively, and numclass refers to the number of categories output by the model.

[0045] A lightweight classifier architecture helps to concentrate expressive power in the encoding and fusion stages, while reducing deployment latency.

[0046] S5. Setting the loss and training strategy.

[0047] Supervision is performed at the point annotation locations using cross-entropy loss; Specifically, the cross-entropy loss of the supervision signal is calculated only at the set of point annotations, thereby driving end-to-end training with an extremely low annotation ratio of 0.01% pixels. Since edge and frequency domain priors are introduced in the encoding stage, and spatial consistency is constrained by structure-aware fusion in the decoding stage, the model can effectively suppress overfitting and maintain boundary clarity under sparse supervision. Combined with the learning rate scheduling of cosine annealing, the entire training process exhibits good convergence stability and cross-scene generalization ability. Here, the edge and frequency domain priors refer to the edge features extracted by Sobel convolution in the SGF-FL module and the wavelet transformation downsampling in the WPD module, respectively.

[0048] It should be noted that all features disclosed in this specification, or all steps in all methods or processes disclosed, may be combined in any way, except for mutually exclusive features and / or steps.

[0049] Furthermore, the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents.

Claims

1. A semantic segmentation method with weak supervision of point annotation, characterized in that, Includes the following steps: S1, Model hyperparameter settings; S1-1: Basic Channel Construction and Phase Expansion; S1-2: Employs a self-attention structure; S1-3: Set up training and inference; S2, Data Preparation and Normalization; S2-1: Remote sensing image cropping and data partitioning; S2-2: Normalization and Alignment; S3. Multi-scale feature encoding based on SGF-Encoder; S3-1: Shallow embedding; S3-2: Three-branch joint feature learning using the SGF-FL module; S3-3: Wavelet-preserving downsampling is performed using the WPD module; S3-4: Multi-stage progressive coding; S3-5: Deep semantic extraction; S4. Multi-level feature fusion decoding based on HSA-Decoder; S4-1: Shallow edge enhancement and noise reduction; S4-2: Overlapping patch embedding is used for mid-to-deep features to achieve cross-stage semantic alignment and context modeling; S4-3, multi-scale stitching and structural perception fusion; S4-4: Upsampling reconstruction and classification output; S5. Setting the loss and training strategy.

2. The semantic segmentation method with weak supervision based on point annotation as described in claim 1, characterized in that, In step S1-1, a multi-stage encoder is constructed starting with the shallowest feature channel number C=32. Each time the next stage is entered, the spatial resolution of the feature map is reduced by 1 / 2 and the number of channels is expanded by 2 times. In steps S1-2, a windowed multi-head self-attention structure is adopted in the Transformer feature extraction layer of the multi-stage encoder; wherein, each Transformer feature extraction layer is fixed with 8 attention heads and the window side length is 8. In steps S1-3, the Adam optimizer is used during the training phase with an initial learning rate of 0.001 and a weight decay of 1e-4. The learning rate is gradually reduced by combining a cosine annealing strategy to achieve smooth convergence. Cross-entropy loss is used as the method for calculating the model optimization loss.

3. The semantic segmentation method with weak supervision based on point annotation as described in claim 2, characterized in that, In step S2-1, the original remote sensing image is cropped into 256×256 patches or 512×512 patches. The patches are divided into three datasets: training set, validation set and test set in a ratio of 6:2:

2. The patches in each dataset are labeled using dot annotation. The point annotations are performed by randomly sampling within the valid regions corresponding to different categories, with the total number of annotated pixels accounting for 0.01%, and the number of categories (numclass) is set for each dataset. In step S2-2, the input patch is normalized to zero mean and unit variance in the channel dimension; simultaneously, global brightness / contrast normalization or histogram matching is further applied; finally, the training sample image X0∈R is obtained for model training. H ×W×3 In this context, R represents a vector, and H and W represent the number of pixels in the vertical and horizontal directions of the image, which are the two dimensions of the three-dimensional vector.

4. The semantic segmentation method with weak supervision based on point annotation as described in claim 3, characterized in that, In step S3-1, multi-channel semantic features are extracted from the input training sample image features, and the spatial resolution is reduced. Specifically, the shallow embedding uses two 3×3 convolutional layers combined with normalization to perform multi-channel semantic priming on the training sample images, with the stride of the first convolution set to 2; in this stage, the spatial resolution becomes half of the input, and the output is F0∈R. (H / 2)×(W / 2)×32 , where R represents a vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively; In step S3-2, at each encoding stage, the three feature extraction branches of the SGF-FL module are expanded in parallel to form a spectral branch, a global branch, and a filtering branch to obtain spectral information, global information, and filtering information from the input features. The spectral information, global information, and filtering information are concatenated in the channel dimension, compressed back to the stage channel number by 1×1 convolution, and then added to the input residual to obtain the stage characterization; In step S3-3, high-frequency and low-frequency information are extracted while downsampling the input image features using wavelet transform. Specifically, the WPD module first performs two-dimensional Haar wavelet decomposition on the output of the SGF-FL module to obtain four components: LL, LH, HL, and HH. Then, the three high-frequency components of LH, HL, and HH are fused, and the pooling features of the input features are obtained through max pooling. Then, the fused high-frequency components are merged with the pooling features, and finally, after lightweight mapping, they are superimposed with the LL low-frequency components by residual summation to form the downsampling result. In steps S3-4, steps S3-2 and S3-3 are repeated 2-3 times in multiple stages to continuously deepen the semantics and gradually compress the spatial scale, while learning semantic features and continuously downsampling the features. In steps S3-5, the semantic features of deep features are extracted using the SGF-FL module, ultimately obtaining a multi-scale tensor set {F1∈R}. (H / 2)×(W / 2)×64 F2∈R (H / 4)×(W / 4)×128 F3∈R (H / 8)×(W / 8)×256 } where F1 is the shallow feature in semantic features, F2 is the medium feature in semantic features, F3 is the deep feature in semantic features, R indicates that the feature is a three-dimensional vector, and H and W are the number of pixels in the vertical and horizontal directions of the image, respectively.

5. A semantic segmentation method with weak supervision based on point annotations according to claim 4, characterized in that, The spectral branching described in step S3-2 uses a combination of 3×3 standard convolution + 5×5 depth-separable convolution + 1×1 channel blending to characterize local textures and detailed contours; The global branch uses a windowed multi-head self-attention structure to establish long-distance dependencies within the local window and enhances the nonlinear representation through a feedforward network. The filtering branch combines the Sobel convolution, which is fixed in the prior image processing, with a learnable bilateral filtering module, and uses dilation-erosion differentiability to achieve a closing operation to generate boundary buffer weights. The specific operation process is as follows: the input feature image is processed by Sobel convolution and a learnable bilateral filtering module to extract edge features and noise reduction features respectively. Then, the Tanh activation function is used to normalize the values ​​of the edge feature map. Then, the boundary buffer of the boundary features is generated by dilation-erosion closing operation. Then, the boundary buffer is multiplied with the input image features to achieve selective edge amplification. Finally, it is added with the noise reduction features.

6. The semantic segmentation method with weak supervision based on point annotation according to claim 5, characterized in that, In step S4-1, the decoder performs shallow feature structure perception on shallow feature F1 and mid-level feature F2 to complete detail correction. This process first adjusts the spatial scale information of shallow feature F1 to be the same as that of mid-level feature F2 through the CBAM module and WPD module. Then, the edge feature extraction method of SGF-FL model in the encoding stage is reused to selectively amplify the real edges of the adjusted shallow feature F1 and mid-level feature F2 and suppress false edges and noise. Finally, the two sets of features are combined by combining the channel dimensions to obtain shallow and mid-level features. The CBAM module is the attention module. In step S4-2, for the intermediate feature F2 and the deep feature F3, the decoder uses overlapping patch embedding, and then normalizes and activates them to unify them with the 1×1 projection to the same channel dimension to obtain the intermediate and deep features. In step S4-3, shallow and mid-level features are concatenated with mid-to-deep features at a uniform scale along the channel dimension. Then, they are recalibrated using a structure perception module that combines channel attention and spatial attention to obtain fused features. Finally, the fused features are regularized into the decoding channel dimension using 3×3 convolutional shaping and 1×1 compression to obtain the fused representation. In step S4-4, the fused representation is gradually restored to the original resolution through two levels of bilinear upsampling, and then lightweight convolution and normalization are used to avoid checkerboard artifacts; a single-layer 1×1 convolution is used in the classification head to directly map to the numclass dimension, and a pixel-level probability map X of the same size as the input is generated through Softmax. output ∈R H×W×numclass Where R represents a vector, H and W are the number of pixels in the vertical and horizontal directions of the image, respectively, and numclass refers to the number of categories output by the model.

7. A weakly supervised semantic segmentation method with point annotations according to claim 6, characterized in that, The overlapping patch embedding described in step S4-2 is specifically a linear projection using a 3×3 convolution in stride-overlap mode.

8. A semantic segmentation method with weak supervision based on point annotations according to claim 7, characterized in that, In step S5, cross-entropy loss is used for supervision at the point annotation location; Specifically, the cross-entropy loss of the supervision signal is calculated only at the set of point annotations, thereby driving end-to-end training with an annotation ratio of 0.01% of pixels.

Images