A point-supervised remote sensing segmentation method based on semantic-preserving hierarchical decoding and spatial representation re-reference

Abstract

The present application relates to the field of remote sensing segmentation, and particularly relates to a point supervision remote sensing segmentation method based on semantic preservation hierarchical decoding and spatial representation reference. The scheme comprises the following steps: obtaining original high-resolution remote sensing images and corresponding sparse artificial point labels; inputting the remote sensing images into a multi-scale feature coding network for multi-scale feature coding; constructing a semantic preservation hierarchical decoding module to obtain decoding features with the highest spatial resolution; inputting the final decoding features with the highest spatial resolution into a main segmentation module to obtain main segmentation prediction probabilities; constructing region expansion supervision labels according to the artificial point labels and the main segmentation prediction probabilities; constructing a spatial representation reference module to generate region expansion prediction probabilities according to the supervision labels; then constructing a joint loss function to train the model, and finally performing inference deployment. The present application is suitable for remote sensing semantic segmentation.

Description

Technical Field

[0001] This invention relates to the field of remote sensing segmentation, specifically to a point-supervised remote sensing segmentation method based on semantically preserved hierarchical decoding and spatial representation re-reference. Background Technology

[0002] Existing remote sensing semantic segmentation methods typically rely on large amounts of pixel-level labeled data for supervised training. However, high-resolution remote sensing images are large in area and have complex land cover types, making pixel-by-pixel manual annotation costly in terms of manpower and time. Therefore, point-supervised semantic segmentation methods that utilize only a small number of point annotations with category information for model training are gaining increasing attention.

[0003] Point annotations typically provide only a limited number of pixel locations and their corresponding categories, failing to directly describe the complete area, boundaries, and spatial structure of the target feature. Therefore, existing point-supervised remote sensing semantic segmentation methods primarily improve feature representation capabilities or obtain denser supervisory information under limited point supervision by employing multi-scale feature enhancement, hierarchical feature fusion, and point annotation expansion.

[0004] Currently, there are several methods and systems for point-supervised semantic segmentation of remote sensing images. The following are some typical technical solutions:

[0005] Patent publication number CN113496221A discloses a point-supervised semantic segmentation method and system for remote sensing images based on deep bilateral filtering. This method first uses a convolutional neural network to extract multi-level features from remote sensing images and then performs deep bilateral filtering on features at different levels. Subsequently, it uses a feature pyramid to progressively upsample and fuse multi-scale features. Finally, it uses point labels to perform weakly supervised training of the semantic segmentation network, achieving pixel-level semantic segmentation of remote sensing images.

[0006] Patent publication number CN121482402A discloses a semantic segmentation method with weak supervision based on point annotations. This method constructs an SGF-Encoder and an HSA-Decoder, utilizes multiple feature extraction branches and wavelet-preserving downsampling to obtain multi-scale features, and performs scale alignment, attention enhancement, and fusion reconstruction on features at different levels through the HSA-Decoder. Finally, it uses a point annotation-supervised network for training to obtain the semantic segmentation result.

[0007] However, the aforementioned existing technologies still have the following shortcomings:

[0008] Existing method 1 (CN113496221A) enhances remote sensing image features at different levels through deep bilateral filtering and utilizes feature pyramids to progressively upsample and fuse multi-scale features, thereby improving feature representation under point-supervised conditions. However, this method still primarily employs direct fusion of high-level and low-level features during decoding. When point annotations are sparse, local responses such as texture and edges in low-level features are difficult to obtain sufficient class constraints, and their direct participation in high-level semantic fusion may interfere with the already formed class semantics.

[0009] Existing method 2 (CN121482402A) acquires multi-scale features through SGF-Encoder and uses HSA-Decoder to perform scale alignment, concatenation, and channel and spatial attention enhancement on features at different levels, achieving multi-level feature fusion and spatial structure restoration. This method can make full use of feature information at different scales, but it still involves the fusion of high-level semantic features and low-level detail features together, without further distinguishing the different roles they play under point supervision. Therefore, while utilizing low-level spatial details, it may still cause unnecessary perturbation to the existing semantic state at higher levels.

[0010] In summary, existing point-supervised remote sensing image semantic segmentation methods mainly suffer from two problems: First, existing multi-scale decoding methods typically use methods such as addition, splicing, or attention enhancement to fuse high- and low-level features. Under sparse point supervision, insufficiently constrained local information in low-level features may affect the stable expression of high-level semantics. Second, existing point annotation expansion methods mainly focus on how to generate more regional supervision information from a small number of point labels, while not adequately considering the knowledge transfer relationships between point-level supervision, regional learning, and the final segmentation network. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a point-supervised remote sensing segmentation method based on semantically preserved hierarchical decoding and spatial representation re-reference. This method uses high-level semantic features as the main body during the decoding process. By extracting the difference information between low-level features and the high-level semantic state and selectively incrementally supplementing it, spatial details are gradually restored while preserving high-level category semantics. Simultaneously, real points are used as category anchors, and extended regions are constructed in conjunction with the main segmentation results. Furthermore, a region extension learning branch is used to learn region-level semantics and original image details. Finally, the region learning results are fed back to the main segmentation network, thereby achieving a gradual transformation from sparse point-level supervision to region-level segmentation knowledge.

[0012] This invention achieves the above objectives by adopting the following technical solution: This invention provides a point-supervised remote sensing segmentation method based on semantically preserved hierarchical decoding and spatial representation re-reference, comprising the following steps:

[0013] S1. Obtain the original high-resolution remote sensing image I and its corresponding sparse artificial point annotations. ;

[0014] S2. Input the remote sensing image I into a multi-scale feature coding network to extract N coding features at different scales. The features are set to l=1,2,…,N, and the coded features at each scale are transformed to a uniform number of decoding channels through a feature mapping layer to obtain the low-level mapped features at the current scale. .

[0015] Preferably, the multi-scale feature coding network is a ConvNeXt series network, which includes four progressive feature extraction stages, each outputting encoded features at four different scales, with the number of feature channels output by each stage increasing progressively.

[0016] More preferably, the multi-scale feature encoding network is a ConvNeXt-Tiny network, in which the number of ConvNeXt Blocks in its four progressive feature extraction stages are 3, 3, 9, and 3, respectively, corresponding to 96, 192, 384, and 768 output channels. Its first layer uses a 4×4 convolution with a stride of 4 for initial downsampling, and subsequent stages use 2×2 convolutions with a stride of 2 to progressively reduce the spatial resolution, extracting features for each of the four encoding stages. The ConvNeXt Blocks internally employ depthwise convolution, layer normalization, channel expansion, GELU activation, channel restoration, and residual connections for feature extraction.

[0017] S3. Construct a semantically preserving hierarchical decoding module to map features at the deepest level. Forming the initial decoding semantic state For the decoding process of the l-th layer, l=N-1,N-2,…,1, the following semantics are performed to maintain incremental updates:

[0018] S31, Decode the previous level's decoding state Upsampling to the current feature scale yields the high-level semantic subject. ;

[0019] S32. Calculate the low-level mapping features at the current scale. With the high-level semantic subject Difference information between And through the incremental mapping function Obtain differential increment ;

[0020] Preferably, the incremental mapping function It consists of 3×3 convolution, Group Normalization, and GELU activation function in sequence.

[0021] S33, the high-level semantic subject Mapping features with the current scale low layer Perform joint mapping to generate compatibility adjustment weights. Where [·] represents feature concatenation, and σ represents the Sigmoid function;

[0022] S34. Adjust the weights using the aforementioned compatibility settings. For the differential increment Adjustments are made and written into the high-level semantic body in residual form. Obtain the updated decoding features The updated decoded features are then residual-smoothed to obtain the final decoded features with the highest spatial resolution at the current scale. ,in Here, ⊙ represents the learnable scale parameter, and ⊙ denotes element-wise multiplication. This represents the residual adjustment function.

[0023] In the semantic-preserving hierarchical decoding process of this invention, the high-level state is located on the identity master path, and the low-level features themselves are not directly written into the high-level semantics. Instead, only the increments after differential extraction, compatibility adjustment, and scale control are allowed to participate in the current semantic state update. Therefore, the main technical function of the semantic-preserving hierarchical decoding is not simply to enhance shallow spatial details, but to use the already formed deep semantics as the semantic subject of the current scale under the condition of only sparse point category supervision, so that shallow information participates in the step-by-step decoding only in a controlled increment relative to the subject.

[0024] S4. Decode the features with the highest spatial resolution. Input the main segmentation module to obtain the main segmentation prediction probability. ;

[0025] Specifically, Input to the main segmentation module to obtain category prediction The output is then restored to the input image space size, and the principal segmentation probability is obtained through Softmax. .

[0026] S5. Based on the aforementioned manual point markings and the main segmentation prediction probability Constructing regional extended supervision labels ;

[0027] S51. Using the aforementioned manual point marking As a category seed, combined with the main segmentation prediction probability Construct a region-expanded candidate pixel set For class k, candidate pixel u satisfies and ,in The confidence threshold for category k;

[0028] S52. Starting from the real point location, iteratively expand the candidate pixels' connectivity under the preset spatial neighborhood relationship: Let the expanded region of category k obtained in the (t-1)th round be... Its neighborhood expansion result is Then the newly added region in round t is ,renew Continue until no new acceptable pixels are found or the preset number of propagations is reached;

[0029] S53. When different types of propagation regions arrive at the same pixel at the same time, remove that pixel from the region expansion supervision as a conflict location.

[0030] S6. Construct a spatial representation re-reference module, and expand the supervision label according to the region. Generate region expansion prediction probability ;

[0031] S61. Decoding features for the highest spatial resolution Gradient stopping is performed to obtain semantic features of gradient decoupling, which are then mapped to regional semantic subjects. ;

[0032] S62. Re-extract shallow native spatial representation from the original high-resolution remote sensing image I. ;

[0033] S63. Based on the semantic subject of the region Generate spatial representation adjustment weights ;

[0034] S64. Represent the original space Inject the semantic subject of the region in the form of controlled residuals The fused region learning features are obtained. ,in For learnable scale parameters;

[0035] S65. Learn the features of the region. Input the region classifier to obtain the region expansion prediction probability. .

[0036] S7. Construct a joint loss function to train the model. The joint loss function includes: based on the manually labeled points. With the main segmentation prediction probability Point monitoring loss Based on the region, the supervision label is extended. With the region expansion prediction probability Regional expansion loss and the main segmentation prediction probability With the region expansion prediction probability Consistency loss between ;

[0037] The joint loss function is specifically defined as follows:

[0038] , and These are the preset loss weighting coefficients;

[0039] Calculation at manually marked locations: ; The unnormalized output of the main segmentation module, Represents cross-entropy loss;

[0040] Calculated within the extended supervision area: ; This is the unnormalized output of the region classifier. This indicates an expanded area of ​​supervision.

[0041] Calculated over all pixel regions excluding real artificial points: , This indicates a region that is not manually accessed.

[0042] In the initial training phase, the main segmentation path is first supervised by real human points, using only the point-supervised loss. The model is optimized to enable the multi-scale feature encoding network and semantically preserving hierarchical decoding module to establish basic category discrimination capabilities. Once the main segmentation network reaches the preset training stage, the region expansion loss is simultaneously activated. and the aforementioned consistency loss Joint optimization was carried out.

[0043] S8. During the inference stage, only the multi-scale feature encoding network, the semantic preservation hierarchical decoding module, and the main segmentation module are retained. After inputting the remote sensing image to be segmented, pixel-level semantic segmentation results are obtained.

[0044] The inference phase retains only the following forward computation path: Remote sensing image I → Multi-scale feature encoding network → Semantic-preserving hierarchical decoding module → Main segmentation module → Pixel-level semantic segmentation result. The spatial representation re-reference module, region expansion supervision construction, and consistency constraints are only used during the training phase and do not participate in the computation during the inference phase.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] First, this invention does not directly fuse shallow coding features and high-level semantic features on an equal footing. Instead, it maintains the high-level decoded semantics as the current subject, allowing only the differential increment of shallow features relative to the high-level subject to participate in the update after compatibility adjustment. Specifically, this invention calculates the differential information between the current-scale low-level mapping features and the high-level semantic subject. The differential increment is obtained through an incremental mapping function, and then adjusted using compatibility-adjusted weights before being written into the high-level semantic subject as a residual. Finally, the residual is processed to obtain the final decoded features at the current scale. This mechanism helps to reduce the non-selective interference of local responses to weak class constraints on high-level semantics under sparse point supervision, and maintains the stable expression of high-level class semantics.

[0047] Second, this invention does not reuse the original image information for region label generation. Instead, it extracts native spatial representations from the original high-resolution image only after region extension supervision has been established, enabling region learning to utilize the local spatial information retained in the original pixel domain. Specifically, this invention constructs an independent shallow spatial coding path, re-establishing a three-level 3×3 convolutional shallow coding path from the original high-resolution remote sensing image. After obtaining native spatial features, it transforms them through feature mapping to a feature space consistent with the semantic subject of the region, enabling region learning to obtain native spatial information that has not undergone deep semantic abstraction.

[0048] Third, the original spatial representation is not unconditionally added to region prediction, but rather adjusted based on the already formed regional semantic state, participating in region learning as a controlled residual. Specifically, this invention is based on the regional semantic subject... Generate spatial representation adjustment weights Then, the original space is represented in the form of controlled residuals. Injecting regional semantic subjects helps reduce the direct impact of original image responses that are inconsistent with the current category semantics on regional prediction.

[0049] Fourth, the main segmentation semantics used in region expansion learning first undergoes stopping gradient processing to prevent pseudo-supervision of the region from directly acting on the multi-scale feature encoding network and the semantically preserving hierarchical decoding module along the semantic input path. Simultaneously, a knowledge interaction is established between the region learning results and the final main segmentation result through relatively weak prediction consistency. Specifically, this invention decodes features with the highest spatial resolution. Gradient stopping After mapping it to a region semantic subject, it is used for region expansion learning, and the region expansion loss is... The gradient cannot be backpropagated to the main segmentation network; it is only propagated through the L2 consistency loss. Establish soft constraints.

[0050] Fifth, the region extension learning path of this invention is only enabled during the training phase. The region extension supervision construction and spatial representation re-reference branch does not need to be run during the inference phase. Therefore, region-level learning can be used for the training of the main segmentation network without adding inference computation of the corresponding auxiliary branches during the deployment phase. Attached Figure Description

[0051] Figure 1 This is a flowchart of a point-supervised remote sensing segmentation method based on semantically preserved hierarchical decoding and spatial representation re-reference provided by an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the overall architecture of a PSPR-Net model provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0054] To facilitate understanding of this invention, some of the terms involved in this invention will be explained first, as follows:

[0055] Native spatial representation: Spatial features obtained by directly reconstructing shallow coding paths from original high-resolution remote sensing images after regional extended supervision has been established. It differs from the network internal features that have undergone multi-level semantic transformation during the ConvNeXt multi-scale coding process, and does not specifically refer to explicit edges, high-frequency or artificial texture features.

[0056] Semantic subject: In the process of semantic preservation hierarchical decoding, it is obtained by upsampling of deeper decoding features and represents the high-level features that have formed the current category semantic state.

[0057] Differential increment: The incremental feature formed after learnable mapping of the difference between the current scale low-level encoded features and the already formed high-level semantic subjects.

[0058] Regional extended supervision: Non-human point regional supervision information obtained based on real human points, principal segmentation prediction categories, corresponding prediction confidence, and spatial connectivity conditions.

[0059] The present invention will now be described in detail.

[0060] like Figure 1 As shown, this invention provides a point-supervised remote sensing segmentation method based on semantically preserved hierarchical decoding and spatial representation re-reference, comprising the following steps:

[0061] Step 1: Obtain the original high-resolution remote sensing image I and its corresponding sparse artificial point annotations. ;

[0062] Step 2, Multi-scale Feature Coding: Input the remote sensing image I into a multi-scale feature coding network to extract N coding features at different scales. The features are set to l=1,2,…,N, and the encoded features at each scale are transformed to a uniform number of decoding channels through a feature mapping layer to obtain the low-level mapped features at the current scale. ;

[0063] Step 3, Semantic Preservation Layered Decoding: Construct a semantic preservation layered decoding module to map features at the deepest layer. Forming the initial decoding semantic state For the l-th layer decoding process, l = N-1, N-2, ..., 1, perform semantically preserved incremental update operations to obtain the final decoded features with the highest spatial resolution. ;

[0064] Step 4, Master Segmentation Prediction: Decode the features with the highest spatial resolution. Input the main segmentation module to obtain the main segmentation prediction probability. ;

[0065] Step 5: Construct a region expansion supervision system based on manual point annotations. and principal segmentation prediction probability Constructing regional extended supervision labels ;

[0066] Step 6: Construct a spatial representation re-reference module, and expand the supervision label according to the region. Generate region expansion prediction probability ;

[0067] Step 7: Construct a joint loss function to train the model. The joint loss function includes: based on the manually labeled points. With the main segmentation prediction probability Point monitoring loss Based on the region, the supervision label is extended. With the region expansion prediction probability Regional expansion loss and the main segmentation prediction probability With the region expansion prediction probability Consistency loss between ;

[0068] Step 8: In the inference stage, only the multi-scale feature encoding network, the semantic preservation hierarchical decoding module, and the main segmentation module are retained. After inputting the remote sensing image to be segmented, pixel-level semantic segmentation results are obtained.

[0069] Example 1: Overall Architecture of the PSPR-Net Model

[0070] like Figure 2 As shown, this invention provides a point-supervised remote sensing segmentation model based on semantically preserved hierarchical decoding and spatial representation re-reference, called Point-Supervised Semantic-Preserving and Native SpatialRepresentation-Guided Region Learning Network (PSPR-Net). PSPR-Net consists of a main segmentation path and a region expansion learning path used in the training phase.

[0071] The main segmentation path includes a ConvNeXt-Tiny multi-scale feature encoding module, a semantically preserving hierarchical decoding module, and a main segmentation module, which are used to form pixel-level category predictions under sparse point supervision.

[0072] The region extension learning path includes region semantic mapping, native spatial representation encoding, semantically guided spatial representation injection, and region prediction. It is used to complete region-level learning by using the main segmentation semantics as the basis for region categories and reusing the native spatial representation of the original remote sensing image when region extension supervision has been formed.

[0073] During training, a region extension supervision is constructed based on real artificial points and the main segmentation results, and the learning results of the region extension are used to participate in the joint optimization of the main segmentation network through weak prediction consistency.

[0074] The overall computational relationships of PSPR-Net are represented as follows:

[0075] ;

[0076] Further construction during the training phase:

[0077] ;

[0078] The regional supervision required for regional extension learning is provided by:

[0079] generate.

[0080] Example 2: Specific Implementation of the Multi-Scale Feature Encoding Module

[0081] The remote sensing image I is input into ConvNeXt-Tiny for multi-level feature encoding.

[0082] like Figure 2As shown, the ConvNeXt-Tiny encoder consists of four progressive feature extraction stages, with the number of ConvNeXt Blocks in each stage being 3, 3, 9, and 3, respectively, corresponding to 96, 192, 384, and 768 output channels. Its first layer uses a 4×4 convolution with a stride of 4 for initial downsampling, and subsequent stages use 2×2 convolutions with a stride of 2 to progressively reduce the spatial resolution.

[0083] ConvNeXt Block1 to ConvNeXt Block4 employs depthwise convolution, layer normalization, channel expansion, GELU activation, channel restoration, and residual connections for feature extraction.

[0084] This embodiment does not use the original ConvNeXt-Tiny classification output, but extracts features from the four encoding stages separately:

[0085] ;

[0086] To enable unified decoding, the four scale features are transformed to the same number of decoding channels using a feature mapping layer:

[0087] ;

[0088] Example 3: Specific Implementation of the Semantic Preservation Layered Decoding Module

[0089] To reduce the unselective interference of low-level weakly constrained features on high-level category semantics under sparse point supervision, this invention constructs a semantically preserved hierarchical decoding module.

[0090] First, the deepest layer of mapping features... Forming the initial decoding semantic state:

[0091] ;

[0092] For the decoding process of the l-th layer, firstly, the decoding state of the previous layer is... Upsampled to the current feature scale:

[0093] ;

[0094] in, This indicates the high-level semantic subject that has been formed at the current scale.

[0095] Unlike directly Compared with current low-level features To sum them up, this application first calculates the difference between the two. Then through the incremental mapping function Obtain relative incremental information The incremental mapping described above consists of 3×3 convolution, Group Normalization, and GELU.

[0096] Furthermore, the current high-level semantic subject With low-level coding features Perform joint mapping to generate compatibility adjustment weights:

[0097] ;

[0098] in, Indicates feature splicing, express function.

[0099] Finally, the compatibility weights are used to adjust the low-level incremental information:

[0100] , ;

[0101] in, This represents the learnable scale parameter. This indicates element-wise multiplication.

[0102] The current code maintains the high-level state on the identity main path. Low-level features are not directly written into the high-level semantics; instead, only increments after differential extraction, compatibility adjustment, and scale control are allowed to participate in the current semantic state update. The updated features are further processed by residual shaping. The final decoded features at the current scale are obtained.

[0103] In this embodiment, the residual adjustment function It consists of 3×3 convolution, Group Normalization, and GELU activation function in sequence.

[0104] In the semantic-preserving hierarchical decoding process of this invention, the high-level state is located on the identity master path, and the low-level features themselves are not directly written into the high-level semantics. Instead, only the increments after differential extraction, compatibility adjustment, and scale control are allowed to participate in the current semantic state update. Therefore, the main technical function of the semantic-preserving hierarchical decoding is not simply to enhance shallow spatial details, but to use the already formed deep semantics as the semantic subject of the current scale under the condition of only sparse point category supervision, so that shallow information participates in the step-by-step decoding only in a controlled increment relative to the subject.

[0105] Example 4: Specific Implementation of the Main Segmentation Module

[0106] After three levels of semantically preserving layered decoding, the decoded semantic features with the highest spatial resolution are obtained. ,Will Inputting the main segmentation module yields the category prediction:

[0107] ;

[0108] The output is then restored to the input image space size, and the principal segmentation probability is obtained through Softmax:

[0109] .

[0110] Example 5: Specific Implementation of the Spatial Representation Rereference Module

[0111] After the model has formed a region extension supervision, this invention further utilizes the native spatial representation in the original high-resolution remote sensing image to assist region learning.

[0112] (I) Construction of Regional Semantic Subjects

[0113] Decode the main segmentation features The stopping gradient, i.e., dec["d1"].detach(), is used as the semantic input to the region expansion branch and then input into the region expansion module.

[0114] ;

[0115] in, For the category semantics of the region learning phase, sg(·) represents the stopping gradient.

[0116] (II) Extraction of native spatial representations

[0117] This invention reconstructs shallow spatial coding paths from original high-resolution remote sensing images:

[0118] ;

[0119] The native spatial encoding path includes three levels of 3×3 convolutions, ultimately yielding native spatial features at a scale of approximately H / 4. Further feature mapping is then performed.

[0120] ;

[0121] Transform it into a feature space consistent with the semantic subject of the region.

[0122] in The original spatial representation represents the shallow local spatial information extracted from the original high-resolution image after regional supervision has been formed, rather than the encoded features after deep semantic abstraction by ConvNeXt.

[0123] (III) Injection of native spatial representation guided by regional semantics

[0124] To avoid unconditionally interfering with the semantics of region categories due to native spatial information, based on the semantic subject of the region... Generate spatial representation adjustment weights:

[0125] ;

[0126] The original spatial representation is then written into the region semantic subject in the form of controlled residuals:

[0127] ;

[0128] in, This is a learnable scale parameter.

[0129] (iv) Regional forecasting

[0130] The fused region learning features Input region classifier = ( ), obtained through Softmax =Softmax( ).

[0131] Example 6: Specific Implementation of Region Expansion Supervision Construction

[0132] Using ground truth pixel annotations as class seeds, and combining them with the main segmentation probability, candidate regions for region expansion are constructed. For class k, candidate pixel u must satisfy at least:

[0133] ;

[0134] as well as

[0135] ;

[0136] in, This is the confidence threshold for category k.

[0137] Subsequently, starting from the real point location, the candidate pixels are expanded through connectivity under a pre-defined spatial neighborhood relationship. Let the expanded region for category k obtained in the (t-1)th round be... The result of its neighborhood expansion is expressed as:

[0138] ;

[0139] The newly added region in round t is:

[0140] ;

[0141] And updated:

[0142] ;

[0143] in, This represents the set of candidate pixels for category k.

[0144] Newly accepted pixels continue to be propagation sources in the next round until no new acceptable pixels exist or the preset number of propagations is reached. The current implementation supports two spatial connectivity modes: 4-neighborhood and 8-neighborhood.

[0145] When different categories of propagation regions simultaneously reach the same pixel, that pixel is treated as a conflict location and removed from the region expansion supervision. This reduces the impact of regions with uncertain class classification on training, ultimately yielding the expanded region label. .

[0146] Example 7: Specific Implementation of Joint Training

[0147] PSPR-Net employs a joint training approach consisting of point-level supervision, region-extended supervision, and weak prediction consistency.

[0148] In the initial training phase, the main segmentation path is first supervised by real human-marked points, and calculations are performed at manually labeled locations:

[0149] ;

[0150] This enables the ConvNeXt-Tiny and semantically preserved hierarchical decoding modules to establish basic category discrimination capabilities.

[0151] After the main segmentation network reaches the preset training stage, the labels are expanded according to the regions. and expanding the scope of non-point supervision Training region expansion prediction:

[0152] ;

[0153] To ensure that the region distribution information obtained from region expansion learning can be applied to the final main segmentation network, pixel regions other than artificial real points (i.e., Calculate the weak consistency loss between the main segmentation probability and the region expansion probability:

[0154] ;

[0155] The complete training objective is represented as:

[0156] ;

[0157] in, Used for real-point supervision Used for regional expansion supervision. Used to constrain the consistency between the main segmentation prediction and the regional prediction.

[0158] In this embodiment, and Experiments verified that the learning rates were set to 0.5 and 0.1 respectively. The AdamW optimizer was used during training, with an initial learning rate of 1e-4 and a cosine annealing learning rate scheduling strategy. The total number of training epochs was set to 200, with the first 50 epochs using only... Training will begin, and the system will be activated simultaneously starting from round 51. and .

[0159] Example 8: Specific Implementation of the Inference Phase

[0160] The model retains only the main segmentation path during the inference phase:

[0161] I→ConvNeXt-Tiny→SPHD→SegHead→Prediction;

[0162] The input remote sensing image undergoes multi-scale feature encoding, semantically preserved hierarchical decoding, and primary segmentation classification to obtain the final pixel-level semantic segmentation result. Region expansion supervised construction, native spatial representation-assisted learning, and consistency constraints are only used during the training phase and do not participate in the final inference.

[0163] During the inference phase, the high-resolution remote sensing image to be segmented is input into the trained PSPR-Net model. The ConvNeXt-Tiny multi-scale feature encoding module extracts multi-scale features, and the semantic-preserving hierarchical decoding module restores the spatial resolution and maintains high-level semantic stability step by step. Finally, the main segmentation module outputs pixel-level category predictions to obtain the final semantic segmentation result map.

[0164] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference, characterized in that, Includes the following steps: S1. Obtain the original high-resolution remote sensing image I and its corresponding sparse artificial point annotations. ; S2. Input the remote sensing image I into a multi-scale feature coding network to extract N coding features at different scales. The features are denoted as l=1,2,…,N, and the coded features at each scale are transformed to a uniform number of decoding channels through a feature mapping layer to obtain the low-level mapped features at the current scale. ; S3. Construct a semantically preserving hierarchical decoding module to map features at the deepest level. Forming the initial decoding semantic state For the l-th layer decoding process, l = N-1, N-2, ..., 1, perform semantically preserved incremental update operations to obtain the final decoded features with the highest spatial resolution. ; S4. Decode the final features at the highest spatial resolution. Input the main segmentation module to obtain the main segmentation prediction probability. ; S5. Based on the aforementioned manual point markings and the main segmentation prediction probability Constructing regional extended supervision labels ; S6. Construct a spatial representation re-reference module, and expand the supervision label according to the region. Generate region expansion prediction probability ; S7. Construct a joint loss function to train the model. The joint loss function includes: based on the manually labeled points. With the main segmentation prediction probability Point monitoring loss Based on the region, the supervision label is extended. With the region expansion prediction probability Regional expansion loss and the main segmentation prediction probability With the region expansion prediction probability Consistency loss between ; S8. During the inference stage, only the multi-scale feature encoding network, the semantic preservation hierarchical decoding module, and the main segmentation module are retained. After inputting the remote sensing image to be segmented, pixel-level semantic segmentation results are obtained.

2. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 1, characterized in that, In step S3, a semantically preserved incremental update operation is performed to obtain the final decoded features with the highest spatial resolution. Specifically, the following steps are included: S31, Decode the previous level's decoding state Upsampling to the current feature scale yields the high-level semantic subject. ; S32. Calculate the low-level mapping features at the current scale. With the high-level semantic subject Difference information between And through the incremental mapping function Obtain differential increment ; S33, the high-level semantic subject Mapping features with the current scale low layer Perform joint mapping to generate compatibility adjustment weights. Where [·] represents feature concatenation, and σ represents the Sigmoid function; S34. Adjust the weights using the aforementioned compatibility settings. For the differential increment Adjustments are made and written into the high-level semantic body in residual form. Obtain the updated decoding features The updated decoded features are then residual-smoothed to obtain the final decoded features at the current scale. ,in Here, ⊙ represents the learnable scale parameter, and ⊙ denotes element-wise multiplication. This represents the residual adjustment function.

3. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 1, characterized in that, The multi-scale feature coding network is a ConvNeXt series network, which contains four progressive feature extraction stages, each outputting encoded features at four different scales, with the number of feature channels output by each stage increasing progressively.

4. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 3, characterized in that, The ConvNeXt series network is the ConvNeXt-Tiny network. The number of ConvNeXt Blocks in its four progressive feature extraction stages are 3, 3, 9 and 3, respectively, with corresponding output channel numbers of 96, 192, 384 and 768. Its first layer uses a 4×4 convolution with a stride of 4 for initial downsampling, and subsequent stages use a 2×2 convolution with a stride of 2 to progressively reduce the spatial resolution.

5. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 2, characterized in that, The incremental mapping function It consists of 3×3 convolution, Group Normalization, and GELU activation function in sequence.

6. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 1, characterized in that, In step S5, the region expansion supervision label The construction methods include: S51. Using the aforementioned manual point marking As a category seed, combined with the main segmentation prediction probability Construct a region-expanded candidate pixel set For class k, candidate pixel u satisfies and ,in The confidence threshold for category k; S52. Starting from the real point location, iteratively expand the candidate pixels' connectivity under the preset spatial neighborhood relationship: Let the expanded region of category k obtained in the (t-1)th round be... Its neighborhood expansion result is Then the newly added region in round t is ,renew Continue until no new acceptable pixels are found or the preset number of propagations is reached; S53. When different types of propagation regions arrive at the same pixel at the same time, remove that pixel from the region expansion supervision as a conflict location.

7. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 1, characterized in that, In step S6, the supervision label is expanded according to the region. Generate region expansion prediction probability Specifically, it includes: S61, Final decoding features for the highest spatial resolution Gradient stopping is performed to obtain semantic features of gradient decoupling, which are then mapped to regional semantic subjects. ; S62. Re-extract shallow native spatial representation from the original high-resolution remote sensing image I. ; S63. Based on the semantic subject of the region Generate spatial representation adjustment weights ; S64, Represent the original space Inject the semantic subject of the region in the form of controlled residuals The fused region learning features are obtained. ,in For learnable scale parameters; S65. Learn the features of the region. Input the region classifier to obtain the region expansion prediction probability. .

8. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 7, characterized in that, In step S62, shallow native spatial representations are re-extracted from the original high-resolution remote sensing image I. The method is to construct independent shallow spatial coding paths. The shallow spatial coding path includes three levels of 3×3 convolutions to encode the original high-resolution remote sensing image I, resulting in native spatial features with a spatial resolution approximately one-quarter that of the original image. Then through the feature mapping layer Will Transform to the semantic subject of the region A consistent feature space is obtained .

9. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 1, characterized in that, The joint loss function mentioned in step S7 is specifically defined as follows: , and These are the preset loss weighting coefficients; Calculation at manually marked locations: ; The unnormalized output of the main segmentation module, Represents cross-entropy loss; Calculated within the extended supervision area: ; This is the unnormalized output of the region classifier. This indicates an expanded area of ​​supervision. Calculated over all pixel regions excluding real artificial points: , This indicates a region that is not manually accessed.

10. The point-supervised remote sensing segmentation method based on semantically preserving hierarchical decoding and spatial representation re-reference as described in claim 1, characterized in that, The training of the joint loss function in step S7 adopts a phased strategy: in the initial stage of training, only the point-supervised loss is used. The model is optimized to enable the multi-scale feature encoding network and semantically preserving hierarchical decoding module to establish basic category discrimination capabilities; once the main segmentation network reaches the preset training stage, the region expansion loss is simultaneously activated. and the aforementioned consistency loss Joint optimization was carried out.

Images