Remote sensing image semantic segmentation method based on geometric perception dynamic class center

Abstract

The invention relates to a remote sensing image semantic segmentation method based on a geometric perception dynamic class center, and belongs to the technical field of computer vision and remote sensing image processing. The objective of the invention is to solve the problem that a static category prototype is difficult to adapt to violent intra-class differences of remote sensing ground features and is lack of geometric structure perception to cause boundary fracture and fuzzy details of slender and irregular targets. According to the technical scheme, multi-scale features are extracted through an encoder, a global dynamic decomposition class center module is constructed to realize adaptive evolution of class representation, a curvature sensing local context aggregation module is introduced to explicitly utilize geometric prior to perform feature alignment and enhancement, and finally a segmentation result is output through feature fusion. According to the method, the adaptive capacity and boundary integrity of the model to a complex scene are remarkably improved, the calculation efficiency is balanced while high precision is guaranteed, and the method has excellent cross-dataset generalization performance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and remote sensing image processing technology, and relates to a semantic segmentation method for remote sensing images based on geometrically perceptual dynamic class centers. Background Technology

[0002] Semantic segmentation of high-resolution remote sensing images aims to assign a clear semantic label to each pixel in the image. In recent years, deep learning technology, with its powerful feature extraction capabilities, has gradually replaced traditional machine learning methods and become the mainstream technology in this field, significantly improving the accuracy and automation level of land cover classification.

[0003] To improve segmentation accuracy, researchers have proposed various network architectures in existing studies. Early work primarily relied on Fully Convolutional Networks (FCNs) and their variants, such as the DeepLab series, to expand the receptive field and capture contextual information through dilated convolutions or pyramid pooling. To further establish long-range dependencies, attention-based methods such as Dual Attention Networks (DANets) and Transformer architectures have been widely introduced. In recent years, research has gradually shifted towards higher-order category-level context modeling, such as Object Context Representation Networks (OCRNets) and Adaptive Context Fusion Networks (ACFNets). These methods attempt to compute global class centers, or class prototypes, by aggregating pixel features, and use this as prior knowledge to enhance feature representations, thereby improving semantic consistency to some extent.

[0004] However, applying the aforementioned existing technologies directly to high-resolution remote sensing imagery still presents significant limitations. First, existing class-level methods generally employ static class prototype representation strategies, assuming each land cover class has a fixed center in the feature space, or only performing simple mean updates during the inference phase. This static assumption cannot adapt to the dramatic intra-class spatiotemporal variance in remote sensing imagery. Land cover of the same class, such as farmland, often exhibits drastically different spectral and textural features under different regional and seasonal lighting conditions—a phenomenon known as heterospectral variation—leading to a significant semantic gap between the currently input local features and the fixed global prototype, thus causing misclassification. Second, existing methods primarily rely on texture features and lack explicit perception of geometric topology, such as curvature direction. When processing numerous elongated, continuous land cover features in remote sensing scenes, such as roads, rivers, and irregularly shaped buildings, full-map attention computation often struggles to achieve spatial alignment between features and target geometric contours, easily resulting in jagged or blurred segmentation boundaries. Furthermore, the complex background textures of high-resolution imagery introduce significant noise; dense full-map computation is not only inefficient but also easily interferes with the feature representation of the target.

[0005] To address the shortcomings of existing technologies, such as the inability of static category prototypes to adapt to the drastic intra-class differences of remotely sensed ground objects and the lack of geometric structure perception leading to blurred details of the broken boundaries of slender and irregular targets, the core technical problem that this invention aims to solve is to construct a category representation mechanism that can dynamically evolve with the input scene to overcome feature mismatch caused by different spectra of the same object. At the same time, it introduces explicit geometric priors and adaptive alignment mechanisms to achieve fine segmentation of ground object targets and maintain boundary integrity in complex backgrounds. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a semantic segmentation method for remote sensing images based on geometric perception and dynamic class center decomposition. Addressing the technical problems in existing remote sensing image semantic segmentation techniques, such as the difficulty of static class prototypes adapting to severe intra-class spatiotemporal variance, and the lack of explicit perception of geometric topology leading to broken boundaries and blurred details of slender and irregular targets, this invention aims to provide a semantic segmentation method for remote sensing images based on geometric perception and dynamic class center decomposition.

[0007] The main objective of this invention is:

[0008] 1. Solving the problem of intra-class variation adaptability: Breaking through the limitations of traditional static class centers, a dynamic decomposition mechanism is used to generate class representations that adapt to the input scene, effectively overcoming the "different spectrums of the same object" problem caused by seasonal, regional and illumination changes in remote sensing images, and improving the model's generalization ability in complex cross-domain scenes.

[0009] 2. Solving the problem of complex geometric boundary modeling: Overcoming the shortcomings of existing methods in lacking geometric structure perception, by explicitly introducing curvature a priori and affine transformation window mechanism, the spatial alignment of feature extraction and target geometric contour is achieved, which significantly improves the segmentation integrity and continuity of slender targets such as roads and rivers and building edges.

[0010] 3. Achieving a balance between accuracy and efficiency: While ensuring high-precision semantic injection, it avoids the high overhead and background noise interference caused by dense attention calculations across the entire image, providing a solution for intelligent interpretation of high-resolution remote sensing big data that combines computational efficiency and segmentation accuracy.

[0011] To achieve the above objectives, the present invention provides the following technical solution: This invention proposes a geometry-aware dynamic class center network (GeoCenter Net) for semantic segmentation of remote sensing images. This method transforms the traditional static classification paradigm into a collaborative process of dynamic adaptation and explicit geometric perception by constructing a deep neural network based on an encoder-decoder structure. The specific technical steps are as follows: Step 1: Multi-scale feature extraction and channel alignment First, construct a neural network model based on deep learning. For example... Figure 1 As shown, this model uses a lightweight backbone network (preferably RepVit-M2.3) as the encoder. The input high-resolution remote sensing image (H×W) is fed into the encoder. The network undergoes four stages of progressive feature extraction and downsampling, generating four feature maps at different resolutions, denoted as R1, R2, R3, and R4, with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. Subsequently, convolution operations are performed on these four feature maps to uniformly adjust their channel dimensions to C (e.g., 256 dimensions), resulting in an aligned feature set Xs. This step aims to obtain multi-scale feature representations containing rich semantic information and spatial details, laying the foundation for subsequent dynamic class center generation and geometric perception.

[0012] Step 2: Construct the Global Dynamic Decomposition Class Center (GDCD) To address the issue of significant differences in the characteristics of the same type of land cover in remote sensing imagery under different seasons, regions, and lighting conditions, this invention designs a GDCD module, such as... Figure 2 As shown, this is used to generate the global class center Cg that dynamically evolves with the input image. This process specifically includes three sub-steps: feature decomposition, feature generation, and dynamic fusion.

[0013] The first step is cross-sample maintenance of common features. The system defines a set of learnable prototype vectors. (Dimensions are the number of categories K × the number of channels C), used to represent the stable semantic basis for each category. During the training phase, the prototype is updated using the exponential moving average (EMA) mechanism. To avoid interference from small sample noise, the system sets a threshold, triggering an update only when the number of valid pixels for a certain category in the current batch exceeds this threshold (e.g., 10). The formula for calculating the update weights is as follows:

[0014] in, This indicates the number of pixels of this type in the current batch. This represents the cumulative number of pixels throughout history. The temperature coefficient is used to balance the old and new information, and ε is a numerical stability term.

[0015] Secondly, there is the adaptive generation of feature inputs. For the current input image features, a lightweight sub-network employing a "compression-expansion-reconstruction" process is designed to capture the semantic increments specific to the current scene. This sub-network first compresses the feature channels through global average pooling and 1×1 convolutions to filter out redundancy, then maps them to a K×8 latent low-dimensional subspace, and finally reconstructs them back to the original dimension C, generating a feature increment vector specific to the current image. .

[0016] Finally, there is dynamic adaptive fusion. The network predicts an adaptive mixing coefficient α (ranging from 0 to 0.3) through fully connected layers to control the adjustment magnitude of characteristic features on the common basis. Dynamic class centers at each scale are then used. The formula is as follows:

[0017] Subsequently, the normalized weights of the four scale features for each category are calculated, and the dynamic centers of the four scales are weighted and aggregated to obtain the final shared global dynamic class center Cg.

[0018] Step 3: Curvature Awareness and Local Context Aggregation (CLCA) To address the boundary breakage and blurring issues of slender, curved targets such as roads and rivers, as well as irregular buildings, this invention designs a CLCA module as the core component of the decoder. For example... Figure 3 As shown, this module explicitly introduces geometric priors, and the specific operation is as follows: First, explicit computation of the curvature a priori is performed. The first derivatives in the horizontal and vertical directions are directly calculated on the feature map using the discrete difference operator. I x , I y and second derivative ( I xx , I yyBased on these derivatives, the curvature feature κ of each pixel is calculated using the following formula:

[0019] The calculated curvature map will be concatenated with the semantic features along the channel dimension to form a geometrically sensitive feature input.

[0020] Next, affine transformation and deformation window alignment are performed. The feature map is divided into non-overlapping local windows (e.g., 4×4 size). For each window, lightweight network branches predict affine transformation parameters, including scaling factors, rotation angles, and translations. A sampling grid is constructed based on the predicted parameters, and the feature map is bilinearly resampled. This step "straightens" or "corrects" the curved or tilted physical geometry in the image into a regular shape in feature space, thereby achieving spatial alignment between features and target contours.

[0021] Finally, a local-global collaborative attention mechanism is executed. Within the geometrically aligned window, the local class center Cl is calculated as the "key". Pixel features are used as the "query", and the global dynamic class center Cg generated in step two is used as the "value", constructing a cross-attention mechanism. The attention weight is obtained by calculating the similarity between the query and the key, and this weight is applied to the value, thereby injecting globally consistent category semantic information into the local pixels. After processing, the enhanced features are restored to their original spatial positions through an inverse affine transformation, completing the repair of geometric details.

[0022] Step 4: Feature Fusion and Result Output A top-down pyramid structure is adopted, progressively fusing high-level and low-level features processed by the CLCA module. The final fused feature map is then classified and mapped through a 1×1 convolutional layer, and upsampled to restore it to its original size, outputting the final semantic segmentation mask result. During training, the cross-entropy loss function is used to calculate the error between the predicted result and the true label, and the network parameters are optimized using the backpropagation algorithm.

[0023] The beneficial effects of this invention are as follows: First, this invention significantly improves the model's adaptability to complex scenes in remote sensing imagery, effectively solving the classification challenge caused by different spectra of the same object. Traditional methods use static category prototypes, which struggle to handle the significant intra-class variance generated by seasonal and regional lighting differences in remote sensing imagery. The Global Dynamic Decomposition Class Center (GDCD) module proposed in this invention creatively decomposes category representation into two dimensions: cross-sample commonality and input-related characteristics. This design allows the model to maintain stable recognition of the basic semantics of land cover categories while dynamically fine-tuning based on the specific features of the current input image. Experiments demonstrate that this mechanism can automatically adjust feature matching strategies when facing land cover such as farmland and water bodies under different temporal conditions, significantly reducing misclassification caused by environmental changes and greatly improving the model's generalization performance.

[0024] Secondly, this invention significantly enhances the ability to model the boundaries of slender, curved, and irregularly shaped targets, resolving the boundary breakage and jagged effects commonly found in traditional methods. By introducing Curvature-aware Local Context Attention (CLCA) modules, this invention, for the first time, explicitly utilizes the second-order differential geometric properties of images, such as curvature, in a semantic segmentation network. Combined with a deformable window mechanism based on affine transformation, the network can proactively perceive the curvature of roads and rivers, as well as the deflection angles of buildings, and spatially align the feature extraction window with the physical contours of the targets. This strategy of geometric alignment followed by semantic injection enables the segmentation results to maintain the continuity of lines and the integrity of contours when processing complex road networks and dense building clusters, significantly improving the quality of refined mapping.

[0025] Furthermore, this invention achieves an effective balance between computational efficiency and memory usage while ensuring high-precision segmentation. Traditional global context modeling methods rely on pixel-level dense attention calculations across the entire image, whose computational complexity increases quadratically with image resolution, making them difficult to apply to large-format remote sensing imagery. This invention employs a class-center-based attention mechanism, reducing the computational dimension from a massive number of pixels to the number of classes. Simultaneously, it combines this with the lightweight backbone network RepVit, achieving effective capture of global contextual information while significantly reducing computational load and memory consumption. This makes the method applicable not only to high-performance servers but also possible for future deployment on satellite processing or edge devices.

[0026] Finally, this invention demonstrates excellent cross-dataset generalization ability and practical application value. On several mainstream datasets with varying resolution and land cover distribution characteristics, such as LoveDA, ISPRSVaihingen, and Potsdam, this method achieves segmentation accuracy superior to existing mainstream technologies like SegFormer and DeepLabv3+, particularly in small target detection and complex boundary handling. This means that this method can be widely applied to various practical business scenarios such as land use surveys, urban planning monitoring, and precision agriculture assessment, providing a robust, efficient, and high-precision technical solution for the automated interpretation of remote sensing big data.

[0027] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0028] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 Geo Center Net network architecture; Figure 2 The global dynamic decomposition class center GDCD module; Figure 3 The CLCA module is used for curvature-aware local contextual attention. Detailed Implementation

[0029] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0030] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0031] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0032] Figures 1-3 This paper introduces the Geo Center Net network architecture and core module details. Figure 1 The overall architecture is as follows: RepVit-M2.3 is used as the encoder to extract multi-scale features, which are then used by the GDCD module to generate global dynamic class centers. Finally, the features are fused by the CLCA pyramid decoder to output the semantic segmentation result. An auxiliary head is set during the training phase. Figure 2 For the GDCD module; decompose features into common and characteristic components, weighted fusion to generate a global dynamic class center, adapting to intra-class differences. Figure 3 For the CLCA module; introduce curve priors, generate deformable windows and associate pixels with the global class center, and optimize boundary modeling.

[0033] To verify the effectiveness and advancement of the proposed Geo Center Net in remote sensing image semantic segmentation, this embodiment conducts systematic experiments and evaluations on three widely used public benchmark datasets. The hardware environment is based on an NVIDIA RTX 4090D graphics processor with 24GB of video memory, the software environment uses the PyTorch 2.1.0 deep learning framework, and the operating system is Ubuntu 22.04.

[0034] I. Experimental Dataset and Data Processing This embodiment selects three representative datasets: LoveDA, ISPRS Vaihingen, and ISPRS Potsdam, which cover different land cover distribution characteristics ranging from complex scenes across urban and rural areas to high-resolution urban centers. To adapt to memory limitations and enhance generalization ability, the images were preprocessed before the experiment: the LoveDA dataset was cropped while maintaining its original resolution, while the ISPRS dataset had its large-format images cropped into 512×512 pixel non-overlapping image patches. During training, online data augmentation strategies were adopted, including random horizontal and vertical flipping, random rotation (90°, 180°, 270°), random Gaussian blur, and random multi-scale scaling with a scaling factor between 0.5 and 1.5, to simulate the varying angle and illumination conditions in remote sensing imaging.

[0035] II. Training Implementation Process The experiment employed an end-to-end supervised training model. The AdamW algorithm was selected as the optimizer, with an initial learning rate set to 1×10⁻⁶. -4 The weight decay coefficient is 1×10 -4 The learning rate adjustment strategy employs polynomial decay with an exponent of 0.9 to ensure the stability of parameter updates in the later stages of training. The batch size is uniformly set to 4. For the LoveDA dataset, the model is trained for 50 epochs; for the ISPRS dataset, given its high-resolution features, sufficient iterative training is also performed until convergence. The loss function uses weighted cross-entropy loss, and an auxiliary head supervises intermediate layer features to accelerate gradient backpropagation. During the inference phase, to further improve prediction accuracy, multi-scale testing and a sliding window prediction strategy are employed, ultimately outputting pixel-level semantic segmentation results.

[0036] III. Experimental Results and Data Analysis On the LoveDA dataset, as shown in Table 1, the proposed method achieves an average Intersection over Union (mIoU) of 54.42% and an overall accuracy (OA) of 72.76%. Compared to the suboptimal method SegFormer, mIoU is improved by 0.51%, and OA is improved by 2.28%. Specifically, in the most challenging category "farmland," the IoU is improved by 6.49% thanks to the adaptability of the dynamic class center module to intra-class differences; in the "road" category, the IoU reaches 56.30% thanks to the curvature sensing module's ability to repair geometric shapes, significantly reducing road breakage.

[0037] Table 1. Comparison of semantic segmentation performance on the LoveDA dataset (unit: %)

[0038] On the ISPRS Vaihingen dataset, as shown in Table 2, our method achieved an mIoU of 84.01% and an F1 score of 91.20%. Particularly noteworthy is the high IoU of 92.13% in the "Buildings" category, demonstrating the advantage of affine alignment windows in handling regular geometric contours. In the smaller object category of "Vehicles," the IoU reached 83.08%, a 1.87% improvement over the comparison methods, showcasing a strong ability to capture fine-grained features.

[0039] Table 2. Comparison of ISPRS Vaihingen datasets (unit: %)

[0040] On the ISPRS Potsdam dataset, as shown in Table 3, our method achieved an mIoU of 88.04% and an overall accuracy of 92.11%, comprehensively outperforming classic methods such as DeepLabv3+ and OCRNet. In the easily confused "low vegetation" category, the IoU was improved by 0.81%, effectively distinguishing grassland from shrubs. Even in complex urban road scenes, the model maintained extremely high boundary integrity. These experimental data fully demonstrate that our invention, by combining geometric priors with dynamic feature representation, can significantly improve the accuracy and robustness of semantic segmentation of high-resolution remote sensing images.

[0041] Table 3. Comparison of semantic segmentation performance on the ISPRS Potsdam dataset (unit: %)

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A semantic segmentation method for remote sensing images based on geometrically perceptual dynamic class centers, characterized in that: Includes the following steps: The input high-resolution remote sensing image is fed into the encoder for multi-scale feature extraction and downsampling, generating feature maps at multiple scales and aligning the channel dimensions. Construct a Global Dynamic Decomposition Class Center (GDCD) to generate global class centers that adaptively evolve with the input image; Curvature-aware local contextual attention (CLCA) is used to explicitly introduce geometric priors and enhance feature representations; It also employs a pyramid structure for feature fusion and upsampling to output a semantic segmentation mask.

2. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The encoder is a RepVit-M2.3 network.

3. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The construction of GDCD involves maintaining a learnable prototype vector and updating the prototype vector using an exponential moving average (EMA) mechanism.

4. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 3, characterized in that: The EMA update is based on the category in the current batch. k number of pixels and historical cumulative pixel count Update weights Calculated as ,in ε is the temperature coefficient, and ε is the numerical stability term.

5. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The construction of GDCD includes dynamically fusing common features and characteristic features, and dynamically class centers. Calculated as ,in It is a common prototype vector. It is a feature increment vector. It is an adaptive mixing coefficient.

6. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The CLCA includes calculating curvature features. , ,in I x and I y These are the first derivatives in the horizontal and vertical directions. I xx and I yy It is the second derivative, and ε is a numerically stable term that is greater than zero.

7. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The CLCA includes dividing the feature map into local windows, predicting affine transformation parameters for each window, and performing bilinear resampling to achieve geometric alignment.

8. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The feature fusion adopts a top-down pyramid structure to fuse high-level and low-level features.

9. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The output semantic segmentation mask includes classification mapping through a 1×1 convolutional layer and upsampling to the original image size.

10. The remote sensing image semantic segmentation method based on geometrically perceptual dynamic class centers according to claim 1, characterized in that: The network parameters are optimized through backpropagation using the cross-entropy loss function.

Images