A remote sensing semantic segmentation system and method

Abstract

The application belongs to the field of remote sensing images, and discloses a remote sensing semantic segmentation system and method, wherein an encoder is used to extract multi-scale features from an input remote sensing semantic image step by step; a quadtree adaptive weighted attention unit is used to decode according to the multi-scale features, generate a region selection mask, and under the constraint of the region selection mask, perform calculation through region attention to highlight complex regions and inhibit simple regions; a global context fusion unit is used to serially fuse features at different decoding stages to obtain fused features with enhanced context consistency; and a segmentation head is used to perform pixel-level classification prediction on the fused features to output a semantic segmentation result consistent in size with the input remote sensing semantic image. The application guarantees high-precision segmentation effect, takes into account inference speed and engineering deployment adaptability, and comprehensively improves the overall precision, operation stability and actual engineering application value of the remote sensing semantic segmentation task.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing imagery, and in particular to a remote sensing semantic segmentation system and method. Background Technology

[0002] Remote sensing semantic segmentation, as a core foundational technology for intelligent interpretation of remote sensing images, has been widely applied in fields such as land surveys, urban planning, disaster monitoring, and environmental assessment. With the continuous improvement of high-resolution remote sensing image acquisition capabilities, images exhibit significant differences in target scale, complex spatial structures, and highly uneven category distributions, placing higher demands on the accuracy and stability of semantic segmentation models. Existing deep learning-based remote sensing semantic segmentation methods mostly employ an encoder-decoder structure, restoring spatial resolution through multi-layer feature extraction and progressive upsampling to achieve pixel-level semantic prediction. However, in complex remote sensing scenarios, relying solely on traditional multi-scale feature fusion methods often fails to simultaneously address both global semantic understanding and local detail representation, becoming a key bottleneck restricting model performance improvement.

[0003] To address the aforementioned challenges, existing solutions primarily focus on multi-scale feature fusion and contextual modeling. These solutions utilize feature pyramid structures, cross-layer skip connections, or the introduction of global context modules to enhance information exchange between features at different scales. These methods typically fuse multi-scale features in parallel or sequentially during the encoding or decoding stages to improve the model's ability to recognize targets at different scales. However, most of these solutions employ static structural designs, lacking a unified global semantic control mechanism during feature fusion. Information transfer between different decoding stages is often one-off, failing to create a continuous cross-layer contextual memory and semantic accumulation process. Therefore, when facing remote sensing scenes with complex structures or uneven semantic distribution, problems such as semantic inconsistency, blurred boundaries, or unstable local predictions still easily arise, making it difficult to fundamentally solve the technical bottleneck of insufficient semantic continuity modeling during cross-scale decoding.

[0004] Specifically, existing technologies have at least the following shortcomings: First, regarding spatial computation strategies, existing semantic segmentation methods typically employ fixed resolution or uniform computation strategies, applying the same computational intensity to the entire remote sensing image. However, different regions within a remote sensing image exhibit significant differences in spatial structural complexity and semantic importance; for example, large background areas (farmland, water bodies, etc.) coexist with small-scale but high-value target areas (vehicles, building edges, dense infrastructure). This approach, lacking spatial adaptive modeling capabilities, results in a significant waste of computational resources on semantically simple regions, while failing to adequately characterize structurally complex regions, thus limiting the model's segmentation accuracy and computational efficiency in complex remote sensing scenarios.

[0005] Second, regarding the balance between model expressiveness and deployment efficiency, existing deep learning semantic segmentation models typically introduce multi-branch convolutional structures or complex operators during the training phase to enhance the network's representational capabilities. However, these structures significantly increase the number of parameters and computational complexity during the inference phase, making it difficult to meet the efficient deployment requirements of resource-constrained devices or large-scale remote sensing data processing scenarios. Simultaneously, existing technologies lack adaptive modeling mechanisms for the importance of branches or channels, making it difficult to effectively simplify the highly expressive structures formed during training during the inference phase. This results in a structural contradiction between high expressiveness during training and high efficiency during inference.

[0006] Third, in terms of cross-scale feature fusion during the decoding stage, existing methods often employ static skip connections, simple concatenation, or independent layer-by-layer processing. The lack of unified global semantic constraints and dynamic modulation mechanisms between features of different scales easily leads to inconsistencies in semantic information, blurred category boundaries, or unstable local predictions during the fusion process. Furthermore, existing decoding structures typically do not explicitly retain or pass cross-layer contextual states during layer-by-layer fusion, lacking a continuous semantic memory and information accumulation mechanism between decoding stages. This makes it difficult for the model to continuously utilize the global semantic cognition formed in previous layers during the decoding process, thus limiting the model's ability to model overall structural relationships and long-range dependencies in complex remote sensing scenes.

[0007] In summary, existing remote sensing semantic segmentation technologies still have significant shortcomings in terms of spatial adaptive computation, the unification of training and inference efficiency, and cross-scale semantic continuity modeling in the decoding stage. There is an urgent need for an improved scheme that can balance computational efficiency, deployment feasibility, and segmentation accuracy to meet the practical needs of high-precision and high-stability semantic segmentation in complex remote sensing scenarios. Summary of the Invention

[0008] The purpose of this invention is to provide a remote sensing semantic segmentation system and method that solves the problem that existing technologies cannot simultaneously balance computational efficiency, deployment feasibility, and segmentation accuracy.

[0009] To achieve the above objectives, the present invention employs the following technical solution: A remote sensing semantic segmentation system includes an encoder, a quadtree adaptive weighted attention unit, a global context fusion unit, and a segmentation head; The encoder is used to extract multi-scale features from the input remote sensing semantic image step by step, forming feature representations with different spatial resolutions and semantic levels; The quadtree adaptive weighted attention unit is used to receive multi-scale features from the encoder, decode them according to the multi-scale features, generate a region selection mask, and under the constraint of the region selection mask, perform region attention calculation to enhance complex regions and suppress simple regions. The global context fusion unit is used to serially fuse features from different decoding stages and continuously introduce global semantic information from the previous stage into the feature processing of the next stage to obtain fused features with enhanced context consistency. The segmentation head is used to perform pixel-level classification prediction on the fused features and outputs a semantic segmentation result with the same size as the input remote sensing semantic image.

[0010] Furthermore, the encoder is equipped with a self-growing reparameterizable base module, which is divided into a heavy structure reparameterizable base module in the training state and a light structure reparameterizable base module in the deployment state. The heavy-weight, reparameterizable basic module in training mode is used to enhance expressive power during the training phase by employing multi-branch convolutional paths. The lightweight, reparameterizable base module in deployment mode is used to replace branch computations with an equivalent convolution during the inference / deployment phase, reducing the number of operators and inference latency.

[0011] Furthermore, the multi-branch convolution path includes at least three types of convolution transformation branches, each of which is used to output the module output obtained by weighted aggregation of normalized modules according to channel-level gating coefficients. The parameters of the equivalent convolution are obtained by folding and fusing the convolution of each branch, the normalization parameters, and the gating coefficients.

[0012] Furthermore, the quadtree adaptive weighted attention unit includes a quadtree region adaptive module and a quadtree weighted attention module; The quadtree region adaptive module is used to receive multi-scale features from the encoder and generate a region selection mask based on the multi-scale features. The quadtree weighted attention module is used to perform region attention calculations under mask constraints, to perform detailed modeling of complex regions and to suppress redundancy in simple regions.

[0013] A remote sensing semantic segmentation method for the system includes: Multi-scale features are extracted step by step from the input remote sensing semantic image to form feature representations with different spatial resolutions and semantic levels; Decoding is performed based on multi-scale features to generate a region selection mask. Under the constraint of the region selection mask, region attention is used to enhance complex regions and suppress simple regions. Features from different decoding stages are sequentially fused, and global semantic information from the previous stage is continuously introduced into the feature processing of the next stage to obtain fused features with enhanced context consistency. The fused features are classified and predicted at the pixel level, and the output is a semantic segmentation result with the same size as the input remote sensing semantic image.

[0014] Furthermore, the multi-scale feature extraction process includes: During the training phase, the heavy structural reparameterizable basic module in the training state adopts a multi-branch convolutional path to enhance its expressive power. After normalization, the output of each branch is weighted and aggregated according to the channel-level gating coefficient to obtain the module output. During the inference / deployment phase, the multi-branch structure of the heavy structural reparameterizable basic module in the training state is equivalently transformed into a single-path lightweight structural reparameterizable basic module in the deployment state through the reparameterization operation. The lightweight, reparameterizable base module in deployment mode replaces branch computation with an equivalent convolution. Through the folding and fusion of each branch convolution, normalized parameters, and gating coefficients, equivalent mapping is performed to obtain multi-scale features.

[0015] Furthermore, the output of the heavy-weight, reparameterizable basic module in the training state is:

[0016] in, Indicates input features, Represents a set of branches. Indicates the first Convolution and normalization combination of each branch Indicates the gating parameters, Indicates the temperature coefficient. Indicates the gating coefficient. This represents the Sigmoid function. This indicates scaling element-wise by channel; The equivalent mapping of the lightweight, reparameterizable base module in deployment state is:

[0017]

[0018] in, This indicates that the deployment state is equivalent to a 3×3 convolution kernel. This indicates the deployment-state equivalent bias. This represents a single-path convolution mapping with equivalent convolution kernel and equivalent bias as parameters.

[0019] Furthermore, the region selection mask is:

[0020] in, Indicates the first Region selection mask at each quadtree level Represents the regional response map. Indicates the first A quadtree-level space pooling operation. Indicates whether it is learnable or has a preset threshold. Indicates an indicator function; Region attention calculation is represented as:

[0021] in, Indicates the first Region attention computation at each quadtree level Indicates the first Input features of a quadtree level, This indicates a region window attention or offset window attention operation.

[0022] Furthermore, the serial fusion process includes: Receive the current stage features and the memory state of the previous stage, preprocess the current stage features, and obtain an intermediate representation suitable for context fusion; When the memory state of the previous stage does not exist, the guiding branch is used to generate the memory state of the current stage based on the features of the current stage. When the memory state of the previous stage exists, it is aligned to the feature scale of the current stage as needed, and then convolved and fused with the features of the current stage to obtain the memory state of the current stage. The memory state generated in the current stage is injected into the main feature path, and the nonlinear expression is enhanced through local convolution and feedforward network to output enhanced features with enhanced context consistency.

[0023] Furthermore, the fusion characteristics are:

[0024] in, Indicates enhanced features, This represents the fusion mapping obtained by injecting the current stage's memory state into the backbone features and enhancing them through convolution / MLP. Indicates the characteristics of the current stage. This indicates the current memory state. This represents the residual projection used for dimension matching.

[0025] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a remote sensing semantic segmentation system. By introducing a quadtree adaptive weighted attention unit, relying on quadtree adaptive region selection and region weighted attention mechanisms, it can automatically distinguish between complex and simple regions within an image, accurately enhancing the segmentation effect of building boundaries, small targets, and densely textured regions, while reducing redundant computation in simple regions, effectively lowering computational overhead. Furthermore, by combining a self-growing reparameterizable convolutional structure with a global context fusion unit, it balances inference efficiency and deployment feasibility while ensuring segmentation accuracy, thus effectively alleviating the problems of high computational cost, insufficient real-time performance, and cross-scale semantic inconsistency faced by existing methods in large-scale remote sensing data processing and engineering implementation scenarios. The global context fusion unit maintains the context memory state through cross-layer serial calls, continuously injecting the global semantic information accumulated in the previous stage into the subsequent stage fusion process, thereby enhancing cross-scale semantic consistency and overall structural stability, and improving robustness and generalization ability in complex scenarios. Compared to solutions requiring significant computing power or relying on additional post-processing, all key modules of this invention can be trained end-to-end within a unified network framework. Furthermore, lightweight execution can be achieved during the inference phase through reparameterization and region selection strategies. While ensuring high-precision segmentation, it also considers inference speed and engineering deployment adaptability, effectively addressing the pain points of existing remote sensing semantic segmentation technologies, such as high computational cost, poor real-time performance, and cross-scale semantic inconsistency. This comprehensively improves the overall accuracy, operational stability, and practical engineering application value of remote sensing semantic segmentation tasks. Adaptable to batch processing of large-format remote sensing images and deployment in various resource-constrained scenarios, it can be widely applied in fields such as land resource surveys, urban planning and renewal, natural resource monitoring, ecological environmental protection, disaster assessment and emergency response, and transportation and infrastructure surveys. It provides efficient and reliable technical support for rapid mapping and dynamic monitoring of large areas, demonstrating significant engineering application value and social benefits.

[0026] Furthermore, this invention introduces a self-growing, reparameterizable basic module. During the training phase, it obtains stronger representational capabilities through multi-branch gating. During the inference and deployment phase, it folds the multi-branch structure into a single convolutional structure, thereby reducing the number of operators and inference latency while maintaining output consistency as much as possible, achieving a balance between strong training representation and high deployment efficiency.

[0027] This invention also provides a remote sensing semantic segmentation method. First, it extracts multi-scale features from remote sensing images level by level. Then, by introducing a quadtree adaptive region selection and region weighted attention mechanism, it generates a region selection mask. Using region attention, it accurately achieves fine modeling of complex regions and suppresses redundancy in simple regions, completing spatial adaptive fine modeling. This effectively improves the segmentation accuracy of key regions such as building boundaries and small targets while reducing computational load. Simultaneously, through spatial adaptive modeling, an efficient and deployable feature reconstruction mechanism, and continuous global context fusion in the cross-scale decoding stage, it continuously transmits global semantic content, strengthens feature context consistency and cross-scale semantic uniformity, and improves the stability and generalization ability of complex scene segmentation. Finally, it achieves pixel-level classification and outputs accurate segmentation results. This invention, through the overall approach of region adaptive computation allocation, context memory fusion, and consistent training deployment, effectively optimizes segmentation accuracy and operational efficiency, significantly improving the problems of uneven semantic expression and high computational consumption in traditional methods. It achieves fine modeling and unified semantic expression for complex remote sensing scenes, thereby improving the overall accuracy, stability, and engineering practicality of remote sensing semantic segmentation tasks, and greatly enhancing the overall effect and practical application value of remote sensing semantic segmentation. It can also be extended to remote sensing vision tasks such as hyperspectral image interpretation, change detection, instance segmentation, and target detection. Furthermore, it can be combined with edge computing, mobile deployment, and cloud batch processing workflows, providing new implementation paths and development space for the lightweight, large-scale, and automated application of remote sensing image intelligent interpretation systems.

[0028] Furthermore, this invention introduces a multi-scale feature-driven quadtree adaptive region partitioning method. By fusing multi-scale features from the encoder to generate a region response map, pooling and thresholding are performed at different levels to output a quadtree-level region selection mask / node set, providing constraints for subsequent region-level calculations.

[0029] Furthermore, this invention employs a weighted attention calculation method based on quadtree region selection, which applies window attention and offset window attention only to the selected region and then integrates it with local convolution and feedforward networks to aggregate the full image features. This improves the ability to model complex regions while reducing redundant computation.

[0030] Furthermore, this invention utilizes the cross-layer serial memory fusion mechanism of the global context fusion unit to continuously inject global semantic information from the previous stage into the feature fusion process of the next stage by maintaining and updating the cross-layer context memory state, thereby enhancing cross-scale semantic consistency and segmentation result stability. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the overall architecture and data flow of the remote sensing semantic segmentation system of the present invention.

[0033] Figure 2 This is a schematic diagram of the heavy-duty reparameterizable basic module in the training state and the light-duty reparameterizable basic module in the deployment state of the present invention.

[0034] Figure 3 This is a schematic diagram of the structure and region selection mask generation process of the quadtree region adaptive module of the present invention.

[0035] Figure 4 This is a schematic diagram of the weighted attention calculation process based on quadtree region selection according to the present invention.

[0036] Figure 5 This is a schematic diagram of the cross-layer memory fusion structure of the global context fusion unit of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0038] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0040] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0041] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0042] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0043] The present invention will now be described in further detail with reference to the accompanying drawings: See Figure 1 The present invention provides a remote sensing semantic segmentation system, including an encoder, a quadtree adaptive weighted attention unit, a global context fusion unit (GCFU), and a segmentation head.

[0044] The encoder is used to extract multi-scale features from the input remote sensing semantic image step by step, forming feature representations with different spatial resolutions and semantic levels. To simultaneously meet the requirements of high-precision representation and efficient inference deployment for remote sensing semantic segmentation, this invention sets up a self-growing reparameterizable basic module in the feature extraction backbone of the encoder, which is divided into a heavy structure reparameterizable basic module (HRBB) in the training state and a light structure reparameterizable basic module (LRBB) in the deployment state. The heavy structure reparameterizable basic module in the training state is used to enhance the expressive power by using a multi-branch convolutional path during the training phase. The multi-branch convolutional path includes at least three types of convolutional transformation branches, each of which is used to output the module output obtained by normalizing and weighting it according to the channel-level gating coefficients. The light structure reparameterizable basic module in the deployment state is used to replace branch calculations with an equivalent convolution during the inference / deployment phase, reducing the number of operators and inference latency. The parameters of the equivalent convolution are obtained by folding and fusing the convolutions of each branch, the normalized parameters, and the gating coefficients.

[0045] The quadtree adaptive weighted attention unit comprises a quadtree region adaptive module and a quadtree weighted attention module. The quadtree region adaptive module receives multi-scale features from the encoder and generates a region selection mask based on these features. The quadtree weighted attention module performs region attention calculations under mask constraints, providing detailed modeling for complex regions and redundancy suppression for simple regions.

[0046] The global context fusion unit is used to serially fuse features from different decoding stages and continuously introduce global semantic information from the previous stage into the feature processing of the next stage to obtain fused features with enhanced context consistency.

[0047] The segmentation head is used to perform pixel-level classification prediction on the fused features and outputs a semantic segmentation result with the same size as the input remote sensing semantic image.

[0048] This invention also provides a remote sensing semantic segmentation method. The input remote sensing semantic image is first processed by an encoder to extract multi-scale features step by step, forming feature representations with different spatial resolutions and semantic levels, denoted as (…). , , , The aforementioned multi-scale features serve as the basic input for the decoding stage and are also fed into the quadtree region adaptive module to generate a region selection mask. This region selection mask reflects the structural complexity and semantic importance of different spatial regions at different scales, indicating which regions require more refined calculation and modeling in subsequent decoding processes. During the progressive restoration of spatial resolution, the quadtree adaptive weighted attention unit introduces the quadtree weighted attention module and, under the constraint of the region selection mask, performs window attention and offset window attention operations only on the selected key regions, thereby achieving focused enhancement of complex regions and computational suppression of simple regions. In the multi-scale decoding and feature fusion process, to further ensure cross-scale semantic consistency, this invention uses a Global Context Fusion Unit (GCFU) to serially fuse features from different decoding stages. By maintaining cross-layer context memory states, global semantic information from the previous stage is continuously introduced into the feature processing of the next stage, resulting in fused features with enhanced context consistency. Finally, the fused features are subjected to pixel-level classification prediction by the segmentation head, outputting a semantic segmentation result with the same size as the input remote sensing semantic image. During the training phase, the network can be jointly optimized by auxiliary supervision branches; during the inference phase, only the backbone path is retained to achieve efficient deployment. Through the collaborative work of the above modules, this invention effectively improves the modeling capability and overall computational efficiency for complex remote sensing scenes while ensuring segmentation accuracy.

[0049] like Figure 2 As shown, HRBB employs a multi-branch convolutional path during the training phase to enhance expressive power, including at least three types of convolutional transformation branches (such as 3×3 branches, 1×1 branches, and concatenated branches). The outputs of each branch are normalized and then weighted and aggregated according to channel-level gating coefficients to obtain the module output. The gating coefficients can be adaptively adjusted during the training process, allowing effective channels / effective branches to gradually "grow," thereby improving the modeling ability for complex terrain boundaries, small targets, and texture details.

[0050] During the inference / deployment phase, to reduce computational load and branching overhead, the multi-branch structure of HRBB is equivalently transformed into a single-path LRBB through reparameterization operations. This means replacing all branch computations with an equivalent 3×3 convolution (including bias). The parameters of this equivalent convolution are obtained by folding and fusing the convolutions of each branch, normalization parameters, and gating coefficients. This significantly reduces the number of operators and inference latency while maintaining output consistency, achieving a balance between strong representation during training and high efficiency during deployment.

[0051] This process can be represented as follows: Let the input features be... The branch set is ,For example , No. The convolution and normalized combination of each branch is denoted as The gating parameters are The temperature coefficient is The output of HRBB can then be expressed as:

[0052] in, These are the channel-level gating coefficients obtained by using the Sigmoid function from the gating parameters and temperature coefficient. Indicates scaling element-wise by channel. Gating factor. It can be updated during training (auto-growth) to control the contribution of different branches / channels to the output.

[0053] By folding the convolution and normalization (and gating coefficients) of each branch, the equivalent parameters of the branches can be obtained. , Further, the 1×1 kernel is aligned to a 3×3 form (zero padding), and the cascaded branches are combined into kernels (convolution kernel compositing) to finally obtain a single equivalent 3×3 convolution parameter:

[0054] Therefore, LRBB is available. To achieve a functional mapping equivalent to HRBB, where, This indicates that the deployment state is equivalent to a 3×3 convolution kernel. This indicates the deployment-state equivalent bias. This represents a single-path convolution mapping with equivalent convolution kernel and equivalent bias as parameters.

[0055] To address the issue of highly uneven spatial structure complexity in remote sensing images, a quadtree adaptive weighted attention unit is used to achieve joint processing of region adaptive modeling and selective attention enhancement during the decoding stage.

[0056] like Figure 3 As shown, in the specific implementation, the quadtree region adaptation module first receives multi-scale features from the encoder and constructs a region response map (heatmap) through scale alignment and feature fusion. This region response map is used to characterize the structural complexity and semantic saliency of different spatial locations. Subsequently, the module pools the region response map at multiple spatial scales and combines a learnable threshold to determine whether to further subdivide the current region, thereby forming the region selection results corresponding to the quadtree structure at different levels. The region response map obtained by multi-scale feature fusion is denoted as... In the At each quadtree level, adaptive pooling is performed and compared with a threshold to obtain the region selection mask:

[0057] in, Indicates the first Region selection mask at each quadtree level Represents the regional response map. Indicates the first A quadtree-level space pooling operation. Indicates whether it is learnable or has a preset threshold. This indicates an indicator function.

[0058] Through this process, the image is adaptively divided into several sets of regions with different spatial scales. Complex regions correspond to deeper subdivision nodes, while simple regions stop being divided at a shallower level.

[0059] like Figure 4 As shown, after obtaining the region selection result, the quadtree weighted attention module, during the decoding process, uses a quadtree weighted attention mechanism to perform window attention and offset window attention operations only within the corresponding selected region at each level, and then fuses the attention results with the output of the local convolution and feedforward networks. In the... Hierarchy, only applicable to those who meet the requirements Performing attention calculations on the region can be represented as:

[0060] in, This represents the input features of the corresponding level. This indicates a region window attention or offset window attention operation.

[0061] Unselected regions are skipped or only undergo lightweight computation. In this way, attention computation is explicitly constrained to key regions, ensuring the ability to model complex structures while effectively reducing overall computational overhead. Finally, the feature results from each level of regions are aggregated according to spatial location to form an enhanced feature representation for subsequent decoding or output.

[0062] In the multi-scale decoding process of remote sensing semantic segmentation, features at different scales exhibit significant differences in semantic abstraction and spatial detail. Static stitching or independent layer-by-layer fusion can easily lead to cross-scale semantic inconsistencies, fragmented boundary details, and unstable local category decisions. To address this, this invention utilizes a Global Context Fusion Unit (GCFU) to sequentially fuse multi-scale / multi-stage features during the decoding phase. Furthermore, by maintaining and updating the cross-layer context memory state, it achieves continuous transmission of global semantic information, thereby improving the consistency and stability of the fusion process.

[0063] like Figure 5 As shown, GCFU receives the current stage characteristics at each invocation. Compared with the previous stage of memory state GCFU first preprocesses the features of the current stage (including convolutional transformations and optional resolution adjustments) to obtain intermediate representations suitable for context fusion. Then, based on the memory state of the previous stage... Does it exist that the memory state of the current stage is generated using different methods? : when If it does not exist, use the guiding branch to generate the memory state of the current stage based on the features of the current stage:

[0064] in, Indicates the memory state of the previous stage; when If it does not exist, it is determined by the characteristics of the current stage. Preprocessed mapping After processing, the initial memory is used to generate a mapping. To obtain the memory state at the current stage .

[0065] when If it exists, align it to the current stage feature scale as needed, concatenate it with the current stage features, and then perform convolutional fusion to obtain the memory state of the current stage. :

[0066] when If it exists, first perform a scale alignment operation. Will Align the features to the current stage scale, and then combine the current stage features with the aligned features. Concatenate by channel and input memory fusion mapping. To obtain the memory state at the current stage .

[0067] in, This represents a preprocessing mapping of the input features. Indicates channel-dimensional splicing. This indicates a scale alignment operation. and These represent the initial memory generation mapping and the memory fusion mapping, respectively. This indicates the memory state of the previous stage. This indicates the current memory state.

[0068] Subsequently, GCFU injects the fused memory state of the current stage into the backbone feature path and enhances the nonlinear expression through local convolution and feedforward networks (MLP), ultimately outputting context-enhanced features. It can be represented as:

[0069] in, This represents the fusion mapping obtained by injecting the current stage's memory state into the backbone features and enhancing them through convolution / MLP. This represents the residual projection used for dimension matching.

[0070] In an end-to-end data stream, GCFU is called serially multiple times using a shared instance, making... The process continuously accumulates and updates across multiple stages of decoding, thereby providing global semantic priors and cross-scale consistency constraints for subsequent stages.

[0071] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote sensing semantic segmentation system, characterized in that, It includes an encoder, a quadtree adaptive weighted attention unit, a global context fusion unit, and a segmentation head; The encoder is used to extract multi-scale features from the input remote sensing semantic image step by step, forming feature representations with different spatial resolutions and semantic levels; The quadtree adaptive weighted attention unit is used to receive multi-scale features from the encoder, decode them according to the multi-scale features, generate a region selection mask, and under the constraint of the region selection mask, perform region attention calculation to enhance complex regions and suppress simple regions. The global context fusion unit is used to serially fuse features from different decoding stages and continuously introduce global semantic information from the previous stage into the feature processing of the next stage to obtain fused features with enhanced context consistency. The segmentation head is used to perform pixel-level classification prediction on the fused features and outputs a semantic segmentation result with the same size as the input remote sensing semantic image.

2. The remote sensing semantic segmentation system according to claim 1, characterized in that, The encoder is equipped with a self-growing reparameterizable base module, which is divided into a heavy structure reparameterizable base module in the training state and a light structure reparameterizable base module in the deployment state. The heavy-weight, reparameterizable basic module in the training state is used to enhance expressive power by employing multi-branch convolutional paths during the training phase. The lightweight, reparameterizable base module in deployment mode is used to replace branch computations with an equivalent convolution during the inference / deployment phase, reducing the number of operators and inference latency.

3. The remote sensing semantic segmentation system according to claim 2, characterized in that, The multi-branch convolution path includes at least three types of convolution transformation branches, each of which is used to output the module output obtained by normalization and weighted aggregation according to the channel-level gating coefficients; The parameters of the equivalent convolution are obtained by folding and fusing the convolution of each branch, the normalization parameters, and the gating coefficients.

4. The remote sensing semantic segmentation system according to claim 1, characterized in that, The quadtree adaptive weighted attention unit includes a quadtree region adaptive module and a quadtree weighted attention module; The quadtree region adaptive module is used to receive multi-scale features from the encoder and generate a region selection mask based on the multi-scale features. The quadtree weighted attention module is used to perform region attention calculations under mask constraints, to perform detailed modeling of complex regions and to suppress redundancy in simple regions.

5. A remote sensing semantic segmentation method for the system according to any one of claims 1 to 4, characterized in that, include: Multi-scale features are extracted step by step from the input remote sensing semantic image to form feature representations with different spatial resolutions and semantic levels; Decoding is performed based on multi-scale features to generate a region selection mask. Under the constraint of the region selection mask, region attention is used to enhance complex regions and suppress simple regions. Features from different decoding stages are sequentially fused, and global semantic information from the previous stage is continuously introduced into the feature processing of the next stage to obtain fused features with enhanced context consistency. The fused features are classified and predicted at the pixel level, and the output is a semantic segmentation result with the same size as the input remote sensing semantic image.

6. The remote sensing semantic segmentation method according to claim 5, characterized in that, The multi-scale feature extraction process includes: During the training phase, the heavy structural reparameterizable basic module in the training state adopts a multi-branch convolutional path to enhance its expressive power. After normalization, the output of each branch is weighted and aggregated according to the channel-level gating coefficient to obtain the module output. During the inference / deployment phase, the multi-branch structure of the heavy structural reparameterizable basic module in the training state is equivalently transformed into a single-path lightweight structural reparameterizable basic module in the deployment state through the reparameterization operation. The lightweight, reparameterizable base module in deployment mode replaces branch computation with an equivalent convolution. Through the folding and fusion of each branch convolution, normalized parameters, and gating coefficients, equivalent mapping is performed to obtain multi-scale features.

7. The remote sensing semantic segmentation method according to claim 5, characterized in that, The output of the heavy-weight, reparameterizable basic module in the training state is: in, Indicates input features, Represents a set of branches. Indicates the first Convolution and normalization combination of each branch Indicates the gating parameters, Indicates the temperature coefficient. Indicates the gating coefficient. This represents the Sigmoid function. This indicates scaling element-wise by channel; The equivalent mapping of the lightweight, reparameterizable base module in deployment state is: in, This indicates that the deployment state is equivalent to a 3×3 convolution kernel. This indicates the deployment-state equivalent bias. This represents a single-path convolution mapping with equivalent convolution kernel and equivalent bias as parameters.

8. The remote sensing semantic segmentation method according to claim 5, characterized in that, The region selection mask is: in, Indicates the first Region selection mask at each quadtree level Represents the regional response map. Indicates the first A quadtree-level space pooling operation. Indicates whether it is learnable or has a preset threshold. Indicates an indicator function; Region attention calculation is represented as: in, Indicates the first Region attention computation at each quadtree level Indicates the first Input features of a quadtree level, This indicates a region window attention or offset window attention operation.

9. The remote sensing semantic segmentation method according to claim 5, characterized in that, The serial fusion process includes: Receive the current stage features and the memory state of the previous stage, preprocess the current stage features, and obtain an intermediate representation suitable for context fusion; When the memory state of the previous stage does not exist, the guiding branch is used to generate the memory state of the current stage based on the features of the current stage. When the memory state of the previous stage exists, it is aligned to the feature scale of the current stage as needed, and then convolved and fused with the features of the current stage to obtain the memory state of the current stage. The memory state generated in the current stage is injected into the main feature path, and the nonlinear expression is enhanced through local convolution and feedforward network to output enhanced features with enhanced context consistency.

10. The remote sensing semantic segmentation method according to claim 5, characterized in that, The fusion features are: in, Indicates enhanced features, This represents the fusion mapping obtained by injecting the current stage's memory state into the backbone features and enhancing them through convolution / MLP. Indicates the characteristics of the current stage. This indicates the current memory state. This represents the residual projection used for dimension matching.

Images