An optical remote sensing image semantic segmentation method
Patent Information
- Application Number
- CN202611060823.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-16
- Publication Date
- 2026-08-18
AI Technical Summary
然而，现有基于Transformer方法的传统空间域架构多数解码器依赖单尺度的查询-键-值特征，限制了模型对低层次空间细节与高层次语义信息之间关联的利用能力对高频特征的捕捉能力不足，而高频特征恰好是表征目标边缘、纹理等精细空间细节的关键，基于Transformer方法的传统空间域架构会导致分割结果不够精准，难以适应遥感图像中多尺度目标共存的场景
在本说明书提供的一种光学遥感图像语义分割方法中，通过将光学遥感图像输入多阶段不同尺度的编码器，得到多尺度特征图；并通过渐进式上下文聚合机制分别生成各解码器对应的查询特征和键值特征；将键值特征按通道分组，后进行多级离散小波分解，并进行自适应加权融合和得到低频特征和高频特征；并所述高频特征与所述低频特征融合后通过残差连接得到频率增强特征；通过注意力计算与上采样后输入多层感知机，生成遥感图像的语义分割掩码。
Smart Images
Figure CN122597429A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of remote sensing image processing technology, and in particular to a semantic segmentation method for optical remote sensing images. Background Technology
[0002] Optical remote sensing image semantic segmentation technology divides remote sensing images into regions with clear semantic information, providing key technical support for multiple fields such as resource exploration, urban planning, disaster early warning, and land use monitoring. With the development of deep learning technology, semantic segmentation models have made significant progress. Among them, methods based on convolutional neural networks perform well in local feature extraction, but are limited by the size of the receptive field, making it difficult to effectively model global contextual information and long-distance dependencies.
[0003] To address this issue, Transformer-based models have been introduced into the field of semantic segmentation. With its self-attention mechanism at its core, the Transformer efficiently captures global information and long-range dependencies, significantly improving model performance. However, most existing Transformer-based traditional spatial domain architectures rely on single-scale query-key-value features, limiting their ability to utilize the correlation between low-level spatial details and high-level semantic information, and their insufficient ability to capture high-frequency features. These high-frequency features are crucial for characterizing fine spatial details such as target edges and textures. Traditional Transformer-based spatial domain architectures result in inaccurate segmentation results, making them ill-suited for scenarios where multi-scale targets coexist in remote sensing images. Summary of the Invention
[0004] Therefore, it is necessary to provide a semantic segmentation method for optical remote sensing images to address the aforementioned technical problems.
[0005] The following technical solution is adopted in this specification: This specification provides a method for semantic segmentation of optical remote sensing images, including: Acquire optical remote sensing images; The optical remote sensing image is input into a pre-trained multi-stage encoder to obtain feature maps at each stage scale; based on the feature maps at each stage scale, query features and key-value features corresponding to each stage are generated through a progressive context aggregation mechanism; the feature map consists of length, width, and channel dimensions. The value features are grouped according to their own channel dimensions to obtain multiple sub-feature groups. Each sub-feature group is then decomposed into low-frequency feature components and multi-level high-frequency feature components using a multi-level discrete wavelet transform. The low-frequency feature components and multi-level high-frequency feature components corresponding to each sub-feature group are then adaptively weighted and fused according to the channel dimensions to obtain low-frequency features and high-frequency features. The wavelet decomposition levels corresponding to different sub-feature groups are different. The query features are used to fuse the high-frequency features and the low-frequency features through a query-guided modulation mechanism to obtain fused features; the fused features are then fused with the value features to obtain frequency-enhanced features; attention calculations are performed on the query features, key features, and frequency-enhanced features corresponding to each stage to obtain the enhanced features of each stage. The enhanced features from each stage are sequentially upsampled to the size of the optical remote sensing image and then stitched together. A semantic segmentation mask is generated using a multilayer perceptron to perform semantic segmentation on the optical remote sensing image.
[0006] Furthermore, the process of generating query features corresponding to each stage includes: When the feature map is the feature map output by the highest stage of the encoder, the query feature is directly taken from the feature map output by the encoder. When the feature map is not the feature map output by the highest stage of the encoder, the feature map output by the encoder in the current stage is fused with the query feature of the previous stage through the channel-space fusion module to generate the query feature of the current stage.
[0007] Furthermore, the channel-space fusion module includes a channel aggregation submodule and a space aggregation submodule; The channel aggregation submodule consists of a global average pooling layer, a 1×1 convolutional layer, a ReLU activation function layer, and a 1×1 convolutional layer connected in sequence. The spatial aggregation submodule consists of average pooling, max pooling, a 7×7 convolutional layer, and an activation function layer, wherein the average pooling layer and the max pooling layer are connected in parallel, and their outputs are input to the 7×7 convolutional layer.
[0008] Furthermore, the process of generating query features through the channel-space fusion module includes: The feature map output by the encoder in the current stage is aligned with the query feature in the previous stage by bilinear interpolation and linear projection. Based on the aligned features, adaptive channel weights are calculated through the channel aggregation submodule and multiplied with the feature map output by the encoder at the current stage; The features after multiplication are fused by the spatial aggregation submodule, and then multiplied again by the feature map output by the encoder at the current stage. After 1×1 convolution and residual connection, the query features at the current stage are obtained.
[0009] Furthermore, the generation process of the key and value features includes: When the feature map is the feature map output by the encoder at the highest stage, the key and value features are directly taken from the feature map output by the encoder. When the feature map is not the feature map output by the encoder at the highest stage, the feature map output by the encoder at the current stage and the feature map output by the decoder at the previous stage are downsampled to the same size by average pooling and linear projection, and then the key and value features of the current stage are spliced along the channel dimension to output the key and value features of the current stage.
[0010] Furthermore, the low-frequency features are used to characterize the target's semantics and contour information; The multi-level high-frequency features each contain three high-frequency sub-bands: HL, LH, and HH, which are used to characterize horizontal edges, vertical edges, and diagonal texture details, respectively.
[0011] Furthermore, the process of adaptively weighting and fusing the low-frequency feature components corresponding to each sub-feature group according to the channel dimension includes: The low-frequency features obtained from each decomposition are adaptively weighted and upsampled, and channel-dimensional weighted fusion is performed on the low-frequency features through learnable parameters to achieve adaptive modulation of each group of low-frequency features; Bilinear interpolation is used to upsample the weighted low-frequency features to the original input feature size. The low-frequency features of different groups are then spliced along the channel dimension to obtain the fused low-frequency features.
[0012] Furthermore, the process of adaptively weighting and fusing the multi-level high-frequency feature components corresponding to each sub-feature group according to the channel dimension includes: The multi-level high-frequency features of each group are fused and adaptively weighted step by step, and the three sub-bands of each high-frequency level are spliced along the channel dimension to obtain single-level high-frequency features. By weighting and fusing single-level high-frequency features using learnable parameters, effective high-frequency details are adaptively enhanced and noise is suppressed. After upsampling to the original size, the enhanced high-frequency features of all levels within the same group are summed element by element to obtain the high-frequency features of that group. The high-frequency features of different groups are spliced along the channel dimension to obtain the fused high-frequency features.
[0013] Furthermore, the multi-stage encoder is a MiT, PVT, and MSCAN encoding architecture.
[0014] This invention provides an optical remote sensing image semantic segmentation system, comprising: Image acquisition module, used to acquire optical remote sensing images; The progressive context aggregation module is used to input the optical remote sensing image into a pre-trained multi-stage encoder to obtain feature maps at each stage scale; based on the feature maps at each stage scale, query features and key-value features corresponding to each stage are generated through the progressive context aggregation mechanism; the feature maps are composed of length, width and channel dimensions. The dynamic wavelet transform attention module is used to group the value features according to their own channel dimensions to obtain multiple sub-feature groups, and to perform multi-level discrete wavelet decomposition on each sub-feature group through a multi-level discrete wavelet transform to obtain low-frequency feature components and multi-level high-frequency feature components; and to adaptively weight and fuse the low-frequency feature components and multi-level high-frequency feature components corresponding to each sub-feature group according to the channel dimensions to obtain low-frequency features and high-frequency features. The attention enhancement module is used to fuse the high-frequency features and the low-frequency features using the query features through a query-guided modulation mechanism to obtain fused features; and to fuse the fused features with the value features to obtain frequency enhancement features; and to perform attention calculations on the query features, key features, and frequency enhancement features corresponding to each stage to obtain the enhanced features of each stage. The semantic segmentation module is used to sequentially upsample the enhanced features of each stage to the size of the optical remote sensing image and then stitch them together. It also generates a semantic segmentation mask through a multilayer perceptron to perform semantic segmentation on the optical remote sensing image.
[0015] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: In the semantic segmentation method for optical remote sensing images provided in this specification, the optical remote sensing image is input into a multi-stage encoder at different scales to obtain a multi-scale feature map; query features and key-value features corresponding to each decoder are generated through a progressive context aggregation mechanism; the key-value features are grouped by channel, then subjected to multi-level discrete wavelet decomposition, and adaptive weighted fusion is performed to obtain low-frequency features and high-frequency features; the high-frequency features and the low-frequency features are fused and then subjected to residual connection to obtain frequency enhancement features; after attention calculation and upsampling, the data is input into a multilayer perceptron to generate a semantic segmentation mask for the remote sensing image.
[0016] The optimized query features and key-value features generated by progressive context aggregation are fed into a multi-level discrete wavelet transform for frequency enhancement, and then standard attention calculation is performed. This collaborative design solves the problem of insufficient multi-scale feature fusion in remote sensing semantic segmentation through progressive context aggregation, and solves the core technical problem of insufficient fine spatial detail extraction through multi-level discrete wavelet transform. The combination of the two improves the representation ability of global context and fine spatial details, and achieves more accurate segmentation.
[0017] Furthermore, a progressive context aggregation mechanism is used to hierarchically generate query features and key-value features. Utilizing a channel-space fusion module and a spatial reduction stitching module, a progressive fusion of low-level spatial details and high-level semantic information is achieved. Compared to traditional decoders based on single-scale query-key-value features, this method can more fully utilize the feature information from each stage of the encoder, effectively adapting to complex scenarios in optical remote sensing images where target scales differ greatly and multiple scales coexist, significantly improving the accuracy of semantic segmentation. A dynamic wavelet transform attention module groups the value features and performs multi-level discrete wavelet decomposition to extract low-frequency semantic features and high-frequency detail features respectively. Through learnable frequency weighting parameters and query guidance vectors, adaptive enhancement and noise suppression of high-frequency details are achieved, and residual connections are used to preserve the original features. This mechanism compensates for the shortcomings of traditional spatial domain architectures in capturing high-frequency features, resulting in clearer target boundaries and more complete preservation of fine structures in the segmentation results. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0019] Figure 1 A flowchart illustrating an optical remote sensing image semantic segmentation method provided by the present invention; Figure 2 This is a schematic diagram of the decoder structure provided by the present invention; Figure 3 A schematic diagram of the channel-space fusion (CSF) module structure provided by the present invention; Figure 4 A schematic diagram of the Space Reduction Concrete (SRC) module structure provided by the present invention; Figure 5 This is a schematic diagram of the dynamic wavelet transform attention module structure provided by the present invention. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.
[0021] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.
[0022] Figure 1 This is a flowchart illustrating a semantic segmentation method for optical remote sensing images as described in this specification, which specifically includes the following steps: Step 1: Data input and feature extraction.
[0023] A 512×512×3 optical remote sensing image is input into a pre-trained multi-scale encoder (MiT, PVT, MSCAN, and other mainstream coding architectures) containing four stages. The encoding process generates multi-scale feature maps corresponding to each stage of the encoder. ,in , Let be the number of channels in the feature map of stage i.
[0024] Step 2: Implementation of the progressive context aggregation mechanism Optimized query features are dynamically generated at each decoder stage through a progressive context aggregation mechanism. and key features Sum value characteristics To achieve progressive fusion of multi-scale features: 1. Query characteristics generate: (0) When At stage 4 (the highest level), the query features are directly taken from the corresponding encoder features, i.e. ; (0) When At 4 o'clock, the encoder features of the current stage are fused through the Channel-Spatial Fusion (CSF) module. Query features compared to the previous stage Generate query features for the current stage: ; (0) CSF module operation: such as Figure 3The CFS module shown includes a Channel Aggregation (CA) submodule and a Spatial Aggregation (SA) submodule. The CA submodule consists of a globally average pooling layer, a 1×1 convolutional layer, a ReLU activation function layer, and another 1×1 convolutional layer connected in sequence. The SA submodule consists of an activation function, a 7×7 convolution, and parallel concatenated average pooling and max pooling. First, bilinear interpolation and linear projection are used to... and Perform feature alignment to obtain Then, adaptive channel weights are calculated through the CA submodule, and multi-scale spatial features are fused through the SA submodule; finally, the original features are modulated by multiplication. F i And output through residual connection, the formula is:
[0025] 0. The generation methods for key features and value features are the same. Taking key features as an example, the generation process is as follows: (0) When At stage 4 (the highest stage), the key-value features are directly taken from the corresponding encoder features, i.e. .
[0026] (0) When At that time, the Spatial Reduction Concatenation (SRC) module fuses the encoder features of the current stage. Compared with the decoder output of the previous stage Generate key-value features for the current stage: ; Among them, such as Figure 2 The decoder shown contains four modules, each corresponding to one of the four stages of the encoder. Each module decodes the small feature maps generated by the encoder at the corresponding stage layer by layer, gradually recovering the spatial size to obtain the segmentation result.
[0027] (0) SRC module operation: such as Figure 4 As shown, average pooling and linear projection are used to... and Downsampling to uniform size Then, the output is spliced along the channel dimension.
[0028] Step 3: Enhancement of the Dynamic Wavelet Transform Attention Module The query features generated in each stage of step two and key features Input as follows Figure 5 The dynamic wavelet transform attention module shown performs frequency feature enhancement.
[0029] 1. Module Initialization Initialize a multi-level discrete wavelet transform (using the db1 wavelet basis, supporting 1~3 level decomposition), adaptive frequency weighting parameters, and 1×1 convolutional fusion units, and perform normal distribution and initialization of the weights to ensure model training stability.
[0030] 2. Feature Grouping and Wavelet Decomposition Value characteristics Divided into 3 groups according to channels, multiple sub-feature groups are obtained. Perform 1-3 level independent two-dimensional discrete wavelet transforms (DWT, db1 wavelet basis) on different groups to decompose them into low-frequency characteristic components. and multi-level high-frequency characteristic components Low-frequency features represent core information such as target semantics and contours. Each level of the multi-level high-frequency features contains three high-frequency sub-bands: HL, LH, and HH, representing horizontal edges, vertical edges, and diagonal texture details, respectively. Different groups employ different decomposition levels to achieve multi-scale frequency feature extraction, adapting to the detail requirements of targets at different scales in remote sensing images.
[0031] 3. Frequency component weighted fusion For each set of low-frequency features obtained from the decomposition, adaptive weighting and upsampling are performed, using learnable parameters. For low-frequency features Channel-dimensional weighted fusion is performed to achieve adaptive modulation of each group of low-frequency features. Bilinear interpolation is used to upsample the weighted low-frequency features to the original input feature size. The low-frequency features from different groups are then concatenated along the channel dimension to obtain the fused low-frequency features. ;
[0032] The multi-level high-frequency features of each group are fused and adaptively weighted step by step. The three sub-bands of each high-frequency level are spliced along the channel dimension to obtain single-level high-frequency features. These features are then processed using learnable parameters. Each high-frequency feature is weighted and fused to adaptively enhance effective high-frequency details and suppress noise. The enhanced high-frequency features at all levels within the same group are summed element-wise to obtain the fused high-frequency features for that group, which are then upsampled to the original size. The high-frequency features from different groups are then stitched together along the channel dimension to obtain the fused high-frequency features. .
[0033] 4. High-frequency modulation and residual fusion Dynamic frequency characteristic modulation is achieved through a query-guided modulation mechanism, using the following formula: in The query features are used as guiding vectors to adaptively modulate high-frequency detail features, enhancing edge and texture information relevant to the current task while preserving low-frequency semantic features. Subsequently, the frequency-enhanced features are fused with the original features via residual connections to obtain... .
[0034] 5. Attention Calculation Attention is applied to the query features, key features, and fused value features generated in step two, and the enhanced features are output. .
[0035] Step 4: Feature Upsampling and Segmentation Output Features output from each decoder stage The images are sequentially upsampled to the original image size, stitched along the channel dimension, and then input into a multilayer perceptron to generate the final semantic segmentation mask.
[0036] Step 5: Model Training and Optimization The cross-entropy loss function was used as the training loss, and the constructed model was trained on the LoveDA and Vaihingen datasets using the AdamW optimizer (betas=(0.9, 0.999), weight decay=0.01) and an initial learning rate of 6×10⁻⁶. −5 A polynomial decay strategy was employed; the training batch size was 4, and the number of iterations was 160k. The trained model was tested and evaluated, and its intersection-over-union ratio (IoU) was calculated as the core evaluation metric to verify the model's semantic segmentation performance on optical remote sensing images.
[0037] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A semantic segmentation method for optical remote sensing images, characterized in that, include: Acquire optical remote sensing images; The optical remote sensing image is input into a pre-trained multi-stage encoder to obtain feature maps at each stage corresponding to the scale. Based on the feature maps corresponding to each stage, query features, key and value features corresponding to each stage are generated through a progressive context aggregation mechanism; the feature maps are composed of length, width and channel dimensions. The value features are grouped according to their channel dimensions to obtain multiple sub-feature groups. Each sub-feature group is then decomposed using a multi-level discrete wavelet transform to obtain low-frequency feature components and multi-level high-frequency feature components. The low-frequency feature components and multi-level high-frequency feature components corresponding to each sub-feature group are adaptively weighted and fused according to the channel dimensions to obtain low-frequency features and high-frequency features. The wavelet decomposition levels corresponding to different sub-feature groups are different. The query features are used to fuse the high-frequency features and the low-frequency features through a query-guided modulation mechanism to obtain fused features; the fused features are then fused with the value features to obtain frequency-enhanced features; attention calculations are performed on the query features, key features, and frequency-enhanced features corresponding to each stage to obtain the enhanced features of each stage. The enhanced features from each stage are sequentially upsampled to the size of the optical remote sensing image and then stitched together. A semantic segmentation mask is generated using a multilayer perceptron to perform semantic segmentation on the optical remote sensing image.
2. The optical remote sensing image semantic segmentation method as described in claim 1, characterized in that, The process of generating query features corresponding to each stage includes: When the feature map is the feature map output by the highest stage of the encoder, the query feature is directly taken from the feature map output by the encoder. When the feature map is not the feature map output by the highest stage of the encoder, the feature map output by the encoder in the current stage is fused with the query feature of the previous stage through the channel-space fusion module to generate the query feature of the current stage.
3. The optical remote sensing image semantic segmentation method as described in claim 2, characterized in that, The channel-space fusion module includes a channel aggregation submodule and a space aggregation submodule; The channel aggregation submodule consists of a global average pooling layer, a 1×1 convolutional layer, a ReLU activation function layer, and a 1×1 convolutional layer connected in sequence. The spatial aggregation submodule consists of average pooling, max pooling, a 7×7 convolutional layer, and an activation function layer, wherein the average pooling layer and the max pooling layer are connected in parallel, and their outputs are input to the 7×7 convolutional layer.
4. The optical remote sensing image semantic segmentation method as described in claim 2, characterized in that, The process of generating query features through the channel-space fusion module includes: The feature map output by the encoder in the current stage is aligned with the query feature in the previous stage by bilinear interpolation and linear projection. Based on the aligned features, adaptive channel weights are calculated through the channel aggregation submodule and multiplied with the feature map output by the encoder at the current stage; The features after multiplication are fused by the spatial aggregation submodule, and then multiplied again by the feature map output by the encoder at the current stage. After 1×1 convolution and residual connection, the query features at the current stage are obtained.
5. The optical remote sensing image semantic segmentation method as described in claim 1, characterized in that, The generation process of the key and value features includes: When the feature map is the feature map output by the encoder at the highest stage, the key and value features are directly taken from the feature map output by the encoder. When the feature map is not the feature map output by the encoder at the highest stage, the feature map output by the encoder at the current stage and the feature map output by the decoder at the previous stage are downsampled to the same size by average pooling and linear projection, and then the key and value features of the current stage are spliced along the channel dimension to output the key and value features of the current stage.
6. The optical remote sensing image semantic segmentation method as described in claim 1, characterized in that, The low-frequency features are used to characterize the target semantics and contour information; The multi-level high-frequency features each contain three high-frequency sub-bands: HL, LH, and HH, which are used to characterize horizontal edges, vertical edges, and diagonal texture details, respectively.
7. The optical remote sensing image semantic segmentation method as described in claim 1, characterized in that, The process of adaptively weighting and fusing the low-frequency feature components corresponding to each sub-feature group according to the channel dimension includes: The low-frequency features obtained from each decomposition are adaptively weighted and upsampled, and channel-dimensional weighted fusion is performed on the low-frequency features through learnable parameters to achieve adaptive modulation of each group of low-frequency features; Bilinear interpolation is used to upsample the weighted low-frequency features to the original input feature size. The low-frequency features of different groups are then spliced along the channel dimension to obtain the fused low-frequency features.
8. The optical remote sensing image semantic segmentation method as described in claim 1, characterized in that, The process of adaptively weighting and fusing the multi-level high-frequency feature components corresponding to each sub-feature group according to the channel dimension includes: The multi-level high-frequency features of each group are fused and adaptively weighted step by step, and the three sub-bands of each high-frequency level are spliced along the channel dimension to obtain single-level high-frequency features. By weighting and fusing single-level high-frequency features using learnable parameters, effective high-frequency details are adaptively enhanced and noise is suppressed. After upsampling to the original size, the enhanced high-frequency features of all levels within the same group are summed element by element to obtain the high-frequency features of that group. The high-frequency features of different groups are spliced along the channel dimension to obtain the fused high-frequency features.
9. The optical remote sensing image semantic segmentation method as described in claim 1, characterized in that, The multi-stage encoder uses MiT, PVT, and MSCAN encoding architectures.
10. A semantic segmentation system for optical remote sensing images, characterized in that, include: Image acquisition module, used to acquire optical remote sensing images; The progressive context aggregation module is used to input the optical remote sensing image into a pre-trained multi-stage encoder to obtain feature maps at each stage scale; based on the feature maps at each stage scale, query features and key-value features corresponding to each stage are generated through the progressive context aggregation mechanism; the feature maps are composed of length, width and channel dimensions. The dynamic wavelet transform attention module is used to group the value features according to their own channel dimensions to obtain multiple sub-feature groups, and to perform multi-level discrete wavelet decomposition on each sub-feature group through a multi-level discrete wavelet transform to obtain low-frequency feature components and multi-level high-frequency feature components; and to adaptively weight and fuse the low-frequency feature components and multi-level high-frequency feature components corresponding to each sub-feature group according to the channel dimensions to obtain low-frequency features and high-frequency features. An attention enhancement module is used to fuse the high-frequency features and the low-frequency features using the query features through a query-guided modulation mechanism to obtain fused features; The fusion feature is then fused with the value feature to obtain the frequency enhancement feature; Attention calculations are performed on the query features, key features, and frequency enhancement features corresponding to each stage to obtain the enhanced features for each stage. The semantic segmentation module is used to sequentially upsample the enhanced features of each stage to the size of the optical remote sensing image and then stitch them together. It also generates a semantic segmentation mask through a multilayer perceptron to perform semantic segmentation on the optical remote sensing image.