Image segmentation method and system based on local-global fusion and channel modulation

Abstract

The application discloses an image segmentation method and system based on local-global fusion and channel modulation, and relates to the technical field of image segmentation. The specific steps are as follows: a local-global fusion attention module and a depth estimation module are combined to encode an input remote sensing image, and a multi-scale fusion feature map is output; the multi-scale fusion feature map is subjected to feature compression and segmentation, cross fusion, channel modulation and adaptive weighting, and a modulation feature map is output; the spatial resolution of the modulation feature map is restored through step-by-step upsampling and convolution, and a pixel-level class prediction map is generated through segmentation. The application optimizes the spatial distribution consistency of features, helps to improve the class consistency of the overall segmentation result, reduces the class confusion phenomenon, and is especially suitable for complex remote sensing image segmentation tasks containing multiple classes and multiple scales.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and more specifically to an image segmentation method and system based on local-global fusion and channel modulation. Background Technology

[0002] Currently, deep learning-based convolutional neural network (CNN) architectures (such as U-Net, DeepLab, and SegNet) have become the mainstream method for remote sensing image segmentation. Through end-to-end feature extraction and decoding mechanisms, they have significantly improved the overall segmentation performance of various land features (such as buildings, water bodies, and vegetation) in images. However, most of these methods focus on local feature modeling and struggle to fully capture the large-scale contextual relationships spanning long distances in remote sensing images, resulting in insufficient expression of dependencies between land features. Furthermore, when processing high-resolution images, the inherent downsampling process of the model easily loses detailed information, severely impacting the segmentation accuracy of small targets and boundary regions. Although the introduction of the Transformer structure enhances global modeling capabilities, it suffers from high computational complexity and weak local perception capabilities. In addition, mechanisms that improve accuracy, such as channel attention and depth estimation, are often used as isolated modules, lacking systematic integration. Therefore, how to design a deep learning model that can effectively unify the modeling of local detail features, global semantic information and spatial structure relationships, and achieve efficient collaborative optimization of the three in high-resolution remote sensing image segmentation tasks, so as to improve the ability to identify and distinguish multi-scale and multi-category land features in complex scenes while maintaining boundary accuracy, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0003] In view of this, the present invention provides an image segmentation method and system based on local-global fusion and channel modulation, which overcomes the above-mentioned defects.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] An image segmentation method based on local-global fusion and channel modulation, the specific steps of which are as follows:

[0006] The input remote sensing image is encoded by jointly using a local-global fusion attention module and a depth estimation module, and a multi-scale fusion feature map is output.

[0007] The multi-scale fused feature map is subjected to feature compression and segmentation, cross-fusion, channel modulation and adaptive weighting to output a modulation feature map;

[0008] The spatial resolution of the modulated feature map is restored by progressive upsampling and convolution, and then segmented to generate a pixel-level category prediction map.

[0009] Optionally, the steps for obtaining the multi-scale fused feature map are as follows:

[0010] The remote sensing image is preprocessed and subjected to depthwise separable convolution to generate initial features;

[0011] The first local-global fusion attention module is used to extract and fuse the local and global features of the initial feature, and output the first feature;

[0012] Multi-level dilated convolution is used to extract terrain structure features based on the input features, and these features are concatenated with the input feature residuals to output a second feature.

[0013] The first feature and the second feature are fused together, and then processed by a parallel dual-branch and a third local-global fusion attention module to output a multi-scale fused feature map.

[0014] Optionally, the steps for generating the first feature are as follows:

[0015] Based on the initial features, a query vector Q, a key vector K, and a value vector V are generated, and a dot product attention mechanism is used to interact and fuse local and global features to generate the first fused feature.

[0016] The first fused feature and the initial feature are then fused together after performing a convolution operation to generate a second fused feature;

[0017] The second fusion feature is input into a linear layer for processing, and then fused with the convolution result of the first fusion feature to generate a third fusion feature.

[0018] After the third fused feature is input into the multilayer perceptron, it is residually connected with the third fused feature itself, and then the first feature is output after being subjected to depthwise separable convolution.

[0019] Optionally, the generation step of the first fusion feature is as follows:

[0020] The similarity matrix between the query vector Q and the key vector K is calculated using a dot product attention mechanism. After normalization by the Softmax activation function, an attention weight matrix is ​​generated. The attention weight matrix and the value vector V are weighted and summed to achieve feature interaction fusion, and the first fused feature is output.

[0021] Optionally, the step of generating the modulation feature map is as follows:

[0022] The multi-scale fused feature map is segmented into three sub-features after global average pooling and MLP compression.

[0023] After convolution processing any two sub-features, the three sub-features are then concatenated to generate an intermediate fused feature;

[0024] The intermediate fused feature is further divided into two sub-features, and the channel modulation weight vector is generated by adding the two sub-features, global average pooling, MLP compression, and activation function.

[0025] The channel modulation weight vector is used to reweight the two sub-features respectively, and the convolution result of the weighted feature and any one of the sub-features is fused to output the modulation feature map.

[0026] Optionally, the generation steps of the pixel-level category prediction map are as follows:

[0027] The spatial resolution of the modulation feature map is restored layer by layer by progressive upsampling and convolution to obtain the restored image;

[0028] The feature vector of each pixel location in the restored image is mapped to the category probability space to generate the pixel-level category prediction map.

[0029] Optionally, the parallel dual branches include a first branch consisting of a second local-global fusion attention module and a third depthwise separable convolutional layer, and a second branch consisting of a second depth estimation module.

[0030] An image segmentation system based on local-global fusion and channel modulation includes: a feature encoding subsystem, a feature modulation fusion subsystem, and a decoding and output subsystem;

[0031] The feature encoding module subsystem is used to encode the input remote sensing image in conjunction with the depth estimation module through a local-global fusion attention mechanism, and output a multi-scale fusion feature map;

[0032] The feature modulation fusion subsystem is used to perform feature compression and segmentation, cross-fusion, channel modulation and adaptive weighting on the multi-scale fused feature map, and output a modulation feature map.

[0033] The decoding and output subsystem is used to recover the spatial resolution of the modulation feature map through progressive upsampling and convolution, and to segment and generate pixel-level category prediction maps.

[0034] As can be seen from the above technical solution, the present invention provides an image segmentation method and system based on local-global fusion and channel modulation, which has the following advantages compared with the prior art:

[0035] 1. Enhanced understanding of complex scenes and recognition of small targets: Through the joint modeling mechanism of local-global fusion attention module and depth estimation module, the model effectively integrates the fine local texture features and large-scale contextual semantic information in remote sensing images, and enhances the modeling ability of terrain undulation and 3D structural information by utilizing depth estimation. This synergistic effect improves the model's overall understanding of complex terrain scenes (such as dense urban areas and mixed vegetation areas). In particular, in high-resolution remote sensing images, the recognition accuracy of small targets (such as small vehicles and independent buildings) and dense areas (such as dense vegetation and congested roads) is significantly improved, effectively alleviating the problems of missed detection and false detection caused by the lack of global information or loss of details in existing methods.

[0036] 2. Significantly Improved Boundary Clarity and Class Consistency: Based on the feature modulation fusion subsystem, combined with channel segmentation, cross-fusion, and adaptive weighting mechanisms, this module enables refined optimization of feature representation from two key dimensions: channel semantics and spatial distribution. This module effectively suppresses redundant information in features while significantly enhancing the feature response of land cover boundary regions. This makes the model more discriminative of boundaries between different land cover categories, significantly improving the boundary clarity of the segmentation results and making pixel-level classification of boundary regions more accurate. Simultaneously, this module optimizes the spatial distribution consistency of features, helping to improve the overall class consistency of the segmentation results and reduce class confusion, making it particularly suitable for complex remote sensing image segmentation tasks containing multiple classes and scale targets. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the method flow provided by the present invention;

[0039] Figure 2 This is a schematic diagram of the feature encoding module subsystem provided by the present invention;

[0040] Figure 3 This is a schematic diagram of the feature modulation fusion subsystem provided by the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] This invention discloses an image segmentation method based on local-global fusion and channel modulation, such as... Figure 1 As shown, the specific steps are as follows:

[0043] Step 1: Encode the input remote sensing image by jointly using the local-global fusion attention module and the depth estimation module, and output a multi-scale fusion feature map;

[0044] Step 2: Perform feature compression and segmentation, cross-fusion, channel modulation and adaptive weighting on the multi-scale fused feature map, and output the modulated feature map;

[0045] Step 3: Restore the spatial resolution of the modulated feature map by progressive upsampling and convolution, and perform segmentation to generate pixel-level category prediction maps.

[0046] In one embodiment, the steps for obtaining the multi-scale fused feature map are as follows:

[0047] Preprocessing and depthwise separable convolution of remote sensing images generate initial features;

[0048] The first local-global fusion attention module is used to extract and fuse the local and global features of the initial features, and output the first feature;

[0049] Multi-level dilated convolution is used to extract terrain structure features based on input features, and these features are concatenated with the input feature residuals to output a second feature.

[0050] The first and second features are fused, and then processed by a parallel dual-branch and a third local-global fusion attention module to output a multi-scale fused feature map.

[0051] In one embodiment, the step of generating the first feature is as follows:

[0052] Based on the initial features, a query vector Q, a key vector K, and a value vector V are generated. A dot product attention mechanism is then used to fuse local and global features to generate the first fused feature.

[0053] The first fused feature and the initial feature are convolved and then fused to generate the second fused feature;

[0054] The second fusion feature is input into a linear layer for processing, and then fused with the convolution result of the first fusion feature to generate the third fusion feature.

[0055] After the third fused feature is input into the multilayer perceptron, it is residually connected with the third fused feature itself, and then output as the first feature after being subjected to depthwise separable convolution.

[0056] In one embodiment, the step of generating the first fusion feature is as follows:

[0057] The similarity matrix between the query vector Q and the key vector K is calculated using the dot product attention mechanism. After normalization by the Softmax activation function, the attention weight matrix is ​​generated. The attention weight matrix and the value vector V are weighted and summed to achieve feature interaction fusion, and the first fused feature is output.

[0058] Furthermore, the step involves constructing a feature encoding process based on a local-global fusion attention module and a depth estimation module, such as... Figure 2 As shown, this paper aims to construct a feature encoding process suitable for remote sensing image segmentation tasks, fusing local details and global contextual information to improve the ability to distinguish land cover categories and understand spatial structure in remote sensing images. It consists of preprocessing of the input remote sensing image, a local-global fusion attention mechanism, a depth estimation module, and a multi-scale fusion module; specifically:

[0059] Step 1.1: The original remote sensing image first undergoes preprocessing operations, such as noise removal (using Wiener filtering and Lee filtering), geometric registration, and image normalization, to obtain image I, which can be input into the neural network. Subsequently, image I is input into a depthwise separable convolutional network to extract initial features X. o Depthwise separable convolution operations have the advantages of fewer parameters and higher computational efficiency, making them suitable for feature extraction from high-resolution remote sensing images.

[0060] Step 1.2: Construct a local-global fusion attention module, such as... Figure 2 As shown in the upper part, to enhance the perception of targets of different scales and categories in remote sensing images, a local-global fusion attention module is constructed; this module uses the initial feature X o As input, the data first passes through convolutional layers and normalization processes, and then through three linear mapping layers to generate query, key, and value vectors respectively.

[0061] Q = LinearQ(X) o ), K = LinearK(X) o V = LinearV(X) o );

[0062] Subsequently, a dot product attention mechanism is used to fuse local and global features. The calculation process is as follows:

[0063]

[0064] Here, Softmax(·) represents the Softmax activation function, which is used to normalize attention weights.

[0065] Next, feature X 1 and X o After being processed separately by convolution, they are fused together, input into a linear layer, and then combined with X. 1 The convolutional features are added together to obtain the fused feature X. 2 :

[0066]

[0067] Where Conv(·) represents a convolution operation, and Linear(·) represents a linear layer. This indicates an element-wise addition operation.

[0068] Ultimately, X 2 After inputting into a multilayer perceptron (MLP), a residual connection is made with it to obtain the output feature X of the local-global fusion attention module. A :

[0069]

[0070] Step 1.3: Construct a depth estimation module, such as... Figure 2 The lower half is shown. The depth estimation module enhances the modeling ability of terrain changes or structural information in remote sensing images, and mainly consists of multiple dilated convolution layers and activation functions. Image I is first processed through ordinary convolution to obtain feature map X. 3 The data is then fed into a depth estimation module, which consists of multiple stacked dilated convolutions. The feature map output from the final layer is compared with X. 3 Perform residual connections to form the output of the depth estimation module:

[0071]

[0072] Here, DilatedConvBlock(·) represents a module consisting of convolutional layers with different dilation rates and nonlinear activation functions.

[0073] Step 1.4: Feature Fusion and Final Output; Feature X 4 With feature X A The features X are obtained by processing the data separately through convolutional layers and depthwise separable convolutional layers, then adding and fusing them. 5 :

[0074]

[0075] Subsequently, feature X 5The input consists of two branches: one is a combination of a local-to-global fusion attention module and a depthwise separable convolutional layer, and the other is a depth estimation module. The outputs of these two branches are then fused together and fed into the local-to-global fusion attention module to obtain the final feature output. This represents the final multi-scale fused feature map output in this step, which will be used for subsequent decoding and image segmentation tasks.

[0076] In one embodiment, the steps for generating the modulation feature map are as follows:

[0077] The multi-scale fused feature map is segmented into three sub-features after global average pooling and MLP compression.

[0078] After convolution processing any two sub-features, the three sub-features are then concatenated to generate an intermediate fused feature;

[0079] The intermediate fused features are further divided into two sub-features. The channel modulation weight vector is generated by adding the two sub-features, global average pooling, MLP compression, and activation function.

[0080] The two sub-features are re-weighted using the channel modulation weight vector, and the convolution result of the weighted feature and any sub-feature is fused to output the modulation feature map.

[0081] In one embodiment, the parallel dual-branch configuration includes a first branch consisting of a second local-global fusion attention module and a third depthwise separable convolutional layer, and a second branch consisting of a second depth estimation module.

[0082] Further, step 2 involves constructing a feature modulation fusion subsystem, such as... Figure 3 As shown. This step aims to refine the multi-scale features extracted in step 1. Further modulation and fusion are performed by introducing global context modeling and feature channel reweighting mechanisms to enhance the model's ability to represent different land cover categories and boundary information. The entire subsystem mainly includes four stages: feature compression and segmentation, cross-fusion, channel modulation, and enhanced representation.

[0083] Step 2.1: Feature Compression and Segmentation. The final feature map output from Step 1 is then processed... As input, a global description vector along the channel dimension is first extracted using global average pooling, and then fed into a multilayer perceptron to generate compressed modulation features. :

[0084]

[0085] Where GAP(·) represents global average pooling operation, and MLP(·) represents fully connected network. Then, for Three-channel segmentation is performed, resulting in three sub-features: This segmentation operation can be viewed as semantically dividing features in the channel dimension or semantic space so that subsequent modules can perform cross-information fusion.

[0086] Step 2.2: Feature cross-fusion. (The rest of the text appears to be incomplete and requires further context.) and Two convolutional layers are input separately to extract local spatial features, and then the features are concatenated to obtain fused features. :

[0087]

[0088] Here, Concat(·) represents a concatenation operation along the channel dimension. Then, The channels were divided again to obtain... and Two sub-features:

[0089]

[0090] This process aims to refine the feature representation to adapt it to subsequent channel modulation mechanisms.

[0091] Step 2.3: Channel modulation weight generation. and Perform element-wise addition and fusion:

[0092]

[0093] Then, channel modulation vectors are generated through global average pooling, multilayer perceptron, and activation functions. Used for subsequent feature weighting:

[0094]

[0095] Where σ(·) represents the activation function (e.g., Sigmoid or ReLU), This represents the generated weight vector, used to adjust the importance of features in each channel.

[0096] Step 2.4: Feature reweighting and output generation. Using modulation weights. To each and After reweighting the channels, we get:

[0097]

[0098] in, This represents channel-by-channel multiplication (element-by-element multiplication under a broadcast mechanism). Next, the two weighted features and the weights themselves are merged:

[0099]

[0100] Combine the merged features with the original The features are concatenated and further extracted through convolutional layers to obtain the fused representation, ultimately outputting the feature modulation fusion subsystem's output features. :

[0101]

[0102] Final features As input to subsequent stages of remote sensing image segmentation networks (such as decoders or segmentation heads), it has stronger discriminative and spatial representation capabilities, which helps to improve boundary clarity and class consistency.

[0103] In one embodiment, the steps for generating the pixel-level category prediction map are as follows:

[0104] The spatial resolution of the modulation feature map is restored layer by layer by progressive upsampling and convolution to obtain the restored image;

[0105] The feature vector of each pixel location in the recovered image is mapped to the class probability space to generate a pixel-level class prediction map.

[0106] Further, step 3 involves constructing the decoder and segmentation head, generating the final segmentation result, and building the loss function. This step aims to transform the high-dimensional semantic feature map processed by the feature encoding and fusion modules... The image is decoded into a pixel-level classification map of the same size as the original image. By constructing multiple loss functions, the spatial consistency and class discrimination ability of the model are optimized, thereby improving the accuracy of remote sensing image segmentation and the quality of boundary representation.

[0107] Step 3.1: Decoder Construction and Upsampling Recovery. The feature map output from Step 2... The input decoder module recovers spatial resolution layer by layer using progressive upsampling and convolution. The decoder contains multiple decoding units, each consisting of a deconvolution or upsampling module, a convolutional layer, and a normalization layer. The output of the i-th decoding layer is defined as... but:

[0108]

[0109] in, Upsample(·) represents an upsampling operation (which can use bilinear interpolation or deconvolution); Conv(·) represents a convolution operation; BN(·) represents batch normalization. During the upsampling process, the decoder may optionally incorporate shallow features from step 1 (such as X). o or X4 Perform cross-layer skip connections to enrich edge and detail information:

[0110]

[0111] in, ⊕ represents encoder features that match the current scale, and ⊕ represents element-wise addition and fusion.

[0112] Step 3.2: Segmentation Head and Class Prediction. Output features in the last layer of the decoder. A lightweight segmentation head is constructed to map high-dimensional features to a class probability space. The segmentation head consists of convolutional layers and a softmax activation function, and its output is a segmentation prediction map with the same size as the input remote sensing image. :

[0113]

[0114] in, H and W represent the height and width of the image, respectively, and C represents the number of classes. Channel mapping is implemented using 1×1 convolution, and the features are normalized to a per-pixel class probability distribution using Softmax.

[0115] Step 3.3: Constructing a multi-loss function for training and optimization. To improve the convergence and generalization ability of the remote sensing image segmentation model, a joint loss function is introduced, combining multiple supervision strategies to optimize classification accuracy and boundary quality. The true label is defined as Y, and the predicted output is... The total loss function is then:

[0116]

[0117] in: Cross-entropy loss, used to optimize pixel-level classification accuracy, is defined as follows:

[0118]

[0119] Dice loss is used to enhance the model's robustness to small objectives and class imbalance problems.

[0120]

[0121] Here, ∈ is a smoothing term to prevent the denominator from being zero.

[0122] Boundary-aware loss models the target edge by modeling the difference between the predicted and true boundaries. The boundary loss can be generated using Laplacian gradient calculations or boundary masks.

[0123]

[0124] in, This represents an edge extraction operator, such as Sobel or Laplacian. The weights λ1, λ2, and λ3 of the loss can be adjusted according to task requirements.

[0125] Finally, after this processing step, the model outputs the segmentation results of the remote sensing image. Each pixel is precisely assigned to its corresponding land feature category, such as buildings, roads, water bodies, and vegetation, exhibiting good spatial consistency and boundary accuracy, making it suitable for high-precision remote sensing image analysis scenarios.

[0126] This embodiment also discloses an image segmentation system based on local-global fusion and channel modulation, including: a feature encoding subsystem, a feature modulation fusion subsystem, and a decoding and output subsystem;

[0127] The feature encoding module subsystem is used to encode the input remote sensing image in conjunction with the depth estimation module through a local-global fusion attention mechanism, and output a multi-scale fusion feature map;

[0128] The feature modulation fusion subsystem is used to perform feature compression and segmentation, cross-fusion, channel modulation and adaptive weighting on multi-scale fused feature maps, and output modulated feature maps.

[0129] The decoding and output subsystem is used to recover the spatial resolution of the modulated feature map through progressive upsampling and convolution, and to segment and generate pixel-level class prediction maps.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0131] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image segmentation method based on local-global fusion and channel modulation, characterized in that, The specific steps are as follows: The input remote sensing image is encoded by jointly using a local-global fusion attention module and a depth estimation module, outputting a multi-scale fusion feature map. The steps for obtaining the multi-scale fusion feature map are as follows: The remote sensing image is preprocessed and subjected to depthwise separable convolution to generate initial features; The first local-global fusion attention module is used to extract and fuse the local and global features of the initial feature, and output the first feature; Multi-level dilated convolution is used to extract terrain structure features from the preprocessed remote sensing image, and these features are concatenated with the residuals of the preprocessed remote sensing image to output a second feature. The first feature and the second feature are fused together, and then processed by a parallel dual-branch and a third local-global fusion attention module to output a multi-scale fused feature map. The parallel dual-branch includes a first branch composed of a second local-global fusion attention module and a third depthwise separable convolutional layer, and a second branch composed of a second depth estimation module. The multi-scale fused feature map is subjected to feature compression and segmentation, cross-fusion, channel modulation and adaptive weighting to output a modulation feature map; The spatial resolution of the modulated feature map is restored by progressive upsampling and convolution, and then segmented to generate a pixel-level category prediction map.

2. The image segmentation method based on local-global fusion and channel modulation according to claim 1, characterized in that, The steps for generating the first feature are as follows: Based on the initial features, a query vector Q, a key vector K, and a value vector V are generated, and a dot product attention mechanism is used to interact and fuse local and global features to generate the first fused feature. The first fused feature and the initial feature are then fused together after performing a convolution operation to generate a second fused feature; The second fusion feature is input into a linear layer for processing, and then fused with the convolution result of the first fusion feature to generate a third fusion feature. After the third fused feature is input into the multilayer perceptron, it is residually connected with the third fused feature itself, and then the first feature is output after being subjected to depthwise separable convolution.

3. The image segmentation method based on local-global fusion and channel modulation according to claim 2, characterized in that, The steps for generating the first fusion feature are as follows: The similarity matrix between the query vector Q and the key vector K is calculated using a dot product attention mechanism. After normalization by the Softmax activation function, an attention weight matrix is ​​generated. The attention weight matrix and the value vector V are weighted and summed to achieve feature interaction fusion, and the first fused feature is output.

4. The image segmentation method based on local-global fusion and channel modulation according to claim 1, characterized in that, The steps for generating the modulation feature map are as follows: The multi-scale fused feature map is segmented into three sub-features after global average pooling and MLP compression. After convolution processing any two sub-features, the three sub-features are then concatenated to generate an intermediate fused feature; The intermediate fused feature is further divided into two sub-features, and the channel modulation weight vector is generated by adding the two sub-features, global average pooling, MLP compression, and activation function. The channel modulation weight vector is used to reweight the two sub-features respectively, and the convolution result of the weighted feature and any one of the sub-features is fused to output the modulation feature map.

5. The image segmentation method based on local-global fusion and channel modulation according to claim 1, characterized in that, The steps for generating the pixel-level category prediction map are as follows: The spatial resolution of the modulation feature map is restored layer by layer by progressive upsampling and convolution to obtain the restored image; The feature vector of each pixel location in the restored image is mapped to the category probability space to generate the pixel-level category prediction map.

6. An image segmentation system based on local-global fusion and channel modulation, characterized in that, include: Feature coding subsystem, feature modulation fusion subsystem, and decoding and output subsystem; The feature encoding module subsystem is used to encode the input remote sensing image in conjunction with the depth estimation module through a local-global fusion attention mechanism, and output a multi-scale fusion feature map. The steps for obtaining the multi-scale fusion feature map are as follows: The remote sensing image is preprocessed and subjected to depthwise separable convolution to generate initial features; The first local-global fusion attention module is used to extract and fuse the local and global features of the initial feature, and output the first feature; Multi-level dilated convolution is used to extract terrain structure features from the preprocessed remote sensing image, and these features are concatenated with the residuals of the preprocessed remote sensing image to output a second feature. The first feature and the second feature are fused together, and then processed by a parallel dual-branch and a third local-global fusion attention module to output a multi-scale fused feature map. The parallel dual-branch includes a first branch composed of a second local-global fusion attention module and a third depthwise separable convolutional layer, and a second branch composed of a second depth estimation module. The feature modulation fusion subsystem is used to perform feature compression and segmentation, cross-fusion, channel modulation and adaptive weighting on the multi-scale fused feature map, and output a modulation feature map. The decoding and output subsystem is used to recover the spatial resolution of the modulation feature map through progressive upsampling and convolution, and to segment and generate pixel-level category prediction maps.

Images