A method and system for semantic segmentation of ground features based on local-global remote sensing images

Abstract

This invention discloses a method and system for semantic segmentation of ground features based on local-global remote sensing images. The method includes the following steps: acquiring and preprocessing a remote sensing image of the target area to be segmented, and extracting features using a convolutional neural network to obtain a feature map F; obtaining global enhanced features by employing channel attention and spatial attention mechanisms based on the global information of feature map F; segmenting feature map F into multiple feature vectors, applying channel attention and spatial attention mechanisms to each feature vector, and then re-integrating them to obtain complete local enhanced features; fusing the global enhanced features and the complete local enhanced features to obtain enhanced features F'; and inputting enhanced features F' into a ground feature semantic segmentation model to obtain the ground feature semantic segmentation result P. This invention introduces spatial attention and channel attention mechanisms, comprehensively utilizing local and global information to improve the accuracy, robustness, and generalization ability of remote sensing image semantic segmentation.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to a method and system for semantic segmentation of ground features based on local-global remote sensing images. Background Technology

[0002] With the continuous development of remote sensing technology, aircraft remote sensing imagery is increasingly widely used in geographic information systems, environmental monitoring, urban planning, and agricultural management. Aircraft remote sensing imagery, acquired by remote sensing sensors mounted on aircraft, has advantages such as high resolution, wide coverage, and fast acquisition speed, providing rich information about the Earth's surface. However, the task of semantic segmentation of ground features from aircraft remote sensing imagery faces many challenges, such as complex surface environments, diverse feature types, and noise interference, all of which affect the accuracy and efficiency of semantic segmentation.

[0003] Traditional semantic segmentation methods for aircraft remote sensing images primarily rely on manually designed feature extraction algorithms, such as texture, color, and shape features, when segmenting buildings, roads, or water bodies. While these methods perform well in certain specific scenarios, they often fail to capture all the information in complex environments, resulting in poor semantic segmentation performance. Furthermore, the feature extraction process for buildings, roads, or water bodies using traditional methods typically requires significant manual intervention, leading to inefficiency and making them unsuitable for large-scale data processing.

[0004] In recent years, with the rapid development of deep learning technology, Convolutional Neural Networks (CNNs) have demonstrated powerful capabilities in image-based semantic segmentation of ground features. CNNs automatically extract features from images of buildings, roads, or water bodies through multi-layer convolutional operations, capturing high-level information in the images and thus significantly improving the accuracy of semantic segmentation. However, relying solely on global feature extraction methods based on CNNs still has some limitations when processing aircraft remote sensing images. Ground features in aircraft remote sensing images typically have multi-scale features, and relying solely on global features makes it difficult to capture these detailed details, thus limiting the performance of semantic segmentation for buildings, roads, or water bodies.

[0005] To address these issues, researchers have proposed various improvement methods. For example, by introducing attention mechanisms, models can focus more on important regions in images, thereby improving the feature extraction of images of buildings, roads, or water bodies. Attention mechanisms mainly include two forms: spatial attention and channel attention. Spatial attention mechanisms can capture local details in images, such as edges and corners, thus improving the accuracy of semantic segmentation; channel attention mechanisms can highlight important features of different channels, such as color and texture, thereby enhancing the discriminative power of features. However, existing attention mechanisms typically only focus on local or global information, making it difficult to comprehensively utilize both local and global information, resulting in incomplete feature extraction.

[0006] Furthermore, the task of semantic segmentation of ground features in aircraft remote sensing images also faces the problem of robustness in complex environments. For example, interference factors such as shadows and cloud cover can affect image quality, thereby reducing the accuracy of semantic segmentation. Traditional methods often struggle to effectively handle these interferences, leading to performance degradation of models in complex environments.

[0007] Therefore, how to extract multi-scale features and maintain high semantic segmentation performance of ground features in complex environments is a problem that needs to be solved by those skilled in the art in the semantic segmentation of aircraft remote sensing images. Summary of the Invention

[0008] In view of this, the present invention provides a method and system for semantic segmentation of ground features based on local-global remote sensing images, which introduces spatial attention and channel attention mechanisms to more accurately capture local details and important features in the images, thereby improving the accuracy and robustness of semantic segmentation of remote sensing images.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention proposes a method for semantic segmentation of ground features based on local-global remote sensing images, comprising the following steps:

[0011] S1. Obtain the remote sensing image of the target region to be segmented, perform preprocessing, and extract features through a convolutional neural network to obtain the feature map F;

[0012] S2. Based on the global information of the feature map F, channel attention mechanism and spatial attention mechanism are used to obtain global enhanced features;

[0013] S3. Divide the feature map F into multiple feature vectors, apply the channel attention mechanism and spatial attention mechanism to each feature vector and reintegrate them to obtain complete local enhancement features;

[0014] S4. Fuse the global enhancement feature and the complete local enhancement feature to obtain the enhancement feature F';

[0015] S5. Input the enhanced feature F' into the land feature semantic segmentation model to obtain the land feature semantic segmentation result P; the land feature semantic segmentation result P includes buildings, roads and water bodies.

[0016] Further, step S1 involves acquiring the remote sensing image of the target region to be segmented, preprocessing it, and extracting features using a convolutional neural network to obtain a feature map F; specifically including:

[0017] The remote sensing image of the target region to be segmented is acquired, the image is denoised, the denoised image is enhanced, and then the image is normalized.

[0018] The normalized image is segmented, and the background part of the segmented image is removed. The remaining segmented image is used as the original image.

[0019] The original image is input into an initial convolutional neural network to obtain low-level features;

[0020] The low-level features are processed by nonlinear transformation, pooling layer and batch normalization layer to obtain feature F0;

[0021] For the feature F 0 Multi-level feature extraction is performed to obtain high-level features, which are used as feature map F; wherein the shape of feature map F is C×H×W, where C is the number of channels, H is the height, and W is the width.

[0022] Further, step S2 involves obtaining globally enhanced features based on the global information of the feature map F, using channel attention and spatial attention mechanisms; specifically including:

[0023] A channel attention mechanism is employed, in which average pooling and max pooling are performed on each channel of the feature map F, and the concatenation results in a global channel attention map; this map is then multiplied with the global matrix of the feature map F to obtain the enhanced features of the global channels.

[0024] A spatial attention mechanism is used to perform average pooling and max pooling on the enhanced features of the global channels in the channel dimension, and the resulting concatenation yields a global spatial attention map; this map is then multiplied with the enhanced features of the global channels to obtain the global enhanced features.

[0025] Furthermore, the channel attention mechanism is employed to perform average pooling and max pooling on each channel of the feature map F, specifically including:

[0026] Adaptive average pooling is used to compress the feature map of each channel into a single average value, resulting in a channel average pooling tensor of shape C×1×1.

[0027] Adaptive max pooling is used to compress the feature map of each channel into a single maximum value, resulting in a channel max pooling tensor of shape C×1×1.

[0028] The channel average pooling tensor and channel maximum pooling tensor are sequentially subjected to convolution, activation, and re-convolution operations to concatenate and compress the number of channels.

[0029] Furthermore, the spatial attention mechanism is employed to perform average pooling and max pooling on the enhanced features of the global channels along the channel dimension, specifically including:

[0030] The enhanced features of the global channel are average pooled in the channel dimension to obtain a spatial average pooling tensor with shape 1×H×W.

[0031] Meanwhile, max pooling is performed in the channel dimension to obtain a spatial max pooling tensor with shape 1×H×W;

[0032] The spatial average pooling tensor and the spatial max pooling tensor are sequentially subjected to convolution, activation, and re-convolution operations to concatenate and compress the number of channels.

[0033] Further, step S3 involves segmenting the feature map F into multiple feature vectors, applying the channel attention mechanism and spatial attention mechanism to each feature vector, and then re-integrating them to obtain complete local enhancement features; specifically including:

[0034] The feature map F is segmented into multiple feature vectors f along the height and width directions. i , where i is a positive integer greater than 1;

[0035] The channel attention mechanism is used to process the feature vector f. i Each channel is subjected to average pooling and max pooling, and the concatenation results in the channel weight vector w. ci ; and the feature vector f i The matrix multiplication yields the enhanced features of different channels;

[0036] Using the spatial attention mechanism, the enhancement features of different channels are subjected to average pooling and max pooling in the channel dimension, and the concatenation is used to obtain a spatial weight vector; then multiplied with the enhancement features of different channels to obtain local enhancement features;

[0037] Based on the spatial location encoding information during the segmentation of the feature map F, the local enhancement features are re-integrated to obtain complete local enhancement features.

[0038] Further, step S5 involves inputting the enhanced feature F' into the land cover semantic segmentation model to obtain the land cover semantic segmentation result P; specifically including:

[0039] The enhanced feature F' is input into the ground cover semantic segmentation model, and a global average pooling operation is performed on each feature map of the enhanced feature F' to obtain a feature map compressed into a single average value.

[0040] The compressed feature map is flattened to obtain the corresponding one-dimensional vector.

[0041] The one-dimensional vector is input into a fully connected layer, where linear transformation and nonlinear activation are performed. Then, the Softmax activation function is applied to obtain the final semantic segmentation result P of the ground features.

[0042] Furthermore, step S5 also includes:

[0043] Cross-entropy loss is used to evaluate the difference between semantic segmentation results and real labels, and is used as the first loss function.

[0044] The global channel attention map, global spatial attention map, channel weight vector, and spatial weight vector are subjected to L2 regularization loss as the second loss function.

[0045] The overlap between the predicted region and the real label region is calculated by using the Dice coefficient, and the probability is thresholded to obtain the Dice loss as the third loss function.

[0046] The first loss function, the second loss function, and the third loss function are combined to obtain a composite loss function, which optimizes the semantic segmentation model of ground features.

[0047] Secondly, this invention proposes a semantic segmentation system for ground features based on local-global remote sensing images, comprising the following modules:

[0048] Acquisition module: used to acquire the remote sensing image of the target region to be segmented, perform preprocessing, and extract features through a convolutional neural network to obtain feature map F;

[0049] Global information-guided feature enhancement module: used to obtain global enhanced features based on the global information of the feature map F, using channel attention mechanism and spatial attention mechanism;

[0050] Local information-guided feature enhancement module: used to segment the feature map F into multiple feature vectors, apply the channel attention mechanism and spatial attention mechanism to each feature vector and then re-integrate them to obtain complete local enhanced features;

[0051] Enhanced Feature Fusion Module: Used to fuse global enhanced features and complete local enhanced features to obtain enhanced feature F';

[0052] The semantic segmentation module for ground features is used to input the enhanced features F' into the semantic segmentation model to obtain the semantic segmentation result P. The semantic segmentation result P includes buildings, roads, and water bodies.

[0053] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for semantic segmentation of ground features based on local-global remote sensing images, which has the following beneficial effects:

[0054] 1) By introducing spatial attention and channel attention mechanisms, this invention can more accurately capture local details and important features of buildings, roads and water bodies in images, thereby improving the accuracy and robustness of semantic segmentation of buildings, roads and water bodies in aircraft remote sensing images.

[0055] 2) By fusing local and global information, the module can maintain high semantic segmentation performance in complex environments and improve the model's generalization ability.

[0056] 3) Finally, by feature fusion, feature information at different scales and levels is comprehensively utilized to further improve the overall performance of semantic segmentation models for buildings, roads and water bodies. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0058] Figure 1 This is a flowchart of a method for semantic segmentation of ground features based on local-global remote sensing images, provided in an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of a module for local-global information-guided feature enhancement provided in an embodiment of the present invention.

[0060] Figure 3 This is a diagram of the neural network model architecture provided in an embodiment of the present invention.

[0061] Figure 4 This is a framework diagram of a land cover semantic segmentation system based on local-global remote sensing images, provided for an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Example 1

[0064] This invention discloses a method for semantic segmentation of ground features based on local-global remote sensing images, referring to... Figure 1 As shown, it includes the following steps S1 to S5:

[0065] S1. Obtain the remote sensing image of the target region to be segmented, perform preprocessing, and extract features through a convolutional neural network to obtain the feature map F;

[0066] S2. Based on the global information of the feature map F, channel attention mechanism and spatial attention mechanism are used to obtain global enhanced features;

[0067] S3. Divide the feature map F into multiple feature vectors, apply the channel attention mechanism and spatial attention mechanism to each feature vector and reintegrate them to obtain complete local enhancement features;

[0068] S4. Fuse the global enhancement feature and the complete local enhancement feature to obtain the enhancement feature F';

[0069] S5. Input the enhanced feature F' into the land feature semantic segmentation model to obtain the land feature semantic segmentation result P; the land feature semantic segmentation result P includes buildings, roads and water bodies.

[0070] In this embodiment of the invention, according to the above steps, a remote sensing image of the city is first obtained, such as the image including clouds, buildings, roads, water bodies, vegetation, vehicles and pedestrians, etc., and then image processing and feature extraction are performed to obtain feature map F.

[0071] Secondly, refer to Figure 2As shown, by introducing spatial attention (SA) and channel attention (CA) mechanisms, the module more accurately captures local details and important features in the feature map F. This enables the ground object semantic segmentation model to more accurately identify and semantically segment various targets in aircraft remote sensing images, such as buildings, roads, and water bodies. By fusing local and global information, the module can maintain high semantic segmentation performance in complex environments. For example, common interference factors in remote sensing images, such as shadows and cloud cover, can be mitigated by the module through multi-scale feature learning and attention mechanisms, thereby improving the model's robustness. The local attention mechanism can accurately capture the boundary information of targets, which is particularly important for the identification of small-scale targets (such as small buildings and vehicles) in remote sensing images. By refining the target boundaries, the module can improve the semantic segmentation model's ability to identify small-scale targets. By comprehensively utilizing local and global information, the module can improve the model's generalization ability, enabling it to maintain high semantic segmentation performance across different scenes and datasets. This is of great significance for the diversity and complexity of semantic segmentation tasks in remote sensing images.

[0072] After performing semantic segmentation of land features using the method of this invention, each pixel can be assigned to its respective land feature category. Through semantic segmentation, an image labeled with various land feature categories can be obtained, where each pixel is given a category label. This fine-level classification can provide basic data for subsequent quantitative analysis, such as calculating the proportion of various land uses, estimating building area, and assessing urban expansion trends. In addition, high-quality semantic segmentation results are also a key support for intelligent decision-making in multiple fields such as precision agriculture, environmental protection, and disaster response.

[0073] The steps described above are explained in detail below:

[0074] Step S1 specifically includes:

[0075] Acquire remote sensing images of the city, including images of clouds, buildings, roads, water bodies, vegetation, vehicles, and pedestrians. Perform denoising, enhancement, and normalization processing on the remote sensing images to improve image quality and reduce noise interference.

[0076] Regarding denoising, methods such as median filtering are used to denoise the input aircraft remote sensing images to reduce the impact of noise on subsequent processing. Denoising improves image clarity and reduces noise interference, thereby improving the accuracy of feature extraction.

[0077] Image enhancement is performed on the denoised image. Histogram equalization and contrast adjustment are used to enhance the image's contrast and brightness. Image enhancement can highlight important details in the image, making feature extraction more effective. Finally, image normalization is performed to normalize the image's pixel values ​​to a fixed range.

[0078] Next, the image is separated into multiple regions of interest through image segmentation, and background noise is removed.

[0079] Image segmentation uses methods such as thresholding, edge detection, or region growing to segment a normalized image into multiple regions of interest (ROIs). Image segmentation can separate different target regions in an image, facilitating subsequent feature extraction and semantic segmentation.

[0080] Background removal can be achieved by removing background components from an image using methods such as background modeling or background subtraction, while preserving the region of interest. Background removal can reduce the impact of background noise on feature extraction and improve the accuracy of feature extraction.

[0081] Furthermore, refer to Figure 3 As shown, a convolutional neural network (CNN) is used for initial feature extraction. Through convolution, activation, pooling and batch normalization operations, high-level features in the image are extracted step by step.

[0082] First, the original image I is input into the initial convolutional layer of the convolutional neural network (CNN). In this embodiment, the initial convolutional layer uses a small kernel (3x3) to capture low-level features in the image, such as edges and textures.

[0083] Next, after the convolution operation, an activation function (such as ReLU) is applied to perform a non-linear transformation on the convolution result to introduce non-linear characteristics. Then, after the activation function, a pooling layer (such as max pooling or average pooling) is applied to downsample the feature map to reduce its size while preserving important features. After the pooling layer, a batch normalization layer can be introduced to normalize the feature map, thereby accelerating the training process and improving the model's stability, resulting in feature F. 0 :

[0084]

[0085] Where Conv represents convolution operation, ReLU represents modified linear unit activation function, Pool represents pooling operation, and BatchNormalization represents batch normalization layer.

[0086] Finally, multi-level feature extraction is performed by repeating the above convolution, activation, pooling, and batch normalization operations to construct a multi-level convolutional neural network and gradually extract high-level features F from the image.

[0087] Step S2 specifically includes:

[0088] First, the channel attention mechanism and the spatial attention mechanism are described based on the input feature map F.

[0089] (1) Regarding the channel attention mechanism, the input feature map F is a feature map with shape C×H×W; where C represents the number of channels, H is the height, and W is the width.

[0090] First, adaptive average pooling is used to compress the feature map of each channel into a single average value, resulting in a tensor of shape C×1×1. Simultaneously, adaptive max pooling is used to compress the feature map of each channel into a single maximum value, resulting in a tensor of shape C×1×1.

[0091] Next, both of the above tensors The number of channels is compressed using 1x1 convolutions, which reduces the computational cost. After convolution, a ReLU activation function is applied to increase non-linearity. Subsequently, another 1x1 convolution is used to restore the number of channels to their original scale, resulting in the final channel attention feature map.

[0092] Finally, and Summing along the channel dimension yields... It integrates feature information from average pooling and max pooling. Then, a sigmoid activation function is applied to map the output to the range [0,1], resulting in a channel attention map of shape C×1×1. This attention map represents the importance weight of each channel.

[0093] (2) Regarding the spatial attention mechanism, the input feature map F is a feature map with shape C×H×W; where C represents the number of channels, H is the height, and W is the width.

[0094] First, the input feature map F is average pooled along the channel dimension (dim=1) to obtain a tensor of shape 1×H×W. It contains the channel mean at each location; simultaneously, max pooling is performed on the input feature map F in the channel dimension (dim=1) to obtain a tensor of shape 1×H×W. It contains the maximum value for each position.

[0095] Next, the results of average pooling and max pooling are concatenated. and By concatenating along the channel dimension (dim=1), a tensor of shape 2×H×W is obtained. It contains average pooling and max pooling information for each spatial location. Then, a convolution operation is performed, using a convolutional layer to process the concatenated feature map, with a kernel size of 7. This convolutional layer compresses the number of input channels from 2 to 1, outputting a tensor of shape 1×H×W.

[0096] Finally, the Sigmoid activation function is applied to map the output to the range [0,1], representing the attention weight at each spatial location. The output is an attention map of shape 1×H×W. Used for weighted input feature maps.

[0097] In this embodiment, a channel attention mechanism is used to perform average pooling and max pooling on each channel of the feature map F, and the resulting concatenation yields a global channel attention map. Multiplying the global matrix of the feature map F yields the enhanced features of the global channels.

[0098] Global information of feature map F Should The features are processed by a channel attention mechanism, with each channel undergoing average pooling and max pooling to obtain... Channel attention mechanisms can highlight important features from different channels, such as color and texture, thereby enhancing feature discrimination. This is particularly important for semantic segmentation of complex scenes in remote sensing images. Then, channel attention is applied to features that guide the global information branch. Obtain the enhanced features of the channel

[0099]

[0100] in, This represents the matrix multiplication operation. By applying channel attention weights, important channel information in the global features can be highlighted, thereby improving the expressive power of the features.

[0101] Next, After applying spatial attention mechanisms, average pooling and max pooling are performed along the channel dimension, and the resulting concatenation yields the global spatial attention map. Then spatial attention is applied to the features The enhanced features obtained after global information-guided feature enhancement are as follows:

[0102]

[0103] in, This represents matrix multiplication. By comprehensively utilizing channel attention and spatial attention, the module can improve the model's generalization ability, enabling it to maintain high semantic segmentation performance across different scenarios and datasets. This is of great significance for addressing the diversity and complexity of semantic segmentation tasks in remote sensing images.

[0104] Step S3 specifically includes:

[0105] The feature map F is divided into n equally sized feature vectors f along the height H and width W directions. i , where i = 1, 2, 3...n; by segmenting the feature map, local features can be processed more meticulously, thereby improving the ability to recognize small-scale targets.

[0106] Next, the channel weight vectors are obtained, and each feature vector f is learned through the channel attention mechanism. i For the eigenvector f i Each channel is subjected to average pooling and max pooling, and the concatenation results in the channel weight vector w. ci .

[0107] Applying channel attention to each small feature block results in...

[0108]

[0109] in, This represents a matrix multiplication operation. Channel weight vectors can highlight important features from different channels, thereby enhancing the discriminative power of features.

[0110] Then, through the By applying spatial attention, average pooling and max pooling are performed along the channel dimension, and the concatenation results in a spatial weight vector. Then to Feature enhancement is obtained

[0111]

[0112] in, This represents a matrix multiplication operation. Such operations can capture local details in an image, thereby improving the accuracy of semantic segmentation.

[0113] Finally, based on the spatial location encoding information during feature map segmentation, all small feature blocks are... Reintegration yields complete local information-guided feature enhancement features.

[0114] Step S4, global augmentation features and complete local enhancement features The fusion is performed to obtain the enhanced feature F';

[0115]

[0116] F' is the final result of the fusion.

[0117] By fusing global and local features, feature information at different scales and levels can be comprehensively utilized, thereby improving the overall performance of the semantic segmentation model. (Fused features) It can more comprehensively express various feature information in an image, including global structure and local details. This comprehensive feature representation capability can significantly improve the adaptability of semantic segmentation models to complex scenes, thereby improving the accuracy and robustness of semantic segmentation.

[0118] Step S5 specifically includes:

[0119] The enhanced feature F' is input into the ground cover semantic segmentation model. This embodiment refers to... Figure 3 As shown, firstly, the system enters a global average pooling layer, where global average pooling is performed on each feature map, compressing each feature map into a single average value. Then, the globally average pooled feature maps are flattened, converting them into one-dimensional vectors. The flattened feature vectors are then input into a fully connected layer for linear transformation and non-linear activation, yielding the ground feature semantic segmentation result.

[0120] After the fully connected layer, the Softmax activation function is applied to convert the output into a probability distribution, yielding the final semantic segmentation result P:

[0121]

[0122] Where GlobalAvgPool represents the global average pooling operation, Flatten represents the flattening operation, FC represents the fully connected layer, Softmax represents the Softmax activation function, and P is the final semantic segmentation probability distribution.

[0123] Step S5 also includes constructing a composite loss function to optimize the semantic segmentation model of ground features.

[0124] In this embodiment, cross-entropy loss is first used as the semantic segmentation loss. Cross-entropy loss can measure the difference between the semantic segmentation result and the real label, and guide the training process of the model.

[0125] Semantic segmentation loss is used as the first loss function; as shown below:

[0126]

[0127] Among them, yj These are the actual labels input to the model. is the predicted probability of the model, and N is the number of samples.

[0128] Next, L2 regularization loss is applied to the weights generated by channel attention and spatial attention. L2 regularization loss can prevent the model from overfitting and improve the model's generalization ability.

[0129] L2 regularization loss is used as the second loss function; as shown below:

[0130]

[0131] Where λ is the regularization coefficient and Γ is all the weights of the model.

[0132] In this embodiment, the output probability represents the probability that the model predicts each pixel belongs to a certain class (usually the foreground class), ranging from [0,1]. The ground truth label represents the actual label, usually 0 or 1, representing the background or foreground, respectively.

[0133] Foreground refers to objects or regions of interest in an image. In this embodiment, the foreground may include buildings, vegetation, vehicles, and pedestrians. Simply put, the foreground is the main object to be identified and extracted.

[0134] Background refers to all other elements besides the foreground. In this embodiment, it may include clouds, roads, water bodies, etc.; it represents areas that are not directly involved in the main analysis or processing.

[0135] This embodiment also obtains a third loss function through probability thresholding; since the Dice coefficient calculates the overlap between the predicted region and the true label region, the output probability value needs to be converted into a binary prediction result during calculation. If the output probability p is greater than a certain threshold (e.g., 0.5), then the pixel is considered to belong to the foreground (1); otherwise, it belongs to the background (0).

[0136] Dice loss is used as the third loss function; as shown below:

[0137]

[0138] By combining the three loss functions mentioned above, a composite loss function can be constructed:

[0139]

[0140] Here, α is a weighting coefficient used to balance the contributions of different loss functions. A composite loss function can comprehensively consider the contributions of multiple loss functions, improving the semantic segmentation performance and robustness of the model.

[0141] Example 2

[0142] This invention discloses a semantic segmentation system for ground features based on local-global remote sensing images, referring to... Figure 4 As shown, it includes the following modules:

[0143] Acquisition module: used to acquire the remote sensing image of the target region to be segmented, perform preprocessing, and extract features through a convolutional neural network to obtain feature map F;

[0144] Global information-guided feature enhancement module: used to obtain globally enhanced features based on the global information of the feature map F, employing channel attention and spatial attention mechanisms.

[0145] Local information-guided feature enhancement module: This module segments the feature map F into multiple feature vectors, applies channel attention and spatial attention mechanisms to each feature vector, and then reassembles them to obtain complete local enhancement features.

[0146] Enhanced Feature Fusion Module: Used to fuse global enhanced features and complete local enhancement features The fusion is performed to obtain the enhanced feature F';

[0147] The semantic segmentation module for ground features is used to input the enhanced features F' into the semantic segmentation model to obtain the semantic segmentation result P. The semantic segmentation result P includes buildings, roads, and water bodies.

[0148] In this embodiment, the acquisition module acquires remote sensing images according to the method in Embodiment 1, performs preprocessing, and extracts features through a convolutional neural network to obtain a feature map F with shape C×H×W.

[0149] The features input into the global information-guided feature enhancement module are: Based on the global information of feature map F, channel attention and spatial attention mechanisms are used to obtain globally enhanced features.

[0150] The feature map F is divided into multiple feature vectors f. i Where i = 1, 2, 3...n, the input is fed into the local information-guided feature enhancement module, where channel attention and spatial attention mechanisms are applied to each feature vector and then re-integrated to obtain complete local enhanced features.

[0151] The global enhanced features are integrated through the enhanced feature fusion module. and complete local enhancement features The enhanced features F' are fused together and input into the land cover semantic segmentation module for prediction, resulting in the land cover semantic segmentation result P.

[0152] This invention, by introducing spatial attention and channel attention mechanisms, can more accurately capture local details and important features in images. This enables the semantic segmentation model to more accurately identify and semantically segment various targets in aircraft remote sensing images, such as buildings, roads, and bodies of water.

[0153] By fusing local and global information, the module can maintain high semantic segmentation performance in complex environments. For example, in remote sensing images, the module can reduce the impact of interference factors such as shadows and cloud cover through multi-scale feature learning and attention mechanisms, thereby improving the robustness of the model.

[0154] This invention improves the generalization ability of the model by comprehensively utilizing local and global information, enabling it to maintain high semantic segmentation performance across different scenarios and datasets. This is significant for addressing the diversity and complexity of semantic segmentation tasks in remote sensing images. Furthermore, by fusing features and comprehensively utilizing feature information at different scales and levels, the overall performance of the semantic segmentation model can be further enhanced. The method of this invention not only improves the accuracy of semantic segmentation but also reduces the resource requirements for model training and deployment through efficient parameter fine-tuning and the use of adapters, making it suitable for remote sensing image processing tasks in various computing environments.

[0155] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0156] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for semantic segmentation of ground features based on local-global remote sensing images, characterized in that, Includes the following steps: S1. Obtain the remote sensing image of the target region to be segmented, perform preprocessing, and extract features through a convolutional neural network to obtain the feature map F; S2. Based on the global information of the feature map F, channel attention mechanism and spatial attention mechanism are used to obtain global enhanced features; S3. Divide the feature map F into multiple feature vectors, apply the channel attention mechanism and spatial attention mechanism to each feature vector and reintegrate them to obtain complete local enhancement features; S4. Fuse the global enhancement feature and the complete local enhancement feature to obtain the enhancement feature F'; S5. Input the enhanced feature F' into the ground feature semantic segmentation model to obtain the ground feature semantic segmentation result P; The semantic segmentation result P of the land features includes buildings, roads, and water bodies; in, Step S1 specifically includes: The remote sensing image of the target region to be segmented is acquired, the image is denoised, the denoised image is enhanced, and then the image is normalized. The normalized image is segmented, and the background part of the segmented image is removed. The remaining segmented image is used as the original image. The original image is input into an initial convolutional neural network to obtain low-level features; The low-level features are processed by nonlinear transformation, pooling layer, and batch normalization layer to obtain feature F. 0 ; For the feature F 0 Multi-level feature extraction is performed to obtain high-level features, which are used as feature map F; wherein, the shape of feature map F is C×H×W, where C is the number of channels, H is the height, and W is the width; Step S2 specifically includes: A channel attention mechanism is employed, in which average pooling and max pooling are performed on each channel of the feature map F, and the concatenation results in a global channel attention map; this map is then multiplied with the global matrix of the feature map F to obtain the enhanced features of the global channels. A spatial attention mechanism is used to perform average pooling and max pooling on the enhanced features of the global channels in the channel dimension, and the resulting concatenation yields a global spatial attention map; this map is then multiplied with the enhanced features of the global channels to obtain the global enhanced features.

2. The method for semantic segmentation of ground features based on local-global remote sensing images as described in claim 1, characterized in that, The channel attention mechanism is employed to perform average pooling and max pooling on each channel of the feature map F, specifically including: Adaptive average pooling is used to compress the feature map of each channel into a single average value, resulting in a channel average pooling tensor of shape C×1×1. Adaptive max pooling is used to compress the feature map of each channel into a single maximum value, resulting in a channel max pooling tensor of shape C×1×1. The channel average pooling tensor and channel maximum pooling tensor are sequentially subjected to convolution, activation, and re-convolution operations to concatenate and compress the number of channels.

3. The method for semantic segmentation of ground features based on local-global remote sensing images as described in claim 2, characterized in that, The spatial attention mechanism is employed to perform average pooling and max pooling on the enhanced features of the global channels along the channel dimension, specifically including: The enhanced features of the global channel are average pooled in the channel dimension to obtain a spatial average pooling tensor with shape 1×H×W. Meanwhile, max pooling is performed in the channel dimension to obtain a spatial max pooling tensor with shape 1×H×W; The spatial average pooling tensor and the spatial max pooling tensor are sequentially subjected to convolution, activation, and re-convolution operations to concatenate and compress the number of channels.

4. The method for semantic segmentation of ground features based on local-global remote sensing images as described in claim 3, characterized in that, Step S3: Divide the feature map F into multiple feature vectors, apply the channel attention mechanism and spatial attention mechanism to each feature vector and reintegrate them to obtain complete local enhancement features; Specifically, it includes: The feature map F is segmented into multiple feature vectors f along the height and width directions. i , where i is a positive integer greater than 1; The channel attention mechanism is used to process the feature vector f. i Each channel is subjected to average pooling and max pooling, and the concatenation results in the channel weight vector w. ci ; and the feature vector f i The matrix multiplication yields the enhanced features of different channels; Using the spatial attention mechanism, the enhancement features of different channels are subjected to average pooling and max pooling in the channel dimension, and the concatenation is used to obtain a spatial weight vector; then multiplied with the enhancement features of different channels to obtain local enhancement features; Based on the spatial location encoding information during the segmentation of the feature map F, the local enhancement features are re-integrated to obtain complete local enhancement features.

5. The method for semantic segmentation of ground features based on local-global remote sensing images as described in claim 4, characterized in that, Step S5: Input the enhanced feature F' into the land cover semantic segmentation model to obtain the land cover semantic segmentation result P; specifically including: The enhanced feature F' is input into the ground cover semantic segmentation model, and a global average pooling operation is performed on each feature map of the enhanced feature F' to obtain a feature map compressed into a single average value. The compressed feature map is flattened to obtain the corresponding one-dimensional vector. The one-dimensional vector is input into a fully connected layer, where linear transformation and nonlinear activation are performed. Then, the Softmax activation function is applied to obtain the final semantic segmentation result P of the ground features.

6. The method for semantic segmentation of ground features based on local-global remote sensing images as described in claim 5, characterized in that, Step S5 also includes: Cross-entropy loss is used to evaluate the difference between semantic segmentation results and real labels, and is used as the first loss function. The global channel attention map, global spatial attention map, channel weight vector, and spatial weight vector are subjected to L2 regularization loss as the second loss function. The overlap between the predicted region and the real label region is calculated by using the Dice coefficient, and the probability is thresholded to obtain the Dice loss as the third loss function. The first loss function, the second loss function, and the third loss function are combined to obtain a composite loss function, which optimizes the semantic segmentation model of ground features.

7. A semantic segmentation system for ground features based on local-global remote sensing images, characterized in that, Includes the following modules: Acquisition module: used to acquire the remote sensing image of the target region to be segmented, perform preprocessing, and extract features through a convolutional neural network to obtain feature map F; Global information-guided feature enhancement module: used to obtain global enhanced features based on the global information of the feature map F, using channel attention mechanism and spatial attention mechanism; Local information-guided feature enhancement module: used to segment the feature map F into multiple feature vectors, apply channel attention mechanism and spatial attention mechanism to each feature vector and then re-integrate them to obtain complete local enhanced features; Enhanced Feature Fusion Module: Used to fuse the global enhanced features and the complete local enhanced features to obtain the enhanced feature F'; The semantic segmentation module for ground features is used to input the enhanced features F' into the semantic segmentation model to obtain the semantic segmentation result P; the semantic segmentation result P includes buildings, roads, and water bodies. in, The acquisition module is specifically used for: The remote sensing image of the target region to be segmented is acquired, the image is denoised, the denoised image is enhanced, and then the image is normalized. The normalized image is segmented, and the background part of the segmented image is removed. The remaining segmented image is used as the original image. The original image is input into an initial convolutional neural network to obtain low-level features; The low-level features are processed by nonlinear transformation, pooling layer, and batch normalization layer to obtain feature F. 0 ; For the feature F 0 Multi-level feature extraction is performed to obtain high-level features, which are used as feature map F; wherein, the shape of feature map F is C×H×W, where C is the number of channels, H is the height, and W is the width; The global information-guided feature enhancement module is specifically used for: A channel attention mechanism is employed, in which average pooling and max pooling are performed on each channel of the feature map F, and the concatenation results in a global channel attention map; this map is then multiplied with the global matrix of the feature map F to obtain the enhanced features of the global channels. A spatial attention mechanism is used to perform average pooling and max pooling on the enhanced features of the global channels in the channel dimension, and the resulting concatenation yields a global spatial attention map; this map is then multiplied with the enhanced features of the global channels to obtain the global enhanced features.

Images