Typical ground object segmentation method based on multi-source data attention feature fusion

Abstract

The present invention discloses a typical ground object target segmentation method based on multi-source data attention feature fusion, comprising the following steps: (1) obtaining a data set including a high-resolution optical image, a SAR image, and a corresponding semantically labeled image, and performing preprocessing; (2) constructing a typical ground object target segmentation model based on multi-source data attention feature fusion to achieve semantic segmentation of typical ground object targets; (3) using the preprocessed data set obtained in step (1) as input to the model constructed in step (2), training the model based on a weighted cross entropy loss function, and achieving typical ground object target segmentation based on multi-source data. The present invention comprehensively considers the observation advantages of SAR and optical data for different ground object targets, and proposes an adaptive fusion algorithm based on an attention mechanism to better combine the respective advantageous features of optical images and SAR images, thereby achieving better semantic segmentation effect in the field of ground object classification.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing and ground object target segmentation and extraction, and specifically relates to a typical ground object target segmentation method based on multi-source data attention feature fusion. Background Art

[0002] Land object classification typically uses semantic segmentation, which assigns a category to each label in remote sensing images. Remote sensing images include optical images and synthetic aperture radar (SAR) images. Land object classification based on remote sensing images relies on the spatial and semantic information provided by the images. Research results can be applied to fields such as land resource planning, mineral exploration, environmental protection, and urban planning. Synthetic aperture radar (SAR), as an active microwave imaging sensor, can observe and image the observation area in all weather conditions and at all times of the day. It is unaffected by external environmental factors such as lighting conditions and climate change, while possessing strong penetration capabilities. SAR images provide rich electromagnetic scattering characteristics and geometric features of land objects. However, SAR acquires grayscale data in a single wavelength band, which can lead to drawbacks such as blurred target details. Optical imaging has the advantage of providing high spatial resolution and rich spectral and texture information, but it is susceptible to observational environmental factors such as cloud, fog, and rain. Because optical and SAR imaging demonstrate excellent complementarity in some practical applications, optical-SAR fusion holds great promise for improving remote sensing target recognition accuracy.

[0003] Deep learning-based optical SAR fusion methods are categorized into decision-level fusion and feature-level fusion. Decision-level fusion uses various methods to acquire semantic features of interest, but struggles to extract high-dimensional feature representations after fusion. Feature-level fusion methods include the Pseudo-Siamese Convolutional Neural Network (PSCNN) and Multisource Remote Sensing Data Classification (MRSDC). PSCNN, a network architecture proposed by Hughes et al. in 2018, can extract corresponding image slices from high-resolution optical and SAR imagery, achieving high accuracy in complex remote sensing data containing numerous buildings. MRSDC developed an architecture based on a dual-tunnel convolutional neural network, utilizing dual-tunnel branches and a cascaded network to achieve classification fusion of hyperspectral imagery and other multi-sensor data. However, these methods fail to fully exploit the distributional characteristics of multi-source data and exhibit limited performance stability. Therefore, a more stable ground object segmentation method with guaranteed classification accuracy is needed. Summary of the Invention

[0004] Purpose of the invention: Based on the semantic consistency constraints of multi-source data, the purpose of the present invention is to provide a typical land object segmentation method based on the fusion of multi-source data attention features, which is used in the field of land object classification and can obtain good semantic segmentation results.

[0005] Technical solution: The present invention provides a typical ground object segmentation method based on multi-source data attention feature fusion, which specifically includes the following steps:

[0006] (1) Obtain a dataset containing high-resolution optical images, SAR images, and corresponding semantically labeled images, divide the dataset into training, validation, and test sets, and perform data preprocessing;

[0007] (2) Constructing a typical ground object target segmentation model based on multi-source data attention feature fusion to achieve semantic segmentation of typical ground object targets; the model includes an encoder and a decoder; the encoder includes a data prior feature construction and guidance module, a feature extraction backbone network and a multimodal cross-attention fusion module;

[0008] (3) The preprocessed dataset obtained in step (1) is used as the input of the model constructed in step (2), and the model parameters are trained and optimized based on the weighted cross entropy loss function to achieve typical ground object segmentation based on multi-source data.

[0009] Furthermore, the data preprocessing process in step (1) is as follows:

[0010] First, the dataset is cut into non-overlapping pixel blocks of 256×256 pixels. Then the images are standardized, the mean is set to [0.485, 0.456, 0.406], and the standard deviation is set to [0.229, 0.224, 0.225]. Finally, data augmentation is performed on the training set images.

[0011] Furthermore, the implementation process of the typical ground object target segmentation model based on multi-source data attention feature fusion in step (2) is as follows:

[0012] First, the data prior feature construction and guidance module extracts the texture features of optical images and the scattering features of SAR images as the neural network prior information for the dedicated feature extraction and second-order hidden feature mining of optical and SAR images. Then, an encoder with the same structure is used to extract the optical image feature vector and the SAR image feature vector, and their corresponding query set feature vector Q, index set feature vector K, and content feature vector V. Secondly, the multimodal cross-attention fusion module calculates the optical image attention feature vector A. OPT =Attention(Q OPT ,K SAR ,V SAR), and the SAR image attention feature vector A SAR =Attention(Q SAR ,K OPT ,V OPT ), and then concatenate to obtain the multimodal cross attention feature vector A cros =concat(A OPT ,A SAR ); Further, the optical image feature vector obtained by the serial encoder, the SAR image feature vector, and the multimodal cross attention feature vector A cros , as the model decoder input to obtain the model prediction results.

[0013] Furthermore, the feature extraction backbone network adopts ResNet50 to perform feature extraction on the input of optical images and SAR images respectively to obtain representative and effective multi-level features of the two types of images; the 2nd and 5th layer features extracted by the feature extraction backbone network are selected as the low-dimensional features and high-dimensional features of the image respectively, the high-dimensional features of the optical image and SAR image are used as the input of the high-dimensional cross-attention feature fusion module, and the low-dimensional features of the two are used as the input of the low-dimensional cross-attention feature fusion module, and finally the cross-attention feature vectors output by the high- and low-dimensional modules are connected in series to obtain multi-dimensional optical SAR cross-attention fusion features; the multi-dimensional optical SAR cross-attention fusion features are input into the decoder, and the final semantic segmentation result is obtained after convolution and upsampling operations.

[0014] Furthermore, the feature extraction backbone network in step (2) extracts features from the image, performs two-dimensional convolution operations on the cut optical image and SAR image using a 3×3 convolution kernel, pads the edges of the input image by 1 pixel, and normalizes the data of each channel by batch normalization; the result passes through the maximum pooling layer to achieve downsampling of the image data; after the above processing twice, two types of 64×64 pixel blocks are obtained as low-dimensional features as input of the inter-domain fusion module; after the above operation three times, two types of 32×32 pixel blocks are obtained, which no longer pass through the maximum pooling layer, and after two convolution and batch normalization operations, two types of 32×32 pixel blocks with a channel number of 2048 are obtained as high-dimensional features as input of the inter-domain fusion module.

[0015] Furthermore, the multimodal cross attention fusion module in step (2) is implemented as follows:

[0016] The high-dimensional features and low-dimensional features are used as input respectively. The cross-attention mechanism can simultaneously consider the feature data information of the optical and SAR domains and calculate their attention weights. First, the query vector Q, key vector K, and value vector V are obtained by inputting the matrix X. The formula is as follows:

[0017] {Q,K,V}={XW Q ,XW K ,XW V}

[0018] Then multiply the Q and K matrices and divide by And after normalization, the formula is as follows:

[0019]

[0020] Where Q, K, and V are matrices obtained after linear mapping of the input matrix, d k is the K matrix dimension, B is the learnable relative position code, and Softmax is the activation function that converts the input real number vector into a probability distribution vector;

[0021] The query vector Q, key vector K, and value vector V from different sources are cross-calculated for correlation to obtain the weighted value of the attention mechanism. The calculation formula of the multimodal cross attention fusion module is as follows:

[0022]

[0023] Z1=LN(MCA(Q1,K2,V2))+Q1

[0024] Z2=LN(MCA(Q2,K1,V1))+Q2

[0025] Where X1 is the embedded representation of the optical input; X2 is the embedded representation of the SAR input; are the weight matrices corresponding to the query vectors Q1 and Q2, key vectors K1 and K2, and value vectors V1 and V2 for the optical input and SAR input, respectively; MCA represents the multi-head cross attention module; LN represents layer regularization, which normalizes all features of a single sample;

[0026] Convolve Z1 and Z2 to obtain the high-dimensional features and low-dimensional features after optical SAR fusion. First, the low-dimensional features and the prior features are concatenated to obtain the concatenated low-dimensional features. Then, the high-dimensional features and the prior features are concatenated and the concatenated high-dimensional features are obtained through atrous spatial pyramid pooling (ASPP). Finally, the concatenated high- and low-dimensional features are connected in series as the input of the decoder.

[0027] Furthermore, the atrous spatial pyramid pooling (ASPP) method first uses atrous convolution branches with different dilation rates to capture feature information at different scales, thereby increasing the receptive field and improving the feature extraction capability without increasing network parameters. The feature map of each branch is then subjected to spatial pyramid pooling to obtain a feature vector of a fixed size. Finally, the feature vectors of each scale are concatenated and the final ASPP feature representation is obtained after dimensionality reduction.

[0028] Furthermore, the decoder in step (2) includes two 1×1 convolutions and one upsampling.

[0029] Furthermore, the weighted cross entropy loss function in step (3) is used to train and optimize the model parameters, which are calculated as follows:

[0030]

[0031] Where x is the input, y is the target, w is the weight, C is the number of categories, and N is the minimum block dimension.

[0032] Beneficial effects: Compared with the existing technology, the beneficial effects of the present invention are: the present invention proposes a multimodal cross-attention feature fusion architecture, which has the function of fusing multi-domain feature information and can effectively suppress redundant information interference; the present invention proposes to use optical image texture features and SAR image scattering features as network optimization prior information, which can provide richer image detail feature information for subsequent deep learning and ground object segmentation, and improve the accuracy of ground object segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the structure of a typical ground object segmentation model based on multi-source data attention feature fusion constructed by the present invention;

[0034] Figure 2 Schematic diagram of the structure of the multimodal cross-attention fusion module proposed in this invention;

[0035] Figure 3 This is the calculation flow chart of the multi-head cross attention mechanism proposed in this invention;

[0036] Figure 4 Examples of optical images, SAR images, and semantically labeled images used in the experiments of the present invention.

[0037] Figure 5 This is the segmentation result diagram of 7 typical ground objects using the algorithm proposed in this invention. DETAILED DESCRIPTION

[0038] The present invention will be further described in detail below with reference to the accompanying drawings.

[0039] This paper proposes a typical ground object segmentation method based on multi-source data attention feature fusion. It comprehensively considers the observation advantages of SAR and optical data for different ground objects and designs an adaptive fusion algorithm, which is then applied to the semantic segmentation task of typical ground objects. Specifically, it includes the following steps:

[0040] Step 1: Obtain a dataset containing high-resolution optical images, SAR images, and corresponding semantically labeled images. Divide the dataset into training, validation, and test sets. First, perform sub-graph cutting, then perform data augmentation on the training set images to obtain the preprocessed dataset.

[0041] Based on the WHU-OPT-SAR dataset, which contains pixel-level optical and SAR images of the same area, the present invention first cuts the dataset into non-overlapping pixel blocks of 256×256 pixels. Then, the images are normalized with the mean set to [0.485, 0.456, 0.406] and the standard deviation set to [0.229, 0.224, 0.225]. Finally, data augmentation is performed on the training dataset.

[0042] Step 2: Construct a typical ground object segmentation model based on multi-source data attention feature fusion, such as Figure 1 As shown in the figure, it consists of two main parts: encoder and decoder. The encoder part includes a data prior feature construction and guidance module, a feature extraction backbone network and a multimodal cross attention fusion module, while the decoder part includes an upsampling interpretation module.

[0043] Among them, the data prior feature construction and guidance module is used to extract the texture features of optical images and the scattering features of SAR images as the prior information of the neural network, which can realize the special feature extraction and second-order hidden feature mining of optical images and SAR images; then, an encoder with the same structure is used to extract the optical image feature vector and the SAR image feature vector, and their corresponding query set feature vector Q, index set feature vector K, and content feature vector V; secondly, a multimodal cross-attention fusion module is constructed to calculate the optical image attention feature vector A respectively. OPT =Attention(Q OPT ,K SAR ,V SAR ), and the SAR image attention feature vector A SAR =Attention(Q SAR ,K OPT ,V OPT ), and then concatenate to obtain the multimodal cross attention feature vector A cros =concat(A OPT ,A SAR ); Further, the optical image feature vector obtained by the serial encoder, the SAR image feature vector, and the multimodal cross attention feature vector A cros , as the model decoder input to obtain the model prediction results.

[0044] The feature extraction backbone network adopts ResNet50, which performs feature extraction on the input of optical images and SAR images respectively to obtain representative and effective multi-level features of the two types of images; in the multimodal cross-attention fusion module, the query set feature vector Q, the index set feature vector K, and the content feature vector V are cross-calculated to obtain fusion features with semantic consistency; the 2nd and 5th layer features extracted by the feature extraction backbone network are selected as the low-dimensional features and high-dimensional features of the image respectively, the high-dimensional features of the optical image and SAR image are used as the input of the high-dimensional multimodal cross-attention feature fusion module, and the low-dimensional features of the two are used as the input of the low-dimensional multimodal cross-attention feature fusion module. Finally, the cross-attention feature vectors output by the high- and low-dimensional modules are connected in series to obtain the multi-dimensional optical SAR cross-attention fusion feature; the multi-dimensional optical SAR cross-attention fusion feature is input into the decoder, and the final semantic segmentation result is obtained after convolution and upsampling operations.

[0045] First, feature extraction is performed on the image using a feature extraction backbone network. A two-dimensional convolution operation is performed on the cropped optical image and SAR image using a 3×3 convolution kernel. One pixel is padded around the edges of the input image, and batch normalization (BN) is used to normalize the data for each channel. The result is passed through a maximum pooling layer to achieve downsampling of the image data. After two rounds of this process, two types of 64×64 pixel blocks are obtained as low-dimensional features, which serve as the input to the inter-domain fusion module. After three rounds of this process, two types of 32×32 pixel blocks are obtained, which are no longer passed through the maximum pooling layer. After two rounds of convolution and batch normalization, two types of 32×32 pixel blocks with 2048 channels are obtained as high-dimensional features, which serve as the input to the multimodal cross-attention fusion module.

[0046] The BN layer normalizes each batch of data in each hidden layer of the neural network so that the data of each feature dimension is in a standard normal distribution, which helps solve problems such as gradient disappearance and gradient explosion, and helps accelerate the convergence of the model; the maximum pooling layer can reduce the size of the feature map and obtain its most significant features.

[0047] The structure of the multimodal cross attention fusion module is as follows: Figure 2 As shown, the purpose is to achieve the fusion of optical image features and SAR image features.

[0048] The high-dimensional features and low-dimensional features obtained in the previous step are used as inputs respectively. The cross-attention mechanism can simultaneously consider the feature data information of the optical and SAR domains and calculate their attention weights. First, the query vector Q (Query), key vector K (Key), and value vector V (Value) are obtained by inputting the matrix X. The formula is as follows:

[0049] {Q,K,V}={XW Q ,XW K ,XW V}

[0050] Then multiply the Q and K matrices and divide by To avoid the problem of too small gradient after normalization by the Softmax function, the normalization formula is as follows:

[0051]

[0052] Where Q, K, and V are matrices obtained after linear mapping of the input matrix, and d k is the K matrix dimension, B is the learnable relative position code, and Softmax is the activation function that converts the input real number vector into a probability distribution vector.

[0053] Based on the above attention mechanism principle, in order to better integrate feature data from different sources, we consider cross-calculating the correlation between query vectors Q, key vectors K, and value vectors V from different sources to obtain the weighted value of the attention mechanism. The calculation formula of the multimodal cross attention fusion module is as follows:

[0054]

[0055] Z1=LN(MCA(Q1,K2,V2))+Q1,

[0056] Z2=LN(MCA(Q2,K1,V1))+Q2,

[0057] In the formula, MCA represents the multi-head cross attention module, and its calculation process is as follows Figure 3 As shown; LN represents layer regularization, which normalizes all features of a single sample.

[0058] Furthermore, Z1 is convolved with Z2 to obtain the high-dimensional features and low-dimensional features after optical SAR fusion. Figure 1 As shown in the figure, low-dimensional features are first concatenated with prior features to obtain concatenated low-dimensional features. High-dimensional features are then concatenated with prior features and then subjected to atrous spatial pyramid pooling (ASPP) to obtain concatenated high-dimensional features. Finally, the concatenated high- and low-dimensional features are connected in series as the decoder input. The ASPP module first employs atrous convolution branches with different dilation rates to capture feature information at different scales, increasing the receptive field without increasing network parameters and improving feature extraction capabilities. Spatial pyramid pooling is then performed on the feature maps of each branch to obtain a fixed-size feature vector. Finally, the feature vectors at each scale are concatenated and, after dimensionality reduction, the final ASPP feature representation is obtained.

[0059] The decoder structure is as follows Figure 1 As shown in Figure 2, the output of the encoder is used as the input of the decoder, and after two 1×1 convolutions and one upsampling, the image is reconstructed to obtain the final semantic segmentation result.

[0060] Step 3: Use the preprocessed dataset obtained in step 1 as the input of the model constructed in step 2, train and optimize the model parameters based on the weighted cross entropy loss function, and realize the segmentation of typical ground objects based on multi-source data.

[0061] This paper uses the WHU-OPT-SAR dataset provided by Wuhan University as the experimental dataset. This dataset contains 100 optical remote sensing images and SAR remote sensing images with a size of 5556×3704 pixels, covering an area of ​​approximately 50,000 square kilometers in Hubei Province. Figure 4 The following diagrams show example SAR images, optical images, and ground truth labels from the experiment, covering seven main categories: farmland, city, village, water, forest, road, and others. This region has a subtropical monsoon climate, with a minimum altitude of approximately 50 meters and a maximum altitude of approximately 3,000 meters. The region boasts a rich variety of terrain and vegetation, effectively simulating the ground object segmentation task in real-world application scenarios.

[0062] The WHU-OPT-SAR dataset was divided into training, validation, and test sets. The model was trained using the Adam optimizer with hyperparameters β1 and β2 set to 0.900 and 0.999, respectively, for calculating the moving average of momentum and squared gradient. The weighted cross entropy loss function was used, calculated as follows:

[0063]

[0064] Where x is the input, y is the target, w is the weight, C is the number of categories, and N is the minimum block dimension.

[0065] The present invention uses accuracy, single category accuracy, category average accuracy, single category intersection-over-union ratio, average intersection-over-union ratio and kappa coefficient as indicators to measure the segmentation effect.

[0066] The WHU-OPT-SAR dataset is taken as an example to verify the present invention. Figure 5 This is the segmentation result of the typical ground object segmentation method proposed in this invention on the WHU-OPT-SAR dataset for a total of 7 types of targets, including farmland, city, village, water body, forest, road and others.

[0067] Table 1. Object segmentation accuracy based on the WHU-OPT-SAR dataset

[0068]

[0069]

[0070] Table 2 Kappa values ​​for ground feature segmentation based on the WHU-OPT-SAR dataset

[0071] Model farmland City village water bodies forest the way other Kappa DeepLabv3+ 0.685 0.600 0.501 0.417 0.111 0.265 0.039 0.519 UNet 0.694 0.616 0.488 0.415 0.146 0.267 0.042 0.524 MCANet 0.715 0.597 0.516 0.457 0.149 0.337 0.039 0.552 The present invention 0.717 0.589 0.512 0.470 0.218 0.341 0.044 0.567

[0072] Tables 1 and 2 compare the accuracy and Kappa coefficient of the proposed object segmentation method with those of the DeepLabv3+, Unet, and MCANet algorithms for seven typical object predictions. Experimental results demonstrate that compared to traditional semantic segmentation algorithms, the proposed method achieves an average 1.83% improvement in accuracy and a 3.53% improvement in Kappa coefficient, demonstrating superior segmentation performance.

[0073] The technical solutions provided by the embodiments of the present invention are described in detail above. Specific examples are used in this patent to illustrate the principles and implementation methods of the embodiments of the present invention. The above embodiments and features in the embodiments can be combined with each other unless there is a conflict. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A typical ground object segmentation method based on multi-source data attention feature fusion, characterized by: The following steps are involved: (1) Obtain a dataset containing high-resolution optical images, SAR images, and corresponding semantically labeled images, divide the dataset into training, validation, and test sets, and perform data preprocessing; (2) Constructing a typical ground object target segmentation model based on multi-source data attention feature fusion to achieve semantic segmentation of typical ground object targets; the model includes an encoder and a decoder; the encoder includes a data prior feature construction and guidance module, a feature extraction backbone network and a multimodal cross-attention fusion module; (3) Using the preprocessed dataset obtained in step (1) as the input of the model constructed in step (2), training and optimizing the model parameters based on the weighted cross entropy loss function to achieve typical ground object segmentation based on multi-source data; The implementation process of the multimodal cross attention fusion module in step (2) is as follows: The feature extraction backbone network extracts features from optical images and SAR images respectively, and obtains high-dimensional features and low-dimensional features of the two types of images as input. The cross-attention mechanism can simultaneously consider the feature data information of the optical and SAR domains and calculate their attention weights. First, the query vector Q, key vector K, and value vector V are obtained by inputting the matrix X. The formula is as follows: {Q,K,V}={XW Q ,XW K ,XW V } Then multiply the Q and K matrices and divide by And after normalization, the formula is as follows: Where Q, K, and V are matrices obtained after linear mapping of the input matrix, d k is the K matrix dimension, B is the learnable relative position code, and Softmax is the activation function that converts the input real number vector into a probability distribution vector; The query vector Q, key vector K, and value vector V from different sources are cross-calculated for correlation to obtain the weighted value of the attention mechanism. The calculation formula of the multimodal cross attention fusion module is as follows: Z1=LN(MCA(Q1,K2,V2))+Q1 Z2=LN(MCA(Q2,K1,V1))+Q2 Where X1 is the embedded representation of the optical input; X2 is the embedded representation of the SAR input; are the weight matrices of query vectors Q1 and Q2, key vectors K1 and K2, and value vectors V1 and V2 corresponding to optical and SAR inputs, respectively; MCA represents the multi-head cross attention module; LN stands for layer normalization, which normalizes all features of a single sample; Convolve Z1 and Z2 to obtain the high-dimensional features and low-dimensional features after optical SAR fusion. First, the low-dimensional features and the prior features are concatenated to obtain the concatenated low-dimensional features. Then, the high-dimensional features and the prior features are concatenated and the concatenated high-dimensional features are obtained through atrous spatial pyramid pooling (ASPP). Finally, the concatenated high- and low-dimensional features are connected in series as the input of the decoder.

2. A typical ground object segmentation method based on multi-source data attention feature fusion according to claim 1, characterized in that: The data preprocessing process in step (1) is as follows: First, the dataset is cut into non-overlapping pixel blocks of 256×256 pixels. Then the images are standardized, the mean is set to [0.485, 0.456, 0.406], and the standard deviation is set to [0.229, 0.224, 0.225]. Finally, data augmentation is performed on the training set images.

3. The typical ground object segmentation method based on multi-source data attention feature fusion according to claim 1 is characterized in that: The implementation process of the typical ground object target segmentation model based on multi-source data attention feature fusion in step (2) is as follows: First, the data prior feature construction and guidance module extracts optical image texture features and SAR image scattering features as neural network prior information for dedicated feature extraction and second-order hidden feature mining of optical and SAR images. Then, an encoder with the same structure is used to extract the optical image feature vector and the SAR image feature vector, as well as their corresponding query set feature vector Q, index set feature vector K, and content feature vector V. Secondly, the multimodal cross attention fusion module calculates the optical image attention feature vector A OPT =Attention(Q OPT ,K SAR ,V SAR ), and the SAR image attention feature vector A SAR =Attention(Q SAR ,K OPT ,V OPT ), and then concatenate to obtain the multimodal cross attention feature vector A cros =concat(A OPT ,A SAR ); Further, the optical image feature vector obtained by the serial encoder, the SAR image feature vector, and the multimodal cross attention feature vector A cros , as the model decoder input to obtain the model prediction results.

4. The typical ground object segmentation method based on multi-source data attention feature fusion according to claim 1 is characterized in that: The feature extraction backbone network uses ResNet50 to extract features from the input of optical images and SAR images respectively to obtain representative and effective multi-level features of the two types of images; The second-layer and fifth-layer features extracted by the feature extraction backbone network are selected as the low-dimensional and high-dimensional features of the image, respectively. The high-dimensional features of the optical image and SAR image are used as the input of the high-dimensional cross-attention feature fusion module, and the low-dimensional features of the two are used as the input of the low-dimensional cross-attention feature fusion module. Finally, the cross-attention feature vectors output by the high- and low-dimensional modules are concatenated to obtain the multi-dimensional optical SAR cross-attention fusion feature; the multi-dimensional optical SAR cross-attention fusion feature is input into the decoder, and the final semantic segmentation result is obtained after convolution and upsampling operations.

5. The typical ground object segmentation method based on multi-source data attention feature fusion according to claim 1 is characterized in that: In step (2), the feature extraction backbone network extracts features from the image, performs two-dimensional convolution operations on the cut optical image and SAR image using a 3×3 convolution kernel, pads the edges of the input image by 1 pixel, and normalizes the data of each channel by batch normalization; the result passes through the maximum pooling layer to achieve downsampling of the image data; after the above processing twice, two types of 64×64 pixel blocks are obtained as low-dimensional features as input of the inter-domain fusion module; after the above operation three times, two types of 32×32 pixel blocks are obtained, which no longer pass through the maximum pooling layer, and after two convolution and batch normalization operations, two types of 32×32 pixel blocks with a channel number of 2048 are obtained as high-dimensional features as input of the inter-domain fusion module.

6. The typical ground object segmentation method based on multi-source data attention feature fusion according to claim 1 is characterized in that: The atrous spatial pyramid pooling (ASPP) method first uses atrous convolution branches with different dilation rates to capture feature information at different scales, thereby increasing the receptive field and improving feature extraction capabilities without increasing network parameters. The feature map of each branch is then subjected to spatial pyramid pooling to obtain a feature vector of a fixed size. Finally, the feature vectors of each scale are concatenated and dimensionality reduced to obtain the final ASPP feature representation.

7. The typical ground object segmentation method based on multi-source data attention feature fusion according to claim 1 is characterized in that: The decoder in step (2) includes two 1×1 convolutions and one upsampling.

Images