High-resolution remote sensing image semantic segmentation method based on anisotropic context network

Abstract

The application discloses a high-resolution remote sensing image semantic segmentation method based on an anisotropic context network, comprising: obtaining a high-resolution remote sensing image to be subjected to semantic segmentation, and inputting the high-resolution remote sensing image to an anisotropic context fusion network model for remote sensing image segmentation to perform semantic segmentation and obtain a segmentation map. The application designs an anisotropic context aggregation module, combines a parallel hollow convolution module with a composite strip pooling module, extracts feature vectors of different scales through the parallel hollow convolution, acquires anisotropic spatial context information through a composite strip pooling operation, further captures anisotropic context information of different scales widely existing in the remote sensing image, and performs upsampling on the two extracted feature vectors after splicing to obtain a final segmentation map. The method has the advantages of high segmentation accuracy and fast segmentation speed, can realize semantic segmentation of the remote sensing image, and provides technical support for high-resolution remote sensing image analysis.

Description

Technical Field

[0001] This invention relates to a semantic segmentation method for high-resolution remote sensing images based on anisotropic context networks, belonging to the field of image processing technology. Background Technology

[0002] With the rapid development of drones and aerial photography equipment, the resolution of remote sensing images is becoming increasingly higher (reaching 5-10 cm resolution). Researching semantic segmentation techniques for high-resolution remote sensing images can provide detailed and accurate semantic information for remote sensing analysis, and has broad application prospects in urban planning, environmental protection, and natural landscape monitoring. However, high-resolution remote sensing images contain many complex objects of varying scales on the ground. For example, there are huge buildings and blocks, as well as miniature cars and trees. Furthermore, these segmented targets are complexly and interwoven in arrangement, with large variations in aspect ratio and anisotropic characteristics, making semantic segmentation of high-resolution remote sensing images a challenging problem.

[0003] Spatial pyramid pooling is a widely used semantic network segmentation model that extracts global information through multi-scale pooling. However, the pooling operations used in spatial pyramid pooling support regions with the same length and width scales. This inevitably includes contaminating information from irrelevant regions and makes it difficult to capture anisotropic contextual information, thus affecting segmentation accuracy. Summary of the Invention

[0004] The technical problem this invention aims to solve is to overcome the shortcomings of existing technologies and propose a high-resolution remote sensing image semantic segmentation method based on anisotropic context networks. This method enables accurate semantic segmentation of remote sensing images with complex backgrounds, irregular target shapes, and multiple target categories. To address the anisotropic characteristics of ground target structures in high-resolution remote sensing images, an anisotropic context aggregation module is proposed. This module can capture contextual information at different scales, obtaining more discriminative feature representations and meeting the segmentation requirements for targets of varying sizes and complex distributions in remote sensing images.

[0005] To achieve the above objectives, this invention provides a high-resolution remote sensing image semantic segmentation method based on anisotropic context networks, comprising:

[0006] A high-resolution remote sensing image is input into a pre-built anisotropic context fusion network model for remote sensing image segmentation to obtain a semantic segmentation map of the high-resolution remote sensing image.

[0007] Furthermore, a pre-defined anisotropic context fusion network model for remote sensing image segmentation includes:

[0008] Construct a semantic segmentation network model for high-resolution remote sensing images;

[0009] Acquire high-resolution remote sensing images for use as the training dataset and annotate semantic segmentation maps;

[0010] An anisotropic context network model for high-resolution remote sensing image segmentation was constructed, and a backbone network and anisotropic context aggregation module were designed to capture contextual feature information of targets at different scales.

[0011] Construct an objective function for optimizing the learning of parameters of the anisotropic context network model, and pre-define the training hyperparameters of the anisotropic context network model;

[0012] The training hyperparameters are input into the anisotropic context network model, and the model is updated and iterated through the Adam optimizer until convergence is reached, based on the objective function.

[0013] Furthermore, an anisotropic context network model for high-resolution remote sensing image segmentation is constructed, and a backbone network and anisotropic context aggregation module are designed to capture contextual feature information of targets at different scales, including:

[0014] High-resolution remote sensing images are segmented and fed into a backbone network to extract feature codes and construct feature maps;

[0015] The feature map is input into the anisotropic context aggregation module, where parallel dilated convolution and compound pooling operations are performed, and the output results are concatenated.

[0016] The anisotropic context aggregation module includes a parallel dilated convolution module and a composite pooling module.

[0017] Furthermore, the parallel dilated convolution module includes multiple parallel branches:

[0018] The residual connection branch, the max pooling branch, and three serially superimposed dilated convolution branches;

[0019] The hole convolution ratio is set to a sawtooth structure, with values ​​of 1 to 1, 2 to 1, 2, and 5.

[0020] Furthermore, the composite pooling operation includes:

[0021] Remote context dependencies between different locations can be obtained using a bar pooling model.

[0022] The feature map is input into the horizontal bar pooling path and the vertical bar pooling path, and pooling operations are performed to obtain the feature map after horizontal bar pooling and the feature map after vertical bar pooling, respectively.

[0023] The feature maps obtained by horizontal and vertical bar pooling are upsampled to the input size and then fused to obtain the bar pooling feature map.

[0024] Short-range context dependencies between different locations are obtained using a spatial pyramid pooling model.

[0025] Short-range dependencies are collected using a spatial pyramid pooling model to obtain feature maps of different sizes;

[0026] Spatial pyramid pooling feature maps are obtained by upsampling feature maps of different sizes to restore them to the input size.

[0027] By mapping and concatenating the strip pooling feature map and the spatial pyramid pooling feature map, semantic information about the distribution of anisotropic targets in high-resolution remote sensing images can be obtained.

[0028] Furthermore, the output results are concatenated, including:

[0029] The outputs of the composite strip pooling operation and the parallel convolution operation are concatenated. The concatenated feature map is then upsampled several times through transposed convolution to restore the feature map to the input size, thus obtaining the final semantic segmentation map.

[0030] Furthermore, the objective function includes the cross-entropy loss function:

[0031]

[0032] Where N is the total number of categories; Let y be the feature value of the i-th target category; i Let be the predicted value for the i-th target category; θ is the parameter to be optimized.

[0033] The training hyperparameters of the anisotropic context network model include:

[0034] Model learning rate, number of iterations, training batch size, network model depth and number of layers, and type of activation function.

[0035] Furthermore, the training hyperparameters are input into the anisotropic context network model, and the anisotropic context network model is updated iteratively through the Adam optimizer according to the objective function, including:

[0036] Perform the following operations on all high-resolution remote sensing images in the training dataset, and iterate L times, where L is the number of iterations in the training hyperparameters:

[0037] Initialize each neural network module in the anisotropic context network model;

[0038] High-resolution remote sensing images are input into an anisotropic context network model to obtain the corresponding output vectors;

[0039] Update the network parameters of each neural network module in the anisotropic context network model based on the backpropagation algorithm:

[0040] ω←ω+α·Adam(ω,d ω );

[0041] Where ω represents the network parameters of each neural network module in the anisotropic context network model; α is the model learning rate; Adam is the gradient descent algorithm; d ω This is for gradient updates of network parameters ω.

[0042] Furthermore, the convergence criteria for the anisotropic context network model during update iterations include: if the objective function value decreases and tends to a constant value during update iterations, then the anisotropic context network model is considered to have converged.

[0043] The beneficial effects achieved by this invention are as follows:

[0044] This invention provides a high-resolution remote sensing image semantic segmentation method based on anisotropic context networks. It combines a parallel dilated convolution module with a composite strip pooling module. On one hand, parallel dilated convolution obtains feature vectors at different scales while avoiding loss of information continuity. On the other hand, the composite strip pooling module obtains spatial context information at different scales, capturing anisotropic context information widely present in remote sensing images, especially those with long-range banded structures or discrete distributions. The two feature vectors are then concatenated and upsampled to obtain the final segmentation image. This method offers high segmentation accuracy and fast segmentation speed, enabling semantic segmentation of remote sensing images and providing technical support for high-resolution remote sensing image analysis. Attached Figure Description

[0045] Figure 1 This is a flowchart of the high-resolution remote sensing image semantic segmentation method based on anisotropic context network provided in the embodiments of the present invention;

[0046] Figure 2 This is a schematic diagram of the anisotropic context network model for high-resolution remote sensing image segmentation in the high-resolution remote sensing image semantic segmentation method based on anisotropic context networks provided in this embodiment of the invention.

[0047] Figure 3 This is a schematic diagram of the anisotropic context aggregation module in the high-resolution remote sensing image semantic segmentation method based on anisotropic context networks provided in this embodiment of the invention.

[0048] Figure 4 This is a comparison of the classification results of various algorithms in the high-resolution remote sensing image semantic segmentation method based on anisotropic context networks provided in this embodiment of the invention for high-resolution remote sensing datasets. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0050] This invention provides a high-resolution remote sensing image semantic segmentation method based on anisotropic context networks. This invention combines a dilated convolution module with a composite strip pooling module. On one hand, parallel dilated convolution obtains feature vectors of different scales while avoiding loss of information continuity. On the other hand, the composite strip pooling module obtains spatial context information of different scales, capturing anisotropic context information widely present in remote sensing images, especially those with long-range banded structures or discrete distributions. The two feature vectors are then concatenated and upsampled to obtain the final segmentation map. This method offers high segmentation accuracy and fast segmentation speed, enabling semantic segmentation of remote sensing images and providing technical support for high-resolution remote sensing image analysis.

[0051] like Figures 1 to 4 As shown, this embodiment of the invention provides a high-resolution remote sensing image semantic segmentation method based on anisotropic context networks, comprising the following steps:

[0052] S101. Acquire high-resolution remote sensing images, label specific semantic segmentation maps, and divide them into training datasets and test datasets.

[0053] In this process, annotation software is used to annotate different target categories in high-resolution remote sensing images at the pixel level. For example, in an embodiment of the present invention, the remote sensing images are divided into six categories according to the target category: buildings are annotated in blue, low vegetation in cyan, trees in green, cars in yellow, cement roads in white, and the background in red. Then, the acquired high-resolution remote sensing images are divided into training dataset and test dataset according to a ratio of 0.6:0.4.

[0054] S102. Construct an anisotropic context network model for high-resolution remote sensing image segmentation, such as... Figure 2 As shown.

[0055] Specifically, step S102 includes the following steps:

[0056] Based on the requirements of remote sensing image segmentation, an anisotropic context network model for high-resolution remote sensing image segmentation is constructed. The network is designed with a multi-scale encoding and decoding structure, such as... Figure 2As shown, the segmented high-resolution remote sensing image is fed into a ResNet18 (backbone network) to extract feature information. The ResNet block serves as the basic feature extractor, yielding a feature vector with dimensions of 16×16×512. This feature vector is then fed into a parallel dilated convolution module and a composite strip pooling module, obtaining feature vectors with dimensions of 16×16×512 and 16×16×522, respectively. Finally, the feature vectors obtained from the two branch networks are concatenated and upsampled to the original image size for segmentation.

[0057] The network model expression is y=σ{U{C[P(R(x)),B(R(x))]}}, where R(*) is the feature extraction module, P(*) is the parallel dilated convolution module, B(*) is the composite strip pooling module, C[*] is the concatenate operation, U(*) is the upsampling operation, and σ is the sigmoid activation function.

[0058] S1021. According to the remote sensing image segmentation requirements, an anisotropic context network is constructed. First, the segmented high-resolution remote sensing image is fed into the backbone network for feature extraction and encoding. In this embodiment of the invention, a ResNet18 backbone network is used, consisting of four convolutional blocks (i.e., layer1, layer2, layer3, and layer4). To retain more spatial information, the first max-pooling layer is removed. Then, the obtained feature map is input into the anisotropic context aggregation module.

[0059] S1022, the anisotropic context aggregation module includes a parallel dilated convolution module and a composite pooling module. The feature maps are processed in parallel through parallel dilated convolution operations and composite pooling operations, and the output feature maps are concatenated.

[0060] The parallel dilated convolution module comprises five parallel branches. Two of these branches are residual connections and max pooling, while the dilated convolutions in the remaining three branches are stacked sequentially. The dilation rates are set to a zigzag pattern with values ​​of 1 to 1, 2 to 1, 2, and 5, respectively. This approach avoids the loss of continuity information while allowing convolutions with large receptive fields to extract and generate more abstract features for large targets, whereas convolutions with small receptive fields perform better for small objects.

[0061] Composite pooling can capture contextual information at different scales, obtaining more discriminative feature representations. This module can simultaneously capture short-range and long-range dependencies between different locations. For long-range context, it is collected through horizontal and vertical bar pooling. Bar pooling can not only obtain semantic information about the distribution of anisotropic targets in the scene, but also obtain contextual information about discretely distributed similar targets in the region. Let x∈R C×H×WLet y be the input feature, where C represents the number of channels. First, the feature map x is input into two parallel paths, which are then subjected to 1×W horizontal bar pooling and H×1 vertical bar pooling, respectively, to obtain y. h ∈R C×H and y v ∈R C×W Then, the pooled feature map is upsampled to H×W through a copy operation. The feature maps are then fused to obtain the output y∈R. C×H×W For scenarios with densely distributed semantic regions, a lightweight pyramid pooling approach can be used to collect short-range dependencies. This approach consists of three standard pooling layers of different sizes (2×2, 3×3, 5×5) to generate feature maps of varying sizes. To reduce the number of parameters and computational cost, a 1×1 convolution is used after each pooling layer, reducing the number of channels in each feature map to 2 / N, where N represents the number of channels in the original feature map. Then, upsampling is applied to upsample the feature maps of different sizes to the input feature map size. Finally, the standard pooling features and the bar pooling maps are concatenated to capture the semantic information of anisotropic target distributions widely present in remote sensing images.

[0062] S1023. The outputs of the composite strip pooling module and the parallel dilated convolution module are concatenated. The resulting feature map is then upsampled using transposed convolution. The specific network structure for upsampling is Conv1×1→BN ReLU→ConTranspose3×3→BN ReLU→Conv3×3→BN ReLU. After three upsampling operations, the feature map is restored to the input size to obtain the final semantic segmentation map. All parameters in the anisotropic context network can be adaptively learned end-to-end from the data.

[0063] S103. Design an objective function to optimize the learning of network model parameters and preset the training hyperparameters of the network model.

[0064] The objective function includes the cross-entropy loss function:

[0065]

[0066] Where N is the total number of categories; Let y be the feature value of the i-th target category; i Let be the predicted value for the i-th target category; θ is the parameter to be optimized.

[0067] The training hyperparameters of a network model include the model learning rate α, the number of iterations L, the training batch size S, the depth and number of layers of the network model, and the type of activation function.

[0068] S104. Input the training data into the network model, and use the Adam optimizer to iteratively optimize and update the model parameters according to the objective function.

[0069] Specifically, S104 includes:

[0070] S1041. Initialize the parameters of each neural network module of the network accordingly; select S training images {x} from the training dataset. (1) ,…,x (s) The data is fed into the network model, and the corresponding output vector {y} is obtained. (1) ,…,y (s)};

[0071] S1042. Update the network parameters ω of each neural network module in the anisotropic context network model using the backpropagation algorithm, ω←ω+α·Adam(ω,d) ω ), where Adam is a gradient descent algorithm; d ω Gradient update of network parameter ω

[0072] S1043. For all images in the entire training dataset, perform steps S401 and S402 sequentially, for a total of L iterations.

[0073] S105. If the network model converges, the trained network model can achieve semantic segmentation of high-resolution remote sensing images. Otherwise, return to step S104.

[0074] Specifically, S105 includes:

[0075] S1051. Determine if the network model has converged: During the network training iteration process, if the objective function value decreases and asymptotically approaches a certain value, then the network is determined to have converged; if the iterative training does not converge, then return to step four.

[0076] S1052. Inputting the processed high-resolution remote sensing image data into the converged network model will enable end-to-end semantic segmentation of the high-resolution remote sensing image.

[0077] To verify the effectiveness of the present invention and the proposed parallel dilated convolution module and composite strip pooling module, simulation and ablation experiments were conducted on the present invention. The test column size was 256×256. The model was trained and tested on a remote sensing image training dataset, and the relevant parameters were set as follows: α=0.0001, L=30, S=16.

[0078] The experiment was evaluated using quantitative analytical methods.

[0079] Through experiments on the test set, the final overall pixel accuracy was obtained as 90.8%.

[0080] Ablation experiments were also conducted on the test set to verify the effectiveness of the parallel dilated convolution module and the composite strip pooling module. The ablation experiment results are shown in Table 1, comparing the results of removing the parallel dilated convolution module and the composite strip pooling module, retaining only the backbone network module, and the complete network structure.

[0081] Table 1

[0082]

[0083] As can be seen from Table 1, compared with the backbone network, the parallel dilated convolution module and the composite strip pooling module can effectively improve the segmentation accuracy.

[0084] The effectiveness was verified through comparative experiments on the same high-resolution remote sensing dataset. The anisotropic context network for high-resolution remote sensing image segmentation was compared with several other state-of-the-art deep learning models, including UNet, ResUNet, PSPNet, DeepLabv3+, EncNet, DANet, and Eanet. As shown in Table 2, UNet performed the worst due to a lack of sufficient contextual information. Compared to UNet, EncNet performed better in OA and F1-Score because it fully utilized the strong correlation between scene context and class probabilities. However, EncNet's performance was still slightly lower than DeepLabv3+ because it ignored edge information by using the last layer of the backbone network. DANet introduced a self-attention mechanism that adaptively integrated local features with their global dependencies, thus achieving better OA, F1-Score, and mIoU metrics than EncNet and DeepLabv3+. As analyzed earlier, existing models struggle to capture anisotropic semantic information, easily leading to misclassification. Unlike the comparative models mentioned above, our anisotropic context network for high-resolution remote sensing image segmentation utilizes anisotropic context aggregation module to address this issue. Compared to other models, our model achieves more coherent and accurate segmentation maps. Comparative experimental results are shown in Table 2.

[0085] Table 2

[0086]

[0087] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A high-resolution remote sensing image semantic segmentation method based on anisotropic context networks, characterized in that: Includes the following steps: High-resolution remote sensing images are input into a pre-built anisotropic context fusion network model for remote sensing image segmentation to obtain semantic segmentation maps of high-resolution remote sensing images. Pre-defined anisotropic context fusion network models for remote sensing image segmentation include: Constructing a semantic segmentation model for high-resolution remote sensing images; Acquire high-resolution remote sensing images for use as the training dataset and annotate semantic segmentation maps; An anisotropic context network model for high-resolution remote sensing image segmentation was constructed, and a backbone network and anisotropic context aggregation module were designed to capture contextual feature information of targets at different scales. Construct an objective function for optimizing the learning of parameters of the anisotropic context network model, and pre-define the training hyperparameters of the anisotropic context network model; The training hyperparameters are input into the anisotropic context network model, and the anisotropic context network model is updated and iterated through the Adam optimizer until the convergence state is reached, based on the objective function. An anisotropic context network model for high-resolution remote sensing image segmentation is constructed, and a backbone network and anisotropic context aggregation module are designed to capture contextual feature information of targets at different scales, including: High-resolution remote sensing images are segmented and fed into a backbone network to extract feature codes and construct feature maps; The feature map is input into the anisotropic context aggregation module, where parallel dilated convolution and compound pooling operations are performed, and the output results are concatenated. The anisotropic context aggregation module includes a parallel dilated convolution module and a composite pooling module; The parallel dilated convolution module includes multiple parallel branches: The residual connection branch, the max pooling branch, and three serially superimposed dilated convolution branches; The hole convolution ratio is set to a sawtooth structure, with values ​​of 1 to 1, 2 to 1, 2, and 5 respectively. Composite pooling operations include: Remote context dependencies between different locations can be obtained using a bar pooling model. The feature map is input into the horizontal bar pooling path and the vertical bar pooling path, and pooling operations are performed to obtain the feature map after horizontal bar pooling and the feature map after vertical bar pooling, respectively. The feature maps obtained by horizontal and vertical bar pooling are upsampled to the input size and then fused to obtain the bar pooling feature map. Short-range context dependencies between different locations are obtained using a spatial pyramid pooling model. Short-range dependencies are collected using a spatial pyramid pooling model to obtain feature maps of different sizes; Spatial pyramid pooling feature maps are obtained by upsampling feature maps of different sizes to restore them to the input size. By mapping and concatenating the strip pooling feature map and the spatial pyramid pooling feature map, semantic information about the distribution of anisotropic targets in high-resolution remote sensing images can be obtained.

2. The high-resolution remote sensing image semantic segmentation method based on anisotropic context networks according to claim 1, characterized in that: The output results are concatenated, including: The outputs of the composite strip pooling operation and the parallel convolution operation are concatenated. The concatenated feature map is then upsampled several times through transposed convolution to restore the feature map to the input size, thus obtaining the final semantic segmentation map.

3. The high-resolution remote sensing image semantic segmentation method based on anisotropic context networks according to claim 1, characterized in that: The objective function includes the cross-entropy loss function: ; Where N is the total number of categories; Let y be the feature value of the i-th target category; i Let be the predicted value for the i-th target category; θ is the parameter to be optimized. The training hyperparameters of the anisotropic context network model include: Model learning rate, number of iterations, training batch size, network model depth and number of layers, and type of activation function.

4. The high-resolution remote sensing image semantic segmentation method based on anisotropic context networks according to claim 1, characterized in that: The training hyperparameters are input into the anisotropic context network model, and the anisotropic context network model is updated iteratively through the Adam optimizer according to the objective function, including: Perform the following operations on all high-resolution remote sensing images in the training dataset, and iterate L times, where L is the number of iterations in the training hyperparameters: Initialize each neural network module in the anisotropic context network model; High-resolution remote sensing images are input into an anisotropic context network model to obtain the corresponding output vectors; Update the network parameters of each neural network module in the anisotropic context network model based on the backpropagation algorithm: ω←ω+α·Adam(ω,d ω ); Where ω represents the network parameters of each neural network module in the anisotropic context network model; α represents the model learning rate; Adam represents the gradient descent algorithm; d ω This is for gradient updates of network parameters ω.

5. The high-resolution remote sensing image semantic segmentation method based on anisotropic context networks according to claim 1, characterized in that: The convergence criteria for anisotropic context network models during update iterations include: During the update iteration, if the objective function value decreases and tends to a constant value, the anisotropic context network model is considered to have converged.

Images