Remote sensing image segmentation method based on matrix decomposition to enhance global features

Abstract

The present invention proposes a remote sensing image segmentation method based on matrix decomposition to enhance global features. First, the codec structure convolutional neural network model framework is improved according to a pre-set strategy, and the improved residual neural network is used as a comprehensive feature extraction network to improve the feature extraction ability of the model; the improved model is trained according to a pre-set training scheme to obtain a high-resolution remote sensing image semantic segmentation model, and finally the trained model is loaded on a PC. The PC receives the high-resolution remote sensing image that needs to be segmented, and the image is cropped to a fixed size and loaded into the model. The model realizes rapid segmentation of the input image. The present invention realizes rapid segmentation of ground objects in remote sensing images, and the model is relatively lightweight and easy to build. It can be easily deployed in conjunction with a mobile terminal or embedded device, and can simultaneously perform real-time high-precision segmentation of objects of different sizes.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision semantic segmentation technology, and in particular relates to a remote sensing image segmentation method based on matrix decomposition to enhance global features. Background Art

[0002] Image semantic segmentation is another fundamental task in computer vision, alongside classification and detection. Compared to image classification and detection, segmentation is a more complex task because it requires classifying each pixel. Semantic segmentation of remote sensing images is a major branch of image segmentation, with broad applications in urban planning, urban-rural change detection, and geographic information.

[0003] While traditional remote sensing image semantic segmentation methods are simple and efficient, the development of high-resolution remote sensing imagery has made accurate segmentation a challenging task. High-resolution remote sensing images contain contextual information about roads and objects, scattered objects and backgrounds, complex backgrounds, and complex boundaries. Traditional methods cannot effectively segment objects because they ignore spatial and contextual information. For better remote sensing image segmentation, a deeper understanding of contextual information, such as object category and shape, is required. Deep learning, which can provide a deep understanding of this contextual information, has gradually become a mainstream method for remote sensing image segmentation.

[0004] CNN-based methods have achieved significant success in remote sensing image segmentation. With the recent rise in popularity of Transformers in natural language processing, ViT has successfully adapted Transformers for computer vision. However, Transformer-based architectures are inefficient in acquiring spatial information and computationally expensive, limited by the amount of pre-training data and the limited training time of devices. CNNs constructed entirely from standard ConvNet modules rival Transformers in accuracy, scalability, and robustness across all major benchmarks.

[0005] Matrix decomposition has important applications in data analysis, machine learning, signal processing, and other fields. It can be used for tasks such as data dimensionality reduction and feature extraction. The core idea of matrix decomposition is to use special matrix structures or properties to transform the original matrix into a more manageable form. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a remote sensing image segmentation method based on matrix decomposition to enhance global features, so as to solve the noise problems such as complex background and unclear boundaries of high-resolution remote sensing images, extract deeper contextual information and spatial information, and obtain a higher performance remote sensing image semantic segmentation method with lower consumption.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A remote sensing image segmentation method based on matrix decomposition to enhance global features includes the following steps:

[0009] S1, improved codec structure image feature extraction convolutional neural network model;

[0010] S2, train the improved model;

[0011] S3. Input the image into the trained model to achieve fast segmentation.

[0012] Furthermore, the specific steps of the S1 improved codec structure image feature extraction convolutional neural network model are as follows:

[0013] S11. Add a global multi-scale convolutional network to the encoder and a Hamburger module to the decoder;

[0014] S12: The upsampled fusion of the feature maps of the three stages after the residual neural network structure is sent as output to the dilated spatial convolution pooling module for multi-scale feature aggregation, and the stage 1 containing shallow semantic information is sent to the decoder; the feature layer obtained by the comprehensive feature extraction network in the first stage is used as an effective feature layer, containing high-resolution low-level semantic information, and is sent to the decoder. The decoder fuses the features of the last three stages and sends the obtained deep fusion features to the dilated spatial convolution pooling module for processing;

[0015] S13. In the dilated spatial convolution pooling module, dilated convolution is used for the deep fusion features obtained by the comprehensive feature extraction network in step S12. When the dilation rate increases, branches with dilation rates of 3, 6, and 18 are selected, the branch with dilation rate = 24 is removed, and a 1×1 convolution branch is added. An image pooling branch is added to the dilated spatial convolution pooling module for global pooling. Then, bilinear interpolation is used to make the image pooling branch have the same resolution as other branches. Finally, the five branches are connected and a 1×1 convolution is performed.

[0016] S14. Enter the GMCN module to enhance the expression of global feature information: In the GMCN module, a combination of 1×k+k×1 and k×1+1×k convolutions is used to achieve dense connection of deep fusion features in the k×k region of the feature map;

[0017] S15: After passing through the GMCN module, high-level semantic information features are obtained. After upsampling to the same size as the low-level features obtained in the first stage of the comprehensive feature extraction network, they are passed to the decoder for feature fusion. The Hamburger module is then used to enhance the extraction of global feature information. Finally, pixel classification convolution and smooth upsampling are performed to obtain the final prediction image.

[0018] S16: After S15 analyzes the global features, it is fused with the input and finally the final prediction image is obtained through pixel classification convolution and smooth upsampling.

[0019] The encoder consists of a comprehensive feature extraction network, an ASPP module, and a GMCN module; the decoder consists of an upsampling processing module, a Hamburger module, a 1×1 convolutional layer, a merging of a bilinear interpolation upsampling module and a smoothing classifier.

[0020] Furthermore, the specific steps of S14 entering the GMCN module to enhance the expression of global feature information are as follows:

[0021] The six sets of results obtained by convolving the input image with 1×k and k×1 strip convolution kernels are restored to the input size through a 1×1 convolution channel. The activation function Softmax() is used to normalize the output values to enhance the global weight of the feature information:

[0022]

[0023] in Represents the matrix The value of the node, represents the number of nodes, and e represents a natural constant;

[0024] The attention map obtained after the activation function is used as the input weight coefficient set, and the weighted summation with the input is used to obtain the global attention feature enhancement map:

[0025]

[0026] Among them, Z g represents the output of the attention map, Represents the feature map matrix nodes; the module input needs to be fused with the input image to compensate for the loss of contextual details:

[0027]

[0028] where Z out represents the final output of the GMCN module, Z in represents the input of the GMCN module, are the weights of the attention map and the input map, respectively.

[0029] Furthermore, S15 enhances the extraction of global feature information through the Hamburger module as follows:

[0030] After inputting the convolutional neural network, a tensor is output as is a set of real numbers, C, H, and W are the number of channels in each of the three dimensions. The input tensor is linearly transformed and organized into a matrix X by column. d and n are the number of channels; solving the global information is regarded as a low-rank matrix That is, there is a dictionary matrix and the corresponding encoding The generation process and corresponding decomposition process of X are expressed as: The noise, redundancy and loss of the image are modeled as residual terms Then solve it through optimization algorithm The specific steps are as follows:

[0031] Input the feature attention map matrix X, initialize the decomposed non-negative matrix D, B, and iterate k times until the model converges;

[0032]

[0033]

[0034]

[0035] Represents the row and column positions of the matrix. After the model iterates k times, it outputs a matrix with global semantic attributes. Then through linear transformation we get B T is the transpose of B.

[0036] Furthermore, S2 trains the improved model in the following specific steps:

[0037] S21. Data Preprocessing

[0038] S211. Unify the size of the Potsdam high-resolution remote sensing dataset images, crop the original images and corresponding labels to 512×512 size to accommodate network input, and then normalize all images for the convenience of training;

[0039] S212, using one-hot encoding to perform vectorized encoding for each pixel category of the label;

[0040] S213, using spatial data augmentation methods, including random rotations at different angles and random vertical or horizontal mirror flips;

[0041] S22. Network pre-training

[0042] S221, initializing the network weight;

[0043] S222. Set hyper parameters;

[0044] S223, use cross entropy loss to calculate the error with the label, and then use the stochastic gradient descent algorithm to update the network parameters;

[0045] S224 uses the ImageNet large-scale image dataset to pre-train the feature extraction network and retains the weight coefficients of the best performing network;

[0046] S23. Network training

[0047] Load the pre-trained weights, set 160 training rounds, train the network according to the pre-training strategy, and select the optimal model parameters by evaluating the model's performance on the validation set. A high-resolution remote sensing image semantic segmentation model is obtained. The model input is a 512×512 high-resolution remote sensing cropped image, and the output is a final segmentation map of the same size as the input image.

[0048] Among them, in said S213, the random rotation at different angles includes 90°, 180°, 270°, and 360°.

[0049] Furthermore, S222 sets the hyperparameters as follows: the exponentially decaying dynamic learning rate is initially set to 0.004, the momentum of the weight-delayed stochastic gradient descent method for gradient optimization is set to 0.9, and the weight delay is 0.0001.

[0050] The beneficial effects of the present invention are:

[0051] The model of the present invention solves the problem that the existing Transformer model may have performance degradation when processing high-dimensional data; and the model of the present invention has lower consumption. Using matrix decomposition for feature extraction and solving the global information in the image data will save more computing overhead than the convolution operation using the attention mechanism. The high-resolution remote sensing image semantic segmentation method based on matrix decomposition to enhance global features realizes the rapid segmentation of remote sensing image objects, and the model is relatively lightweight and easy to build. It can be easily deployed in conjunction with mobile terminals or embedded devices, and can simultaneously perform real-time and high-precision segmentation of objects of different sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is the overall network architecture diagram for remote sensing image semantic segmentation in the present invention;

[0054] Figure 2 This is the structural diagram of the GMCN attention mechanism in the present invention;

[0055] Figure 3 It is a schematic diagram of the Hamburger matrix decomposition structure in the present invention. DETAILED DESCRIPTION

[0056] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] S1. Improve the convolutional neural network model framework of the encoder-decoder structure: use the improved residual neural network as the comprehensive feature extraction network to improve the feature extraction ability of the model; add the designed global multi-scale convolutional network to the encoder to improve the model's ability to model global information, and add the Hamburger module to the decoder to further enhance long-distance feature association; thus obtaining the improved model.

[0058] S11, such as Figure 1 As shown in the figure, the model architecture of the present invention adopts an encoder-decoder structure, in which the encoder is composed of a comprehensive feature extraction network, an ASPP module, and a GMCN module; the decoder is composed of an upsampling processing module, a Hamburger module, a 1×1 convolutional layer, a combination of a bilinear interpolation upsampling module, a smoothing classifier, etc.

[0059] S12. The feature extraction network is crucial to the semantic segmentation effect. The present invention will improve ResNet-101 to enhance the feature extraction capability and improve the segmentation accuracy of the model: the upsampling fusion of the feature maps of the last three stages of the residual neural network structure is sent as output to the void space convolution pooling module for multi-scale feature aggregation, and the stage 1 containing shallow semantic information is sent to the decoder to enhance the decoding end's reproduction of contextual information. The feature layer obtained in the first stage of the comprehensive feature extraction network is used as an effective feature layer, containing high-resolution low-level semantic information, and is sent to the decoder. The decoder fuses the features of the last three stages and sends the obtained deep fusion features to the void space convolution pooling module for processing.

[0060] S13. In the dilated spatial convolution pooling module, dilated convolution is used for the deep fusion features obtained by the comprehensive feature extraction network in step S12. When the dilation rate is large, only the middle parameters of the original 9 parameters of the 3×3 convolution are valid. Therefore, when the dilation rate becomes larger, the 3×3 convolution degenerates into a 1×1 convolution. Therefore, dilation rates of 3, 6, and 18 are selected, the branch with dilation rate = 24 is removed, and a 1×1 convolution branch is added. In addition, in order to obtain global information, an image pooling branch is added to perform global pooling, and then bilinear interpolation is performed to the same resolution as other branches. Finally, the five branches are connected and 1×1 convolution is performed (to ensure that the number of channels remains unchanged). Use dilated convolutions with different dilation rates to improve the network's receptive field and enhance feature expression. Then enter the last module of the decoder, the GMCN module, to enhance the expression of global feature information.

[0061] S14, such as Figure 2 In the GMCN module, in order to avoid the huge number of parameters brought by directly using large kernels or global convolution, a combination of 1×k+k×1 and k×1+1×k convolutions is used to achieve dense connections in the k×k region of the feature map, enhancing the extraction of global feature information. The specific steps are as follows:

[0062] After the input image is convolved with 1×k and k×1 strip convolution kernels respectively, the six sets of results are restored to the input size through a 1×1 convolution channel. The output values are normalized using the activation function to enhance the global weight of the feature information:

[0063]

[0064] in Represents the matrix The value of the node, represents the number of nodes, and e represents a natural constant;

[0065] The attention map obtained after the activation function is used as the input weight coefficient set, and the weighted summation with the input is used to obtain the global attention feature enhancement map:

[0066]

[0067] Among them, Z g represents the output of the attention map, X represents the feature attention map matrix, Z represents the input feature map matrix, Represents the feature map matrix nodes; the module input needs to be fused with the input image to compensate for the loss of contextual details:

[0068]

[0069] where Z out represents the final output of the GMCN module, Z in represents the input of the GMCN module, are the weights of the attention map and the input map, respectively.

[0070] S15, such as Figure 1 After passing through GMCN, high-level semantic information features are obtained, which are upsampled to the same size as the low-level features obtained in the first stage of the comprehensive feature extraction network and then passed into the decoder for feature fusion. The extraction of global feature information is then enhanced through the Hamburger module, and finally the final prediction map is obtained through pixel classification convolution and smooth upsampling.

[0071] The specific steps of extracting global feature information through the Hamburger module are as follows:

[0072] After inputting the convolutional neural network, a tensor is output as is a set of real numbers, C, H, and W are the number of channels in each of the three dimensions. Using the idea of matrix decomposition, the input tensor undergoes linear transformation and is organized into a matrix X by column. d and n are the number of channels; solving the global information is regarded as a low-rank matrix That is, there is a dictionary matrix and the corresponding encoding The generation process and corresponding decomposition process of X are expressed as: The noise, redundancy and loss of the image are modeled as residual terms Then solve it through optimization algorithm Based on the selection of a learnable optimization strategy, after time testing and complexity measurement, the present invention chooses to use the non-negative matrix factorization method as the optimization algorithm. The solution steps are as follows:

[0073] Input the feature attention map matrix X, initialize the decomposed non-negative matrix D, B, and iterate k times until the model converges.

[0074]

[0075]

[0076]

[0077] Represents the row and column positions of the matrix. After the model iterates k times, it outputs a matrix with global semantic attributes. Then through linear transformation we get

[0078] S16: After S15 analyzes the global features, it is fused with the input and finally the final prediction image is obtained through pixel classification convolution and smooth upsampling.

[0079] S2. Train the improved model according to the pre-set training plan:

[0080] S21. Data Preprocessing

[0081] S211. Unify the size of the Potsdam high-resolution remote sensing dataset images, crop the original images and corresponding labels to 512×512 size to accommodate network input, and then normalize all images for the convenience of training.

[0082] S212. Use one-hot encoding to vectorize each pixel category of the label.

[0083] S213. Use spatial data augmentation, including random rotations of different angles (90°, 180°, 270°, 360°) and random vertical or horizontal mirror flipping.

[0084] S22. Network pre-training

[0085] S221. Initialize the network weights.

[0086] S222. Set hyperparameters. The exponentially decaying dynamic learning rate is initially set to 0.004. The momentum of the weight-delayed stochastic gradient descent method for gradient optimization is set to 0.9, and the weight delay is 0.0001.

[0087] S223. Use cross entropy loss to calculate the error with the label, and then use the stochastic gradient descent algorithm to update the network parameters.

[0088] S224 uses the ImageNet large-scale image dataset to pre-train the feature extraction network and retains the weight coefficients of the best performing network.

[0089] S23, Network Training Phase: Load pre-trained weights, set 160 training rounds, and train the network according to the pre-training strategy. By evaluating the model's performance on the validation set, the optimal model parameters are selected. Finally, the model is evaluated on the test set. This results in a high-resolution remote sensing image semantic segmentation model. The model input is a 512×512 high-resolution remote sensing cropped image, and the output is a final segmentation map of the same size as the input image.

[0090] S3. The image is cropped to a fixed size and loaded into the trained model, which then quickly segments the input image.

[0091] The present invention provides a remote sensing image segmentation method based on matrix decomposition to enhance global features. Existing techniques ignore spatial and contextual information, making it difficult to deeply understand contextual information such as object category and shape. High-resolution remote sensing images also contain contextual information about roads and objects, as well as scattered objects, complex backgrounds, and complex boundaries. Conventional techniques cannot effectively segment these images. The method provided by the present invention addresses noise issues such as complex backgrounds and unclear boundaries in high-resolution remote sensing images, extracting deeper contextual and spatial information. It is easy to implement, requires minimal computation, and can achieve high-performance semantic segmentation of remote sensing images with low power consumption.

[0092] As shown in Table 1, the performance of the present invention is compared with UNet, ResUNet, PSPN, DeepLabV3+, EncNet, DANet, and EaNet on the Potsdam dataset.

[0093] Table 1 Comparison of segmentation indicators of different network models on the Potsdam dataset

[0094]

[0095] As can be seen from Table 1, the overall accuracy OA and precision F1 of the measurement model of the present invention on the Potsdam dataset are higher than those of the existing technology. This shows that the remote sensing image segmentation method based on matrix decomposition to enhance global features of the present invention has an excellent ability to correctly predict positive samples, and the model accuracy is higher than that of the existing technology; in addition, the average joint cross-pollination of the present invention also performs best on the Potsdam dataset.

[0096] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are described briefly because they are generally similar to the method embodiments. For related portions, reference can be made to the description of the method embodiments.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the scope of protection of the present invention.

Claims

1. A remote sensing image segmentation method based on matrix decomposition to enhance global features, characterized in that: The following steps are involved: S1, improved codec structure image feature extraction convolutional neural network model; The specific steps are as follows: S11. Add a global multi-scale convolutional network to the encoder and a Hamburger module to the decoder; S12: The upsampled fusion of the feature maps of the three stages after the residual neural network structure is sent as output to the dilated spatial convolution pooling module for multi-scale feature aggregation, and the stage 1 containing shallow semantic information is sent to the decoder; the feature layer obtained by the comprehensive feature extraction network in the first stage is used as an effective feature layer, containing high-resolution low-level semantic information, and is sent to the decoder. The decoder fuses the features of the last three stages and sends the obtained deep fusion features to the dilated spatial convolution pooling module for processing; S13. In the dilated space convolution pooling module, dilated convolution is used for the deep fusion features obtained by the comprehensive feature extraction network in step S12. When the dilation rate becomes larger, branches with dilation rates of 3, 6, and 18 are selected, and the branch with dilation rate = 24 is removed, and the branch with dilation rate = 24 is increased. Convolution branch; add image pooling branch to the void space convolution pooling module, perform global pooling, and then use bilinear interpolation to make the image pooling branch have the same resolution as other branches; finally, connect the five branches and do convolution; S14, enter the GMCN module to enhance the expression of global feature information: In the GMCN module, use and The combination of convolution enables deep fusion features in the feature map Achieve dense connectivity within the region; S15: After passing through the GMCN module, high-level semantic information features are obtained. After upsampling to the same size as the low-level features obtained in the first stage of the comprehensive feature extraction network, they are passed to the decoder for feature fusion. The Hamburger module is then used to enhance the extraction of global feature information. Finally, pixel classification convolution and smooth upsampling are performed to obtain the final prediction image. S16, after S15 parsing the global features, it is fused with the input, and finally the final prediction image is obtained through pixel classification convolution and smooth upsampling; S2, train the improved model; S3. Input the image into the trained model to achieve fast segmentation.

2. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 1, characterized in that: The encoder is composed of a comprehensive feature extraction network, an ASPP module, and a GMCN module; the decoder is composed of an upsampling processing module, a Hamburger module, It consists of a convolutional layer, a bilinear interpolation upsampling module, and a smoothing classifier.

3. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 1, characterized in that: The specific steps of S14 entering the GMCN module to enhance the expression of global feature information are as follows: The input images are respectively and The 6 groups of results obtained after the strip convolution kernel performs convolution operation are obtained through The convolution channel is restored to the input size and the activation function is used Normalize the output value to enhance the global weight of feature information: , in Represents the matrix The value of the node, represents the number of nodes, and e represents a natural constant; The attention map obtained after the activation function is used as the input weight coefficient set, and the weighted summation with the input is used to obtain the global attention feature enhancement map: , in, represents the output of the attention map, Represents the feature map matrix nodes; the module input needs to be fused with the input image to compensate for the loss of contextual details: , in represents the final output of the GMCN module, represents the input of the GMCN module, 、 are the weights of the attention map and the input map, respectively.

4. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 1, characterized in that: S15 enhances the extraction of global feature information through the Hamburger module as follows: After inputting the convolutional neural network, a tensor is output as , is a set of real numbers, C, H, and W are the number of channels in each of the three dimensions. The input tensor is linearly transformed and organized into a matrix by column. , , d and n are the number of channels; solving the global information is regarded as a low-rank matrix , that is, there is a dictionary matrix and the corresponding encoding , the generation process of X and the corresponding decomposition process are expressed as: , where the noise, redundancy and loss of the image are modeled as residual terms ; Then solve it through optimization algorithm .

5. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 4, characterized in that: The optimization algorithm solves The details are as follows: Input feature attention map matrix and initialize the decomposed non-negative matrix , , iterate k times until the model converges; , , , Represents the row and column positions of the matrix. After the model iterates k times, it outputs a matrix with global semantic attributes. , and then through linear transformation we get , for The transpose of .

6. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 1, characterized in that: S2 trains the improved model in the following steps: S21. Data Preprocessing S211. Unify the size of the Potsdam high-resolution remote sensing dataset images, crop the original images and corresponding labels to 512×512 size to accommodate network input, and then normalize all images for the convenience of training; S212, using one-hot encoding to perform vectorized encoding for each pixel category of the label; S213, using spatial data augmentation methods, including random rotations at different angles and random vertical or horizontal mirror flips; S22. Network pre-training S221, initializing the network weight; S222. Set hyper parameters; S223, use cross entropy loss to calculate the error with the label, and then use the stochastic gradient descent algorithm to update the network parameters; S224 uses the ImageNet large-scale image dataset to pre-train the feature extraction network and retains the weight coefficients of the best performing network; S23. Network training Load the pre-trained weights, set 160 training rounds, train the network according to the pre-training strategy, and select the optimal model parameters by evaluating the model's performance on the validation set. A high-resolution remote sensing image semantic segmentation model is obtained. The model input is a 512×512 high-resolution remote sensing cropped image, and the output is a final segmentation map of the same size as the input image.

7. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 6, characterized in that: In S213 , the random rotation at different angles includes 90°, 180°, 270°, and 360°.

8. The remote sensing image segmentation method based on matrix decomposition and global feature enhancement according to claim 6, characterized in that: The hyperparameters of S222 are set as follows: the exponentially decayed dynamic learning rate is initially set to 0.004, the momentum of the weight-delayed stochastic gradient descent method for gradient optimization is set to 0.9, and the weight delay is 0.0001.

Images