Remote sensing image desert shrub extraction method

Abstract

The invention provides a remote sensing image desert shrub extraction method, which is characterized in that an FETNet model is obtained through improvement based on an OfficientNet-B0 baseline model, an original framework predefined in the baseline model is reserved at an encoder part, an EdgeViT module is introduced at a jump connection part of an encoder and a decoder, the global perception capability of the model is enhanced, and the algorithm is simple and easy to implement. An FEM module is introduced into a decoder part to ensure the reservation of space detail information, so that a finally generated desert shrub segmentation result is more accurate and meticulous, a preprocessed remote sensing image is input into an FETNet model, a stochastic gradient descent optimizer, a cosine annealing strategy and a cross entropy loss function are used for model training, the optimal weight of the FETNet model is obtained, and the final desert shrub segmentation result is obtained. The FETNet model is compared with a conventional model, and under the same test condition, all evaluation indexes in the remote sensing image desert shrub segmentation task obtain optimal results.

Description

Technical Field

[0001] The invention relates to the field of remote sensing image recognition, and in particular to a method for extracting desert shrubs from remote sensing images. Background Art

[0002] Desert shrubs play an important role in maintaining the ecological balance of the desert and preventing land desertification. Using remote sensing images for target recognition is an important means to study the distribution of desert shrubs. However, desert shrubs are mostly dense small targets and are complex in morphology. Existing desert shrub information extraction methods still have many limitations when extracting desert shrub information from high-resolution remote sensing images, especially when dealing with complex backgrounds and small-target desert shrubs.

[0003] At present, the schemes for extracting desert shrub information from remote sensing images include threshold methods, pixel-based classification methods, and object-based image analysis methods. The threshold method usually uses spectral indices such as the normalized difference vegetation index to distinguish different vegetation. However, when extracting desert shrub information, complex backgrounds such as bare surface or sand dunes will interfere with and destroy the vegetation features in remote sensing images. The pixel-based classification method analyzes the spectral characteristics of each pixel in the remote sensing image and classifies it to extract desert shrub information, but this method is mainly suitable for low-resolution remote sensing images. The object-based image analysis method uses more spatial, shape, and contextual information to improve classification accuracy. However, the data processing process of obtaining desert shrub information from high-resolution remote sensing images is very complicated. Although the above methods are all effective to a certain extent, they are mainly based on small-area mapping based on UAV platforms. Due to the difficulty and high cost of obtaining UAV data, it is difficult to apply it on a large scale, and it usually relies on manual feature creation, has poor generalization ability, and lacks autonomous learning ability. Summary of the invention

[0004] Purpose of the invention: The purpose of the invention is to provide a method for extracting desert shrubs from remote sensing images with high accuracy and applicable to complex backgrounds.

[0005] Technical solution: A method for extracting desert shrubs from remote sensing images, comprising the following steps: (1) Collect remote sensing images of the desert area, preprocess the remote sensing images, obtain a desert shrub dataset, and divide it into a training set, a validation set, and a test set; (2) Construct a FETNet model. The FETNet model uses the EfficientNet-B0 model as the baseline model. The FETNet model includes an encoder and a decoder. The encoder retains the original architecture predefined by the baseline model. The EdgeViT module is introduced at the jump connection between the encoder and the decoder, and the FEM module is introduced between the decoders to obtain the FETNet model. (3) Train the FETNet model, set the initial learning rate, select the cross entropy loss function and use the stochastic gradient descent method to find the parameter configuration that minimizes the loss function, input the training set into the FETNet model to start training, input the validation set into the FETNet model after each round of training, evaluate the performance of the FETNet model, and record the evaluation indicators and the model weights of each round of training; (4) After reaching the set training rounds, the model weight of the corresponding FETNet model training round with the best evaluation index is selected as the optimal weight, and the optimal weight is applied to the test set to generate the desert shrub segmentation extraction result.

[0006] Specifically, the preprocessing includes: performing atmospheric correction and orthorectification on the remote sensing image, then fusing the panchromatic band and multispectral band of the remote sensing image, cropping the remote sensing image to a set size, marking the desert shrubs and background area on the cropped remote sensing image, and using the marked data to generate a label map of a set size.

[0007] Specifically, the FETNet model includes 7 encoders and 4 decoders. Four EdgeViT modules are introduced at the jump connection between the encoder and the decoder. The EdgeViT module is connected to the encoder output and the corresponding decoder input, and the FEM module is introduced between adjacent decoders.

[0008] Specifically, the cross entropy loss function formula is: , Where: is the cross entropy loss, is the actual label value, is the predicted value of the model; Specifically, the stochastic gradient descent method updates the parameters by calculating the gradient of each sample, and randomly selects a sample or a batch of samples in each update. The formula is: , Where: w is the model parameter, t is the update round, Update the gradient of the model parameters after a sample input, η is the learning rate, and i is the sample index selected in the training set at the current iteration.

[0009] Specifically, step (3) also includes: using the cosine annealing algorithm to update the learning rate of the FETNet model, the formula is as follows: , Where: is the learning rate of the tth training round, is the lower limit of the learning rate, is the upper limit of the learning rate, that is, the initial learning rate, is the current number of training rounds, is the maximum number of training rounds.

[0010] Specifically, the evaluation index includes one or more of pixel accuracy, average pixel accuracy, F1 score, frequency-weighted cross-overlap, and average intersection-over-union.

[0011] Specifically, the average intersection-union ratio formula is: , Where: is the value of the average intersection-union ratio, is a true positive, is a false positive. A false negative is a true negative, is the number of categories.

[0012] Preferably, in step (4), the model weight corresponding to the FETNet model training round with the largest average intersection-over-union ratio value is taken as the optimal weight.

[0013] Preferably, the remote sensing image desert shrub extraction method further comprises the following steps: (5) Using the desert shrub dataset, the FETNet model and several different deep learning models were tested under the same conditions, and the evaluation indicators of the FETNet model and several different deep learning models were compared to verify their effectiveness; the deep learning models include the HRRNet model, DFANet model, DeepLabv3+ model, HRNet model, FCN16s model, ESPNetv2 model, ExtremeC3Net model, FCN8s model and SegNet model.

[0014] Beneficial effects: Compared with the prior art, the significant effects of the present invention are: the FETNet model is improved based on the deep learning baseline model, the model automatically scales and expands the network length, width and resolution, and does not change the original architecture predefined in the baseline network, so as to obtain better model performance, the encoder part adopts EfficientNet-B0 as the backbone network to extract the multi-scale features of desert shrubs, introduces EdgeViT (Transformer module) to synthesize context information, and enhances the global perception ability of the model, and introduces the FEM (feature enhancement) module in the decoder part to ensure the retention of spatial detail information, so that the final generated desert shrub segmentation result is more accurate and detailed, the preprocessed remote sensing image is input into the FETNet model, and the model is trained using a stochastic gradient descent optimizer, a cosine annealing strategy and a cross entropy loss function to obtain the optimal weight of the FETNet model, and the FETNet model proposed in the present invention is compared with the commonly used deep learning model. Under the same test conditions, all evaluation indicators of the FETNet model have achieved optimal results. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of Example 1 of the present invention.

[0016] Figure 2 It is a structural diagram of the FETNet model of Example 1 of the present invention.

[0017] Figure 3 This is the visualization result of desert shrub extraction in Example 1 of the present invention Figure 1 .

[0018] Figure 4 This is a partially enlarged view of the visualization result of desert shrub extraction in Example 1 of the present invention.

[0019] Figure 5 This is the visualization result of desert shrub extraction in Example 1 of the present invention Figure 2 . DETAILED DESCRIPTION

[0020] The present invention is further described below in conjunction with the accompanying drawings.

[0021] Example 1 See also Figure 1 As shown, this embodiment provides a method for extracting desert shrubs from remote sensing images, comprising the following steps: (1) Remote sensing images of desert areas are collected and preprocessed to obtain a desert shrub dataset, which is then divided into a training set, a validation set, and a test set.

[0022] In this embodiment, high-resolution remote sensing images from the GF-2 satellite are used to collect remote sensing image data in a desert area, and the quality of the remote sensing images is ensured to be suitable for the input of the deep learning model.

[0023] The acquired remote sensing images were atmospherically corrected and orthorectified to eliminate atmospheric interference and image deformation, ensuring that the images truly reflected the features of the ground objects; the panchromatic band and the multispectral band were fused to generate high-resolution color images, and suitable test areas were cropped out; pixel-level annotation was performed on the cropped images, and the desert shrub areas and background areas were manually marked to ensure accurate distinction between the desert shrub areas and the background areas. After the annotation was completed, the corresponding label map was generated using the annotated data, and it was cropped to a size that met the input requirements of the deep learning model to form a desert shrub dataset, which was divided into a training set, a validation set, and a test set according to the set ratio.

[0024] (2) Construct a FETNet model. The FETNet model uses the EfficientNeT-B0 model as the baseline model. The FETNet model includes an encoder and a decoder. The encoder retains the original architecture predefined in the baseline model to extract the multi-scale features of desert shrubs. The model automatically scales and expands the network length, width and resolution without changing the original architecture predefined in the baseline network, thereby obtaining better model performance. The EdgeViT module is introduced at the jump connection between the encoder and decoder of the FETNet model to synthesize contextual information and enhance the global perception ability of the model. The decoder part is used to gradually restore the resolution of the image, combining shallow and deep features. At the same time, the FEM module is introduced between the decoders to ensure the retention of spatial detail information, so that the final generated segmentation result is more accurate and detailed, and finally the FETNet model proposed in the present invention is obtained.

[0025] Please refer to Figure 2As shown, in this embodiment, the FETNet model uses the EfficientNet-B0 model as the baseline model, and designs 7 encoder modules (Encoder block) and 4 decoder modules (Decoder block). Four EdgeViT modules are set at the jump connection between the encoder and decoder of the FETNet model; the EdgeViT module is connected to the encoder output, and then connected to the corresponding decoder input to ensure that the feature information can be effectively transmitted during the encoding and decoding process. In addition, the FETNet model also adds 3 layers of FEM modules (feature enhancement modules) between adjacent decoder modules to further enhance the information interaction and feature transmission between different decoder modules. Through this structural design, the FETNet model can effectively integrate the efficient feature extraction capability of the backbone network, and further enhance the transmission of information flow through the EdgeViT module and the FEM module, so that the overall architecture has better feature processing capabilities. The overall structure of the FETNet model is composed of encoders, EdgeViT modules, decoders and FEM modules to form a complete deep learning network architecture.

[0026] The basic module of the EfficientNet-B0 model is the MBConv (Mobile Inverted Bottleneck Convolution) module, which consists of the following parts: Depthwise Separable Convolution, SE (SESqueeze-and-Excitation) module, and Swish activation function. First, use 1x1 convolution to increase the dimension, add an attention mechanism (SENet) after the depthwise separable convolution operation, and finally use 1x1 convolution to reduce the dimension and merge it with the output.

[0027] The EdgeViT module achieves the local-global-local information exchange bottleneck through three key operations. First, local aggregation integrates local information of neighboring markers through deep convolution; second, sparse attention provides a small number of representative markers in the global scope, and promotes long-distance information sharing through the self-attention mechanism; finally, local propagation uses transposed convolution to propagate the learned global contextual information from representative markers to neighboring markers. Therefore, the EdgeViT module can effectively extract small-scale desert shrubs and identify large-scale shrub areas at the same time. Self-focus technology can effectively learn global information and long-distance dependencies, help avoid interference from terrain undulations, diversity of imaging conditions, and similarity between desert shrubs and background features, thereby achieving efficient segmentation of desert shrubs.

[0028] The FEM module (i.e., feature enhancement module) enhances the feature expression ability of small targets through multi-branch convolution and rotation convolution techniques. In the overall structure of the FEM module, there are 4 branches, each of which first performs a 1×1 convolution operation on the input feature image to preliminarily adjust the number of channels in preparation for subsequent processing. The first branch adopts a residual structure to retain the key feature information of small targets; the other three branches perform cascaded standard convolution operations with convolution kernel sizes of 1×3, 3×1, and 3×3, respectively. Additional texture convolution layers are added to the two middle branches so that the extracted feature maps can retain more contextual information, thereby further improving the accuracy of small target detection. In this way, the FEM module can significantly improve the performance of the desert shrub extraction network, especially when dealing with small targets and complex backgrounds. The FEM module can not only extract rich semantic information, but also expand the receptive field, making the model perform better when dealing with densely distributed small-scale desert shrubs.

[0029] (3) Train the FETNet model, set the initial learning rate, select the cross entropy loss function and use the stochastic gradient descent method to find the parameter configuration that minimizes the loss function. Input the training set of the desert shrub dataset into the FETNet model to start training. After each round of training, input the validation set of the desert shrub dataset into the FETNet model to evaluate the performance of the FETNet model and record the evaluation indicators and model weights of each round of training.

[0030] In this embodiment, the PyTorch framework is adopted, and the stochastic gradient descent (SGD), cosine annealing strategy and cross entropy loss function are used for model training.

[0031] In this embodiment, the initial learning rate of the FETNet model is set to 0.001, which is gradually decayed during the training process. In addition, the batch size is set to 2 during training, and the maximum number of iterations is 300.

[0032] Stochastic gradient descent is a gradient-based optimization algorithm used to find the parameter configuration that minimizes the loss function. It updates the parameters by calculating the gradient of each sample and randomly selects one or a batch of samples in each update. The formula is:

[0033] Where: w is the model parameter, t is the update round, Update the gradient of the model parameters after a sample input, η is the learning rate, and i is the sample index selected in the training set at the current iteration.

[0034] The cross entropy loss function compares the model's predicted value with the true value through logarithmic operation, reflecting the quality of the model's prediction in the binary classification problem. The formula is:

[0035] Where: is the cross entropy loss, is the actual label value, is the predicted value of the model.

[0036] The cosine annealing strategy is a method for adjusting the learning rate. During the training process, the learning rate is gradually reduced according to the curve of the cosine function. This can reduce the learning rate in the later stages of training and help the model better converge to the global optimal solution. In the cosine annealing strategy, the learning rate changes as the training progresses according to the following formula:

[0037] Where: is the learning rate of the tth training round, is the lower limit of the learning rate, is the upper limit of the learning rate, that is, the initial learning rate, is the current number of training rounds, is the maximum number of training rounds.

[0038] In each round of training, the performance of the model on the validation set is measured based on five evaluation indicators: pixel accuracy (PA), mean pixel accuracy (MPA), F1 score, frequency-weighted intersection overlap (FWIOU), and mean intersection over union (MIOU). The formula is as follows: , , , , , , , Where: is a true positive, is a false positive. A false negative is a true negative, is the number of categories; is the accuracy of the model, is the model recall rate.

[0039] (4) After reaching the set training rounds, the model weight of the corresponding FETNet model training round with the best evaluation index is selected as the optimal weight, and the optimal weight is applied to the test set of the desert shrub dataset to generate the desert shrub segmentation extraction results.

[0040] In this embodiment, the average intersection-over-union ratio is used as the main evaluation index to evaluate the comprehensive performance of the model in the semantic segmentation task. In each training cycle, the changes in the MIOU index are continuously monitored, and the model weight of each cycle is recorded. During the model training process of a set round, when the model achieves the maximum MIOU on the validation set, the corresponding model weight is saved as the optimal model weight. Finally, the optimal model weight is selected for the prediction of the test set to ensure the best performance of the model in the desert shrub extraction task.

[0041] (5) Using the desert shrub dataset constructed in step (1), the FETNet model and several different deep learning models are tested under the same conditions, and the evaluation indicators of the FETNet model and several different deep learning models are compared to verify the effectiveness. In this embodiment, the deep learning models used for comparison testing include HRRNet model, DFANet model, DeepLabv3+ model, HRNet model, FCN16s model, ESPNetv2 model, ExtremeC3Net model, FCN8s model and SegNet model.

[0042] In this embodiment, five indicators, PA, MPA, F1 score, MIOU and FWIOU, are used for evaluation. The results are shown in Table 1 below.

[0043] Table 1

[0044] The FETNet model showed the highest segmentation accuracy in the desert shrub segmentation task and outperformed other models in all indicators.

[0045] In high-resolution remote sensing images, desert shrubs vary in size and shape and are mostly small, dense objects. The training and prediction results of different deep learning models are also different. In order to intuitively demonstrate the effectiveness of the FETNet model in performing desert shrub segmentation tasks on high-resolution remote sensing images, this example selects the experimental results of several representative deep learning models, and selects several representative types of desert shrub extraction effect diagrams for comparison. Figures 3 to 5 As shown ( Figures 3 to 5 In the figure, each row is the original remote sensing image / segmentation result map of the same area, and each column is the segmentation result map of the same original remote sensing image / same model). Figure 3The original remote sensing images in three areas and the desert shrub segmentation results of different models in the corresponding areas, where sub-image a is the original remote sensing image input to the model, sub-image b is the accurate segmentation result of the desert shrub after manual annotation, sub-image c is the desert shrub segmentation result of SegNet, and sub-image d is the desert shrub segmentation result of the FETNet model; Figure 4 for Figure 3 The enlarged local images selected from the three regions are shown in Figure 1. Sub-image a is the original remote sensing image input to the model, sub-image b is the accurate segmentation result of desert shrubs after manual annotation, sub-image c is the segmentation result of desert shrubs by SegNet, and sub-image d is the segmentation result of desert shrubs by the FETNet model. Figure 5 For Figure 3 The desert shrub segmentation results of three models with the same area are selected in the figure, where sub-image a is the desert shrub segmentation result of DeepLabv3+, sub-image b is the desert shrub segmentation result of ESPNetv2, and sub-image c is the desert shrub segmentation result of FCN16s; Figure 3 and Figure 4 The red frame marks the desert shrub segmentation area with typical characteristics. Figure 3 As can be seen from the sub-figure d of , the FETNet model performs well in the desert shrub segmentation task, especially in the detection and detail capture of small-scale shrubs, which is better than other models. Figure 5 As can be seen from sub-image a of , the classification results of DeepLabv3+ missed a large number of small-scale shrub targets, and the classification effect was not ideal. Figure 5 As can be seen from sub-images b and c, ESPNetv2 and FCN16s perform well in capturing the overall distribution, but are insufficient in processing edge details and complex backgrounds. Figure 3 The subgraph c and Figure 4 As can be seen from sub-figure c, SegNet has certain similarities with the FETNet model in segmentation results, but there are significant deficiencies: in the first row of red box marks, SegNet misidentifies the shadow of the building as a shrub; in the second row of red box marks, the road in the desert is mistakenly identified as a shrub; in the third row of red box marks, SegNet misses detection in dense small-scale shrub areas. In contrast, the FETNet model can capture the edges of shrubs more accurately and maintain a high segmentation accuracy in complex backgrounds, making the segmentation results closer to the actual shrub distribution. In summary, FETNet shows higher accuracy and stability in the desert shrub segmentation task, especially in the extraction of small target shrubs. It shows higher accuracy and robustness, effectively solving the shortcomings of existing technologies.

[0046] In order to verify the effectiveness of introducing the EdgeViT module and FEM module into the baseline model of the FETNet model, an ablation experiment was used to test it. Using the desert shrub dataset constructed in step (1), different modules were gradually added under the same test conditions to evaluate their impact on the model performance. The results are shown in Table 2 below.

[0047] Table 2

[0048] First, only the baseline model (baseline) EfficientNet-B0 is used for feature extraction, and the segmentation results are generated by direct upsampling. Then the EdgeViT module, FEM module, and the combination of EdgeViT module and FEM module are added in turn to observe the improvement effect of EdgeViT module and FEM module on segmentation performance. After the introduction of the EdgeViT module, the pixel accuracy (PA) is slightly improved, and the average intersection over union (MIOU) is also improved, which shows that the EdgeViT module can enhance the ability to capture global information; the FEM module is mainly used for feature enhancement, especially to enhance the feature extraction ability of small targets. After adding the FEM module, the pixel accuracy and average intersection over union are significantly improved, indicating that the module performs particularly well when processing small-scale shrubs, further improving the segmentation accuracy of the model; when the EdgeViT module and the FEM module are added at the same time, the pixel accuracy of the model reaches 98.72%, and the average intersection over union is increased to 92.49%, which is significantly better than the result of using the baseline model alone. This result proves that after adding the EdgeViT module and the FEM module, the performance of the obtained FETNet model in the desert shrub segmentation task has been maximized.

Claims

1. A method for extracting desert shrubs from remote sensing images, characterized in that: The following steps are involved: (1) Collect remote sensing images of the desert area, preprocess the remote sensing images, obtain a desert shrub dataset, and divide it into a training set, a validation set, and a test set; (2) Constructing a FETNet model, wherein the FETNet model uses the EfficientNet-B0 model as a baseline model, including an encoder and a decoder. The encoder retains the original architecture predefined by the baseline model, introduces an EdgeViT module at the jump connection between the encoder and the decoder, and introduces a FEM module between the decoders to obtain the FETNet model; (3) Train the FETNet model, set the initial learning rate, select the cross entropy loss function and use the stochastic gradient descent method to find the parameter configuration that minimizes the loss function, input the training set into the FETNet model to start training, input the validation set into the FETNet model after each round of training, evaluate the performance of the FETNet model, and record the evaluation indicators and the model weights of each round of training; (4) After reaching the set training rounds, the model weight of the corresponding FETNet model training round with the best evaluation index is selected as the optimal weight, and the optimal weight is applied to the test set to generate the desert shrub segmentation extraction result.

2. The method for extracting desert shrubs from remote sensing images according to claim 1, characterized in that: The preprocessing includes: performing atmospheric correction and orthorectification on the remote sensing image, then fusing the panchromatic band and multispectral band of the remote sensing image, cropping the remote sensing image to a set size, marking the desert shrubs and background area on the cropped remote sensing image, and generating a label map of a set size using the marked data.

3. The method for extracting desert shrubs from remote sensing images according to claim 1, characterized in that: The FETNet model includes 7 encoders and 4 decoders. Four EdgeViT modules are introduced at the jump connection between the encoder and the decoder. The EdgeViT module is connected to the encoder output and the corresponding decoder input. A FEM module is introduced between adjacent decoders.

4. The method for extracting desert shrubs from remote sensing images according to claim 1, characterized in that: The cross entropy loss function formula is: ， Where: is the cross entropy loss, is the actual label value, is the predicted value of the model; The stochastic gradient descent method formula is: ， Where: w is the model parameter, t is the update round, Update the gradient of the model parameters after a sample input, η is the learning rate, and i is the sample index selected in the training set at the current iteration.

5. The method for extracting desert shrubs from remote sensing images according to claim 1, characterized in that: The step (3) further includes: using a cosine annealing algorithm to update the learning rate of the FETNet model, the formula is as follows: ， Where: is the learning rate of the tth training round, is the lower limit of the learning rate, is the upper limit of the learning rate, that is, the initial learning rate, is the current number of training rounds, is the maximum number of training rounds.

6. The method for extracting desert shrubs from remote sensing images according to claim 1, characterized in that: The evaluation indicators include one or more of pixel accuracy, average pixel accuracy, F1 score, frequency-weighted intersection overlap, and average intersection-over-union.

7. The method for extracting desert shrubs from remote sensing images according to claim 6, characterized in that: The average intersection-over-combination ratio formula is: ， Where: is the value of the average intersection-union ratio, is a true positive, is a false positive. A false negative is a true negative, is the number of categories.

8. The method for extracting desert shrubs from remote sensing images according to claim 6, characterized in that: In step (4), the model weight corresponding to the FETNet model training round with the largest average intersection-to-union ratio value is taken as the optimal weight.

9. The method for extracting desert shrubs from remote sensing images according to claim 1, characterized in that: The remote sensing image desert shrub extraction method further comprises the following steps: (5) Using the desert shrub dataset, the FETNet model and several different deep learning models were tested under the same conditions. The evaluation indicators of the FETNet model and several different deep learning models were compared to verify their effectiveness.

10. The method for extracting desert shrubs from remote sensing images according to claim 9, characterized in that: The deep learning models include HRRNet model, DFANet model, DeepLabv3+ model, HRNet model, FCN16s model, ESPNetv2 model, ExtremeC3Net model, FCN8s model and SegNet model.

Images