Training and terrace extraction method based on improved semantic segmentation model

Abstract

The invention provides a training and terrace extraction method based on an improved semantic segmentation model, a network structure of the improved semantic segmentation model is composed of an encoder and a decoder, the encoder adopts ResNet50, and the extraction method comprises the following steps: obtaining a GF-2 image, DEM data and land cover data of a to-be-extracted terrace area; the GF-2 image comprises a multispectral image and a panchromatic image; fusing the multispectral image, the panchromatic image and the DEM data to obtain a fused image of the to-be-extracted terrace area; inputting the fusion image of the to-be-extracted terrace area into a pre-trained improved semantic segmentation model for prediction to obtain a terrace segmentation image; and post-processing the terrace segmentation image based on the DEM data and the land cover data so as to correct the terrace segmentation image and obtain a final terrace segmentation image. The method improves the extraction precision of the terraced field.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a training and terrace extraction method based on an improved semantic segmentation model. Background Art

[0002] Terraces are the main measure for soil and water conservation and improving land productivity. Terraces are widely distributed and of many types, and there are terrace projects of different specifications and utilization methods, such as horizontal terraces (composed of a series of horizontal platforms) built in different historical periods, sawtooth terraces (platforms inclined inward 10-15°), slope-separated terraces (slopes between adjacent terraces) and slope terraces (terraces are inclined and embankments span the slope). Secondly, the fragmented terrain and different types and shapes of terrace patches pose significant challenges to terrace extraction.

[0003] In the prior art, terraces are generally extracted through deep learning semantic segmentation models. However, due to the large terrain undulations in the terraced area, the types of land objects are diverse, including different landscape types such as water bodies, vegetation, buildings, roads, etc., and the land cover types of terraced landscapes are diverse, and different vegetation covers have different spectral characteristics. These complex information bring multiple challenges to the deep learning semantic segmentation model, resulting in the terraced images extracted by the existing deep learning semantic segmentation model to have problems such as category imbalance, blurred boundaries, and loss of details, which in turn leads to low extraction accuracy of the existing semantic segmentation model. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a training and terrace extraction method based on an improved semantic segmentation model to solve at least one of the above problems.

[0005] In a first aspect, a terrace extraction method based on an improved semantic segmentation model is provided. The network structure of the improved semantic segmentation model consists of an encoder and a decoder. The encoder adopts ResNet50. The method includes:

[0006] Obtain GF-2 images, DEM data and land cover data of the terraced field area to be extracted; GF-2 images include multispectral images and panchromatic images;

[0007] The multispectral image, panchromatic image and DEM data are fused to obtain the fused image of the terraced field area to be extracted;

[0008] Input the fused image of the terraced field area to be extracted into the pre-trained improved semantic segmentation model for prediction to obtain the terraced field segmentation image;

[0009] The terrace segmentation image is post-processed based on DEM data and land cover data to correct the terrace segmentation image and obtain the final terrace segmentation image.

[0010] Optionally, the multispectral image, the panchromatic image and the DEM data are fused to obtain a fused image of the terrace area to be extracted, including:

[0011] Performing first preprocessing on the multispectral image and the panchromatic image respectively; the first preprocessing includes one or more combinations of the following: radiation calibration, atmospheric correction, orthorectification;

[0012] The multispectral image after the first preprocessing and the panchromatic image are preliminarily fused to obtain a preliminary fused image including four multispectral bands; the four multispectral bands are red, green, blue and near infrared bands;

[0013] The DEM data is converted to obtain a DEM raster image with the same resolution as the initial fused image;

[0014] The DEM raster image is fused with the preliminary fused image to obtain a fused image of the terraced area to be extracted, which contains four multispectral bands and DEM bands.

[0015] Optionally, post-processing the terrace segmentation image based on the DEM data and the land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image includes:

[0016] The edge of the predicted terrace segmentation image is preliminarily corrected by the CRF model to obtain a preliminarily corrected image;

[0017] Generate a terrace slope map based on the DEM data of the terrace to be extracted;

[0018] The land cover data, terrace slope map and preliminary correction map are superimposed to obtain the spatial distribution map of terraces;

[0019] The spatial distribution map of terraces is masked to correct the non-terrace areas as the background and obtain the final terrace segmentation image.

[0020] In a second aspect, a training method for improving a semantic segmentation model is provided, the training method comprising:

[0021] Obtain the GF-2 image history data and DEM data of the target terrace area;

[0022] The target fused image set is obtained by fusing the historical data of GF-2 images and the DEM data of the target terraced area;

[0023] Performing a second preprocessing on the target fused image set to obtain training samples; the second preprocessing at least includes labeling and data expansion and enhancement processing;

[0024] The training samples are input into the improved semantic segmentation model to be trained for iterative training until the preset training conditions are met. The preset training conditions are to reach a preset total number of iterations or the loss of the model is minimized, wherein the parameters of the improved semantic segmentation model are adjusted by a preset loss function in each iterative training.

[0025] Optionally, performing a second preprocessing on the target fused image set to obtain a training sample includes:

[0026] Selecting a fused image set to be labeled in the target fused image set;

[0027] Label the fused image set to be labeled to obtain a label image set;

[0028] Data augmentation and enhancement processing is performed on each frame image in the label image set and the fused image set to be annotated to obtain training samples; the data augmentation and enhancement processing at least includes rotation, horizontal and vertical flipping, diagonal mirroring, salt and pepper, and Gaussian noise enhancement.

[0029] Optionally, the improved semantic segmentation model includes an encoder and a decoder. The encoder adopts ResNet50, including a feature extraction module, a convolution module, a maximum pooling module, a plurality of residual modules and a global pooling module connected in sequence. Each iterative training process of the improved semantic segmentation model includes:

[0030] Extracting initial features of each image in the training sample based on the feature extraction module to obtain an initial feature image;

[0031] Downsampling the initial feature image based on the convolution module and the maximum pooling module to obtain a first intermediate feature image;

[0032] Calculating residual information of the first intermediate feature image based on multiple residual modules, and adding the intermediate feature image to the residual information to obtain a second intermediate feature image;

[0033] Performing global pooling processing on the second intermediate feature image based on the global pooling module to obtain a global feature image;

[0034] The decoder decodes the global feature image and outputs the predicted terrace segmentation image.

[0035] Optionally, the preset loss function consists of a Dice Loss function and a cross entropy loss function, and the expressions of each function are:

[0036] L=λ·L Dice +(1-λ)·L CE

[0037]

[0038]

[0039] Among them, L is the preset loss function; λ is a weight parameter between 0 and 1; L Dice is the Dice Loss function, which is used to measure the similarity between the predicted terrace segmentation image and the true label image; L CE is the cross entropy loss function, which is used to measure the error between the probability distribution of the predicted terrace segmentation image and the probability distribution of the true label image; Y pred is the terrace segmentation image predicted by the model; Y gt is the true label image; Y gt (c) is the one-hot representation of the true label image; Y pred (c) is the probability distribution diagram of the predicted terrace segmentation image.

[0040] In a third aspect, a terrace extraction device based on an improved semantic segmentation model is provided. The network structure of the improved semantic segmentation model consists of an encoder and a decoder. The encoder adopts ResNet50. The device includes:

[0041] The acquisition unit is used to acquire the GF-2 image, DEM data and land cover data of the terrace area to be extracted; the GF-2 image includes multispectral image and panchromatic image;

[0042] A fusion unit is used to fuse the multispectral image, the panchromatic image and the DEM data to obtain a fused image of the terrace area to be extracted;

[0043] A prediction unit, used for inputting the fused image of the terrace area to be extracted into the trained improved semantic segmentation model to predict and obtain a terrace segmentation image;

[0044] The post-processing unit is used to post-process the terrace segmentation image based on the DEM data and the land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image.

[0045] In a fourth aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0046] Memory, used to store computer programs;

[0047] The processor is used to implement the method steps described in any one of the first aspect or the second aspect when executing the program stored in the memory.

[0048] According to a fifth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps described in any one of the first aspect or the second aspect are implemented.

[0049] The embodiment of the present invention provides a training and terrace extraction method based on an improved semantic segmentation model, the extraction method comprising: obtaining GF-2 images, DEM data and land cover data of the terrace area to be extracted; the GF-2 images include multispectral images and panchromatic images; fusing the multispectral images, panchromatic images and DEM data to obtain a fused image of the terrace area to be extracted; inputting the fused image of the terrace area to be extracted into a pre-trained improved semantic segmentation model for prediction to obtain a terrace segmentation image; post-processing the terrace segmentation image based on the DEM data and land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image. The present invention segments the terrace image based on the improved semantic segmentation model, the improved semantic segmentation model uses ResNet50 as an encoder, can reduce the loss of terrace details, such as the loss of long strip terraces, and collects multi-source data such as DEM and GF-2 images and fuses them into the improved semantic segmentation model, thereby improving the extraction accuracy of the terraces. In addition, the extracted terrace segmentation image is corrected by post-processing means, further improving the extraction accuracy of the terraces.

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of the structure of the ResU-Net model provided in an embodiment of the present invention is shown;

[0053] Figure 2 A schematic diagram of the structure of the U-Net++ model provided in an embodiment of the present invention is shown;

[0054] Figure 3 A schematic diagram of the structure of the RVTransUNet model provided in an embodiment of the present invention is shown;

[0055] Figure 4 A schematic diagram of the structure of the XDeepLabV3+ model provided in an embodiment of the present invention is shown;

[0056] Figure 5 A schematic diagram of the structure of the ResPSPNet model provided by an embodiment of the present invention is shown;

[0057] Figure 6 A flowchart of a training method for improving a semantic segmentation model provided by an embodiment of the present invention is shown;

[0058] FIG. 7( a ) shows a GF-2 fused image to be annotated according to an embodiment of the present invention;

[0059] FIG. 7( b ) shows a label image diagram according to an embodiment of the present invention;

[0060] Figure 8(a) shows the extraction effect of terraces of different models in the area covered by vegetation;

[0061] Figure 8(b) shows the terrace extraction effects of different models in an area with both vegetation cover and bare ground;

[0062] Figure 8(c) shows the terrace extraction effects of different models in the bare ground area;

[0063] Fig. 9 A flow chart of a terrace extraction method based on an improved semantic segmentation model provided by an embodiment of the present invention is output;

[0064] Fig.10 A schematic diagram of the structure of a terrace extraction device based on an improved semantic segmentation model provided by an embodiment of the present invention is shown;

[0065] Fig.11 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0067] Taking into account the problems of category imbalance, blurred boundaries, detail overload, etc. in terraced field images extracted by existing deep learning semantic segmentation models, an improved semantic segmentation model is provided in an embodiment of the present invention, which is described below through an embodiment.

[0068] The improved semantic segmentation model provided in an embodiment of the present invention adopts a deep learning network model, which can be one of a ResU-Net model, a U-Net++ model, a RVTransUNet model, a XDeepLabV3+ model or a ResPSPNe model.

[0069] like Figure 1 As shown in the figure, the ResU-Net model is a deep learning model that combines the residual network (ResNet) and U-Net architecture.

[0070] The U-Net++ model is an improved network based on the traditional U-Net model. Its model structure is as follows: Figure 2 Compared with the U-Net network, U-Net++ significantly improves the segmentation performance and robustness of the model by introducing dense skip connections, deep supervision, and improved feature fusion. The U-Net++ model has the advantage of capturing image feature details, and has stronger expression ability and better small sample learning ability. It has obvious advantages in multi-scale feature extraction and can better capture boundary details.

[0071] The TransUNet model is an improved model that addresses the limitations of the U-Net model in terms of model long-range dependencies and the Transformer model’s lack of shallow feature details, which results in limited positioning capabilities. Figure 3 As shown in the figure, the left side of the TransUnet model is the encoder, which chooses to combine the ResNet-50 network and the ViT model with 12 Transformer encoding layers to form a hybrid encoder, namely the R50-ViT encoder; and the ResNet-50 network and the ViT (Vision Transformer) model have been pre-trained on the self-made data set, where the ResNet-50 network is used to extract feature maps, and the ViT model encodes the feature maps into input sequences. The right side is the decoder, which is a cascade upsampler. There are large differences in the features of remote sensing images. The same target often has different spectral features and shapes, and has strong mixed pixel characteristics. Using R50-ViT as the encoder, ResNet-50 enables the TransUNet model to effectively capture local features and details, and overcome complex spatial and spectral information. ViT enables the TransUNet model to learn long-distance dependencies in the entire image, thereby capturing more complex global features. The R50-ViT encoder enables the model to have stronger expressiveness, adaptability and generalization capabilities.

[0072] The DeepLabV3+ model also uses an encoder-decoder structure. Because GF-2 images and high-resolution DEM data are large in size and have more details, such as Figure 4As shown in the figure, the DeepLabV3+ model uses Xception as the backbone network of the DeepLabV3+ model. As the backbone network of DeepLabv3+, Xception effectively solves the problems of feature learning efficiency, spatial information capture, multi-scale processing, overfitting risk, migration and reasoning speed through innovative designs such as deep separable convolution. This enables DeepLabv3+ to achieve excellent performance in semantic segmentation tasks while keeping the model lightweight and efficient.

[0073] The ResPSPNet model is an improvement on the PSPNet model. When PSPNet discriminates small local targets, if more global information is introduced in the segmentation layer, the probability of erroneous extraction can be relatively reduced. Due to the limitations of the high-resolution GF-2 image due to its large data volume and diverse background details, the PSPNet model is relatively complex and has limited effect on the recognition of small terraces. Figure 5 As shown in the figure, ResNet50 is used as the backbone network of the PSPNet model. First, ResNet50 is relatively light, which helps to improve the processing efficiency of the model on high-resolution images. Secondly, ResNet50 can effectively alleviate the gradient vanishing problem in deep neural networks, allowing the PSPNet model to learn more complex features. In addition, the multi-level network structure of ResNet50 can effectively extract multi-scale features from local to global. Combined with the PSPNet pyramid pooling module, the model's ability to segment objects of different sizes and in complex backgrounds is greatly enhanced.

[0074] From the introduction of the above models, we can understand that these five models all adopt the encoding-decoding structure and have adjusted the backbone network part.

[0075] In the implementation of the present invention, the ResU-Net model is preferably used because it has been experimentally proven that the extraction effect of the ResU-Net model is the best. The experimental comparison data of these five models will be described in the following model training embodiment and will not be repeated here.

[0076] The following mainly describes the ResU-Net model structure. Figure 1As shown, the improved semantic segmentation model includes an encoder and a decoder. The encoder adopts ResNet50, which is composed of a 50-layer network structure, specifically including 4 feature extraction modules, a convolution module, a maximum pooling module, 4 residual modules and a global pooling module; the input channels of the 4 feature extraction modules are set to 5, and the size of the input image is set to 256*256. This is because the image trained in the embodiment of the present invention adopts GF-2 images, which are composed of multispectral images of four bands and panchromatic images of panchromatic bands, with a total of 5 bands, that is, corresponding to five channels. Even if the multispectral image and the panchromatic image are fused to obtain an image of four bands, since the input image of the improved semantic segmentation model is also combined with DEM (Digital Elevation Model) data, the DEM data is used as a separate band, so there are still five bands, still corresponding to five input channels, and the image fusion will be described in detail in the following embodiments, which will not be repeated here.

[0077] The convolution module and the maximum pooling module receive an input image of 256*256*5. After a 7*7 convolution layer with a stride of 2 and a 3*3 maximum pooling layer, the spatial resolution of the input image is reduced to 64*64.

[0078] Among them, GF-2 images are more complex to calculate and have a large amount of data compared to other low-resolution images. When existing deep learning models read complex image information, they tend to lose a lot of detail information during the convolution process, such as losing small areas of narrow terraced fields. Therefore, the improved semantic segmentation model provided in an embodiment of the present invention sets four residual modules for processing, and each residual module gradually adjusts the spatial size and number of channels of the feature map. Specifically, it includes a first residual module, a second residual module, a third residual module and a fourth residual module; the first residual module contains 3 residual blocks, the second residual module contains 4 residual blocks; the third residual module contains 6 residual blocks; the fourth residual module contains 4 residual blocks. Each residual block contains three convolutional layers: 1*1, 3*3, and 1*1. The working principle of these four residual modules is to perform residual calculation on the 64*64*5 feature map output by the maximum pooling through the first residual module, and add the calculated residual information to the input image of the first residual module through a shortcut connection to output a 64*64*256 feature image, indicating that the size of the feature image is 64*64 and the number of channels is 256. In the same principle, the feature image output by the first residual module is input to the second residual module, and is added to the residual information calculated by the second residual module through a shortcut connection to output a 32*32*512 feature image; the 32*32*512 feature image is input to the third residual module, and is added to the residual information calculated by the third residual module through a shortcut connection to output a 16*16*1024 feature image; the 16*16*1024 is input to the fourth residual module, and is added to the residual information calculated by the fourth residual module through a shortcut connection to output a 8*8*2048 feature image. These four residual modules directly add the original input feature image to the output through shortcut connections. Without the need for transformations in each intermediate layer, it can reduce redundant calculations on the one hand, and retain useful information of the input image on the other hand, effectively reducing the loss of image information, and can extract small and narrow terrace areas. This makes the terrace extraction more accurate.

[0079] After all residual modules, a global pooling module is applied to convert the local feature maps into 2048-dimensional global feature maps to provide high-dimensional feature images for the decoder.

[0080] Finally, the decoder gradually upsamples the image output by the encoder to restore the spatial resolution of the image and generate a high-resolution segmentation map that is consistent with the size of the encoder's input image.

[0081] The multi-layer features extracted based on the ResNet50 encoder can accurately capture the detailed features of the terraces, especially the slender edges and complex terrain information, and its feature extraction capability has been significantly enhanced. In addition, the ResNet50 encoder can load pre-trained weights, so that the network can converge quickly on a smaller data set while improving generalization capabilities.

[0082] The embodiment of the present invention provides a training method for improving a semantic segmentation model. Figure 6 As shown, the training method includes the following steps:

[0083] Step S601: Acquire GF-2 image history data of the target terrace area and DEM data of the target terrace area.

[0084] In an embodiment of the present invention, the target terrace area can be a loess hilly terrace area, in which the terrain is fragmented, the landscape is diverse, the terraces are widely distributed, and there are many types. The terrain runs from south to north, and the altitude is distributed between 1392m-2816m. The complex background elements and spectral characteristics interfere greatly with the extraction of terraces. Therefore, an embodiment of the present invention uses GF-2 images as one of the data sources. The GF-2 image is composed of a multispectral image with a resolution of 4m and a panchromatic image with a resolution of 1m. It can combine spectral images of multiple bands, which provides rich and reliable data for model training.

[0085] DEM data can be obtained through the BIGMAP platform with a resolution of 4m.

[0086] Step S602: Fusing the GF-2 image history data and the DEM data of the target terraced field area to obtain a target fused image set.

[0087] In this step, the multispectral image is first calibrated by radiometric calibration, atmospheric correction and orthorectification; the panchromatic image is calibrated by radiometric calibration and orthorectification; and the NNDiffuse translation sharpening tool is used to fuse the preprocessed multispectral image and panchromatic image together. After fusion, the GF-2 fused image has four bands: red, green, blue and near infrared, with a spatial resolution of 1m.

[0088] The DEM data is converted into a visual image format, then terrain corrected and upsampled to a spatial resolution of 1m, and then temporally aligned with the GF-2 fused image. After alignment, the DEM image corresponding to the same timestamp is fused with the GF-2 fused image. The fused image has five bands, with the DEM band added to the red, green, blue, and near-infrared bands.

[0089] Since the types of landforms in the area where the terraces are located are complex, the terrain is diverse, and the scenes are changeable, it is easy to make mistakes when training only through GF-2 images. For example, the river valley and the plateau are non-terraced areas, but these areas have spectral characteristics similar to those of the terraces, which will cause these areas to be identified as terraced areas. However, the DEM data of the river valley and the terraces are different. Therefore, the embodiment of the present invention improves the accuracy of model training by integrating the DEM data as a single band into the GF-2 image for training, and combining the DEM data as training data based on multi-spectral features.

[0090] Step S603: performing a second preprocessing on the target fused image set to obtain training samples; the second preprocessing at least includes labeling and data expansion and enhancement processing.

[0091] In a feasible implementation manner, performing a second preprocessing on the target fused image set to obtain training samples comprises the following steps:

[0092] Step S603A: Selecting a fused image set to be labeled from the target fused image set.

[0093] In this step, the target fused image set contains fused images of multiple geographical areas. First, 30 representative fused image sets are selected and cropped into 256*256 pixel blocks. The shuffle method is used to perform several random selections among all the cropped pixel blocks. The number of pixel blocks belonging to the same area is counted in the results of these random selections, and the areas are sorted from large to small according to the number. The fused images in the top 5 areas are selected as the fused image set to be labeled.

[0094] Step S603B: label the fused image set to be labeled to obtain a labeled image set.

[0095] In this step, manual or automatic annotation can be performed. Specifically, the pixel value of the terrace is set to 255, and RGB (255, 255, 255) is marked with white. The pixel value of other parts is set to 0, and RGB (0, 0, 0) is marked with black. Then, the vector is converted to a raster to complete the annotation. Finally, the overlapping sliding window method (256*256 pixels) is used to crop the label image and the fused image corresponding to the label image, and format them. As shown in Figure 7, the fused image to be annotated is Figure 7(a), and the label image set is Figure 7(b).

[0096] Step S603C: performing data expansion and enhancement processing on each frame image in the label image set and the fused image set to be labeled to obtain training samples.

[0097] The data augmentation and enhancement processing includes at least rotation (180°), horizontal and vertical flipping, diagonal mirroring, salt and pepper, and Gaussian noise enhancement.

[0098] Step S604: input the training samples into the improved semantic segmentation model to be trained for iterative training until the preset training conditions are met. The preset training conditions are reaching a preset total number of iterations or minimizing the loss of the model, wherein the parameters of the improved semantic segmentation model are adjusted by a preset loss function in each iterative training.

[0099] Before training, first configure the various parameters of the improved semantic segmentation model, including parameters such as input image size, batch size, learning rate, number of iterations, loss function and gradient descent strategy. In a specific example, the image size of each training sample is set to 256*256 pixels. A learning rate that is too high may cause the model to diverge, and a learning rate that is too low will slow down the training process or converge the model to a local optimal solution. In this embodiment, a learning rate of 0.0001 is selected. The model is basically stable after 80 iterations. In this embodiment, the number of iterations is selected to be 100 and the batch size is selected to be 16. Since the GF-2 image is rich in details, the gradient may be relatively sparse, so Adam is selected for the gradient descent strategy. The loss function is the DiceLoss+cross entropy joint loss function. The remaining parameters all select the network default parameters.

[0100] In this step, the images in the training samples are input into the improved semantic segmentation model. After segmentation, a predicted terrace probability distribution map is first obtained, and then the areas with probabilities greater than 0.8 are determined as terrace areas, and the areas with probabilities less than or equal to 0.8 are determined as non-terrace areas, and then the final predicted terrace segmentation map is output.

[0101] In the embodiment of the present invention, the preset loss function is composed of a Dice Loss function and a cross entropy loss function, and the expressions of each function are:

[0102] L=λ·L Dice +(1-λ)·L CE (1);

[0103]

[0104]

[0105] Among them, L is the preset loss function; λ is a weight parameter between 0 and 1, which is used to balance the relative importance of the two loss functions. After multiple experimental comparisons, the λ value is determined to be 0.35; L Dice is the Dice Loss function, which is used to measure the similarity between the predicted terrace segmentation image and the true label image; L CEis the cross entropy loss function, which is used to measure the error between the predicted terrace probability distribution map and the true label distribution; Y pred is the terrace segmentation image predicted by the model; Y gt is the true label image; Y gt (c) is the one-hot representation of the true label image; Y pred (c) is the predicted probability distribution map of terraces.

[0106] Terraces have complex shapes and slender structures, and are densely distributed. The terrain has a great influence on the distribution differences of terraces. In order to reduce the impact of the sample category imbalance problem on the training results and enhance the performance of the model. Therefore, the embodiment of the present invention selects the joint loss function Dice Loss + cross entropy as the loss function of the improved semantic segmentation model. Dice Loss can capture the details of the terraces, and cross entropy can reduce the imbalanced distribution of terrace categories and non-terrace categories. Under the optimization of multiple loss functions, the model's tuning and characterization capabilities are improved, and the extraction accuracy of the model is improved to a certain extent.

[0107] After the improved semantic segmentation model is trained, in order to evaluate the performance of the model, the test samples can be used to evaluate the performance of the model. Specifically, the samples obtained in step S203 after the second preprocessing of the target fused image set can be divided into training samples and test samples in a ratio of 8:2. The training samples are used to train the model, and the test samples are used to test the performance of the model.

[0108] In order to quantitatively evaluate the accuracy of model training, the image of each test sample is input into the trained improved semantic segmentation model for prediction, and the predicted result is compared with the true label pixel by pixel. And create an M×M confusion matrix based on the comparison results, where M is the number of categories. In the embodiment of the present invention, the main task is to extract terraces, so M=2, that is, terrace categories and non-terraced categories. In the confusion matrix, the comparison results include true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN). True positives indicate that the actual category is terraces, and the predicted result is also identified as terraces; true negatives indicate that the actual category is non-terraced, and the predicted result is also identified as non-terraced; false positives indicate that the actual category is non-terraced, and the predicted result is also identified as non-terraced; false positives indicate that the actual category is non-terraced, and the predicted result is incorrectly identified as terraces; false negatives indicate that the actual category is terraces, and the predicted result is incorrectly identified as non-terraced.

[0109] Then, the constructed M×M confusion matrix is ​​used to calculate the overall accuracy (OA), mean pixel accuracy (MPA), mean intersection over union (MIOU), precision (Precision), recall (Recall) and F1-Score of the model to quantitatively evaluate the extraction effect of the model. The model evaluation indicators and the corresponding calculation methods are shown in Table 1:

[0110] Table 1: Model evaluation indicators and corresponding calculation methods

[0111]

[0112]

[0113] In another embodiment, in order to improve the accuracy of terrace extraction, the results predicted by the improved semantic segmentation model can also be post-processed, and the post-processing process includes: preliminarily correcting the edges of the predicted terrace segmentation image through the CRF (Conditional Random Field) model to obtain a preliminary corrected image; generating a terrace slope map according to the DEM data of the terrace to be extracted; superimposing the land cover data SinoLC-1, the terrace slope map and the preliminary corrected map to obtain a terrace spatial distribution map; masking the terrace spatial distribution map to correct the non-terraced area to the background, and obtaining the final terrace segmentation image. If the pixel belongs to a road, building, water body or the slope is less than 2°, it is modified to the background value, otherwise the original value is retained.

[0114] The ResU-Net model, U-Net++ model, RVTransUNet model, XDeepLabV3+ model and ResPSPNet model are all trained using the above method and obtain relevant evaluation indicators.

[0115] After training, the extraction results of each model were analyzed globally and locally, and it was found that each model successfully detected terraced and non-terraced areas, but there were large differences in the extraction between the models. Through the analysis of the results, the terraces are mainly concentrated in the hilly areas with steep terrain in the south and middle of the target basin, and a few are distributed on the east and west sides of the basin. The northern part of the basin is a plateau area with fewer distributions. The ResU-Net model not only performs well in the central and southern mountainous areas, but also performs better than other models in the northern plateau area. The main differences among the remaining models are in the plateau area in the northern part of the basin. Through field investigations, the northern part of the basin is mostly where watermelons are grown, and its spectral characteristics are similar to those of terraces, so there are many areas of misextraction.

[0116] The accuracy of the five models is high, and they have basically made effective identification of terraces. This shows that the terrace patch extraction method based on high-resolution GF-2 images and deep learning is feasible in the loess hilly area. However, topography and spectral characteristics have a significant impact on terrace extraction. In the experiment, three typical areas were selected to verify the extraction effect of the model, such as Figure 8(a)-Figure 8(c)As shown in Figure 8(a), it is a terrace covered with vegetation; Figure 8(b) is a terrace with vegetation and bare ground; Figure 8(c) is a terrace with bare ground. The results show that ResU-Net has the best effect in extracting terraces in these three areas. ResU-Net has made relatively accurate extractions of terraces with different spectral characteristics and terrain characteristics, especially small terraces, because ResU-Net uses ResNet50 as the encoder of the U-Net network and can integrate features at different levels. U-Net++ has incorrect extractions in areas with similar spectral characteristics, especially in steep slope areas. RVTransUNet is less sensitive to spectral features and has more incorrect extractions in areas with more complex terrain. XDeepLabV3+ does not make accurate segmentation of the boundaries, and has strong adhesion, especially in densely distributed areas with small terrace spacing. The ResPSPNet model has a large area segmentation. Due to the lack of a deeper or more complex network structure, the ResPSPNet model does not capture the local characteristics of the terraces very well, so the model tends to generate a smoother regional segmentation, resulting in the segmentation of the entire area. Due to the complex spectral characteristics of terraces covered with vegetation and terraces on vegetation and bare ground, the five models will make mistakes when extracting terraces in these two areas. In contrast, the spectral characteristics of bare ground are single, the boundaries are clear, and there is no vegetation or tree occlusion. The five models have the best effect on the extraction of terraces on bare ground.

[0117] The extraction effects of the five models can also be compared by combining the method of using the M×M confusion matrix in the above example to calculate the overall accuracy (OA), mean pixel accuracy (MPA), mean intersection over union (MIOU), precision (Precision), recall rate (Recall) and F1-Score and other evaluation indicators of the model. The accuracy evaluation comparison results of the five models are shown in Table 2 below:

[0118] Table 2: Accuracy evaluation results of five models (unit: %)

[0119]

[0120] The results show that the six indicators of the ResU-Net model are all the highest. Except for MIoU, the other indicators are all above 90%. The RVTransUNet is followed by the ResPSPNet model with the lowest performance evaluation indicator value. The OA, MPA, MIoU, Precision, Recall and F1-Score of ResU-Net and ResPSPNet differ by 10.61%, 10.84%, 17.04%, 11.20%, 11.18% and 8.93% respectively.

[0121] In addition, the effect of post-processing correction can be verified in the field. Taking the ResU-Net model as an example, before post-processing, the OA, F1-Score and MioU of the terrace segmentation image extracted based on the ResU-Net model were 92.76%, 90.50% and 85.41% respectively. After post-processing correction and field verification, it was found that the OA of the terrace segmentation image extracted based on the ResU-Net model increased by 1.98% and 0.83% respectively after using terrain data, namely DEM data and land cover correction, and the final OA was 96.67%; F1-Score increased by 1.73% and 0.75% respectively, and the final F1-Score was 93.94%; MIoU increased by 2.03% and 0.81% respectively, and the final MIoU was 89.37%. Through the above data analysis, it can be found that terrain data has the most significant impact on terrace extraction, followed by land cover data correction.

[0122] In addition, the area of ​​the extracted terraces can be calculated to evaluate the extraction effect of each model. As an indicator of the overall distribution of terraces, the area of ​​terraces can be an effective reference for comparing the prediction results of the models.

[0123] In a feasible implementation, three methods can be used to estimate the area of ​​terraces: (1) pixel counting (PC), (2) sample proportion (SP), and (3) model-assisted estimation (EM) based on the error matrix. The PC method directly estimates the area of ​​terraces by calculating the number of pixels in the terraces; the SP method randomly selects sample points and calculates their proportions to infer the area of ​​terraces; the EM method combines the terrace probability distribution map and the test samples, and corrects the area estimation result obtained by the PC method by calculating the sample proportion. The specific calculation methods of these three methods are all prior art, and those skilled in the art should understand them, so they will not be repeated here. Assuming a 95% confidence interval, the three methods are used to estimate the area of ​​the five models before and after post-processing correction, as shown in Table 3.

[0124] Table 3: Estimated area of ​​terraced fields (unit: km 2 )

[0125]

[0126]

[0127] Comparing the five models, it is found that the terrace areas estimated by the three methods are relatively similar, and the ResU-Net model has the best effect. Among the five models, ResU-Net, U-Net++, RVTransUNet and XDeepLabV3+ have small differences in area, because these four models all extract terrace patches with high accuracy. In contrast, ResPSPNet has the largest area, because only ResPSPNet extracts the entire area, and cannot accurately segment the boundaries, resulting in large areas of incorrect extraction.

[0128] It can also be found that the ResU-Net model has the smallest area difference before and after post-processing correction, and the area difference before and after correction is less than 7km 2 This also indirectly verifies that the ResU-Net model has the best extraction effect, and the extracted data has high reliability, which can provide reliable data support for subsequent decision-making and research on terrace ecosystem services.

[0129] After the above experiments, we can draw the following conclusions:

[0130] The overall accuracy of the terrace segmentation map predicted by the ResU-Net model is high, with OA, MPA, Precision, Recall, F1-Score and MIoU of 92.76%, 93.10%, 91.51%, 93.80%, 90.50% and 85.41% respectively, and it shows good edge segmentation accuracy. These results show that in the Loess Hilly Region, the semantic segmentation method based on deep learning is feasible for terrace segmentation at a resolution of 1 meter. In addition, this implementation extracts fine patch terraces from GF-2 images by improving the model structure and loss function, and then optimizes the extraction of local information by fusing morphological, textural and spectral features to solve the problems of poor adhesion and poor edge segmentation performance, and achieve high-standard terrace mapping. Based on the model extraction results, after correction by pixel masking and other methods, the overall extraction accuracy reaches 96.67%. In addition, the model-assisted estimation (EM) method based on the error matrix can estimate the terrace area of ​​the target basin to be 2557±117.96km2. We provide more accurate, detailed and higher-resolution spatial distribution data of terraces, which will provide a solid data foundation for future research and decision-making on ecosystem services such as soil and water conservation and food production in the Zuli River Basin.

[0131] Based on the above-trained improved semantic segmentation model, the model can be deployed in actual business scenarios to extract terraces in other areas. Therefore, another embodiment of the present invention provides a terrace extraction method based on an improved semantic segmentation model. The network structure of the improved semantic segmentation model consists of an encoder and a decoder, and the encoder uses ResNet50. Fig. 9As shown, the method comprises the following steps:

[0132] Step S901: Obtain GF-2 images, DEM data and land cover data of the terraced field area to be extracted; GF-2 images include multispectral images and panchromatic images.

[0133] In this step, the land cover data SinoLC-1 can be obtained through the Zenodo platform. Zenodo is an open access data repository supported by CERN (European Organization for Nuclear Research) that allows researchers to upload and share their research results. It is not limited to the field of physical sciences, but is open to all disciplines, including life sciences, social sciences, humanities, etc.

[0134] Step S902: Fusing the multispectral image, the panchromatic image and the DEM data to obtain a fused image of the terraced field area to be extracted.

[0135] The fusion method is described in the following embodiments and will not be repeated here.

[0136] DEM data provides terrain data, multispectral images provide multispectral features, and panchromatic images provide texture features. By integrating multi-source data of morphological, textural and spectral features, it provides image data with more dimensions and details, providing more accurate data support for improving semantic segmentation models.

[0137] Step S903: inputting the fused image of the terraced field area to be extracted into the pre-trained improved semantic segmentation model for prediction to obtain a terraced field segmentation image.

[0138] Step S904: post-processing the terrace segmentation image based on the DEM data and the land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image.

[0139] Through post-processing, the extracted terrace segmentation image is made more precise and accurate.

[0140] The present invention segments the terraced field image based on an improved semantic segmentation model, which uses ResNet50 as an encoder, can reduce the loss of terraced field details, such as the loss of long strip terraced fields, and collects multi-source data such as DEM and GF-2 images and fuses them into the improved semantic segmentation model, thereby improving the extraction accuracy of terraced fields. In addition, the extracted terraced field segmentation image is corrected by post-processing means, further improving the extraction accuracy of terraced fields.

[0141] Based on the above embodiment, the multispectral image, the panchromatic image and the DEM data are fused to obtain a fused image of the terraced field area to be extracted, including:

[0142] Step S902A: performing first preprocessing on the multispectral image and the panchromatic image respectively; the first preprocessing includes one or more combinations of the following: radiation calibration, atmospheric correction, orthorectification.

[0143] In this step, radiometric calibration is to convert DN (Digital Numbers, raw numbers) into physical units, such as radiance or reflectance, to eliminate the specific response variations of remote sensing sensors. In one example, the radiometric calibration of a multispectral image can convert the DN value of each band into radiance or surface reflectance by applying the calibration coefficients provided by the manufacturer.

[0144] Atmospheric correction includes scattering and absorption to reduce the impact of atmospheric effects in order to obtain data that is closer to the true reflectance characteristics of the surface. For example, an atmospheric model (such as the 6S model, FLAASH, etc.) can be used to estimate and remove the atmospheric path radiation and the atmospheric attenuation component of direct solar radiation to obtain the surface reflectivity.

[0145] Orthorectification is the correction of geometric distortion caused by terrain undulations, sensor attitude angles, and earth curvature, so that each pixel in the image corresponds to an accurate position on the earth's surface. In one example, a DEM digital elevation model and an accurate sensor model can be used for orthorectification to ensure the spatial accuracy of multispectral images.

[0146] Step S902B: Preliminarily fuse the multispectral image after the first preprocessing and the panchromatic image to obtain a preliminary fused image including four multispectral bands; the four multispectral bands are red, green, blue and near-infrared bands.

[0147] In this step, the NNDiffuse translation sharpening tool can be used to fuse the GF-2 multispectral image and the panchromatic image. After fusion, the initial fused image has red, green, blue, and near-infrared bands with a spatial resolution of 1m.

[0148] Step S902C: converting the DEM data to obtain a DEM raster image with the same resolution as the initial fused image.

[0149] In this step, DEM data is acquired through the BIGMAP platform with a resolution of 4m, and then the DEM data is terrain corrected and upsampled to a spatial resolution of 1m. The timestamp of the DEM raster image is aligned with the timestamp of the preliminary fusion image, and then the DEM raster image and the preliminary fusion image with the same timestamp are fused.

[0150] Step S902D: Fusing the DEM raster image with the preliminary fused image to obtain a fused image of the terraced field area to be extracted that includes four multispectral bands and the DEM band.

[0151] The embodiment of the present invention can reduce the probability of erroneous extraction of objects with the same spectral characteristics but belonging to different objects by integrating DEM data into GF-2 images as a single band, thereby improving the extraction accuracy.

[0152] Based on the above embodiment, post-processing the terrace segmentation image based on DEM data and land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image includes the following steps:

[0153] Step S904A: Preliminary correction is performed on the edge of the predicted terrace segmentation image using the CRF model to obtain a preliminary corrected image.

[0154] In this step, the conditional random field is a directed graph model used to model the relationship between pixels, which is particularly suitable for solving image segmentation problems. The predicted terrace segmentation image is post-processed by using the conditional random field to further improve the accuracy of the terrace contour.

[0155] The principle of CRF is to optimize the conditional distribution of pixel labels to achieve modeling of label similarity and smoothness between adjacent pixels, thereby effectively eliminating noise and maintaining edge details. Specifically, CRF combines the multispectral information of the input image with the predicted terrace segmentation image, and optimizes the label of each pixel in the terrace segmentation image by defining node energy and edge energy: node energy reflects the credibility of a single pixel being classified as a specific label, and edge energy takes into account the label consistency between adjacent pixels. Through CRF post-processing, the edges of the terrace segmentation image are clearer, improving the terrace extraction effect.

[0156] By comprehensively utilizing advanced deep learning models and an efficient CRF post-processing stage, this method can significantly improve the recognition ability of complex terrain and details and improve the overall accuracy of terrace extraction.

[0157] Step S904B: Generate a terrace slope map based on the DEM data of the terrace to be extracted and the preliminary corrected image.

[0158] In this step, DEM data is a raster data. First, the slope of each raster is calculated using DEM data. In one example, the slope can be calculated using the terrain analysis tool in the GIS software. Most GIS software has a built-in slope calculation function that can be directly applied to DEM data. A terrace slope map can be generated based on the slope data of each raster.

[0159] Step S904C: superimposing the land cover data with the terrace slope map to obtain a terrace spatial distribution map.

[0160] Land cover data includes plants, buildings, roads, water bodies, etc. It should be noted that when overlaying, it is necessary to ensure that the land cover data and the terrace slope map are in the same format, either raster format or vector format. In one example, the spatial analysis tool in the GIS software can be used to perform overlay analysis.

[0161] Step S904D: Masking is performed on the terraced field spatial distribution map to correct the non-terraced field area as the background, thereby obtaining a final terraced field segmentation image.

[0162] In this step, the pixels in the mask are evaluated one by one. If the pixel belongs to a road, a building, a water body, or the slope is less than 2°, it is modified to the background value, otherwise the original value is retained.

[0163] By correcting DEM data and land cover data, areas that have nothing to do with terraces, such as buildings, roads, and water bodies, can be eliminated, thereby improving the accuracy and precision of terrace extraction.

[0164] Based on the same inventive concept, a terrace extraction device based on an improved semantic segmentation model is provided. The network structure of the improved semantic segmentation model consists of an encoder and a decoder, and the encoder adopts ResNet50. Fig.10 As shown, the device comprises:

[0165] The acquisition unit 1001 is used to acquire the GF-2 image, DEM data and land cover data of the terrace area to be extracted; the GF-2 image includes a multispectral image and a panchromatic image;

[0166] A fusion unit 1002 is used to fuse the multispectral image, the panchromatic image and the DEM data to obtain a fused image of the terrace area to be extracted;

[0167] A prediction unit 1003 is used to input the fused image of the terraced field area to be extracted into the trained improved semantic segmentation model to perform prediction to obtain a terraced field segmentation image;

[0168] The post-processing unit 1004 is used to perform post-processing on the terrace segmentation image based on the DEM data and the land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image.

[0169] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, such as Fig.11 As shown, it includes a processor 1101 , a communication interface 1102 , a memory 1103 and a communication bus 1104 , wherein the processor 1101 , the communication interface 1102 , and the memory 1103 communicate with each other via the communication bus 1104 .

[0170] Memory 1103, used for storing computer programs;

[0171] The processor 1101 is used to implement the steps of the terrace extraction method or the improved semantic segmentation model training method when executing the program stored in the memory 1103:

[0172] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0173] The communication interface is used for communication between the above electronic device and other devices.

[0174] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0175] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0176] The computer program product for terrace extraction provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.

[0177] The device for terrace extraction based on GF-2 images provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0178] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0182] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0183] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A terrace extraction method based on an improved semantic segmentation model, characterized in that: The network structure of the improved semantic segmentation model consists of an encoder and a decoder, the encoder adopts ResNet50, and the method includes: Obtain GF-2 images, DEM data and land cover data of the terraced field area to be extracted; the GF-2 images include multispectral images and panchromatic images; The multispectral image, the panchromatic image and the DEM data are fused to obtain a fused image of the terraced field area to be extracted; Inputting the fused image of the terraced field area to be extracted into a pre-trained improved semantic segmentation model for prediction to obtain a terraced field segmentation image; The terrace segmentation image is post-processed based on the DEM data and the land cover data to correct the terrace segmentation image to obtain a final terrace segmentation image.

2. The method according to claim 1, characterized in that The fusing of the multispectral image, the panchromatic image and the DEM data to obtain a fused image of the terraced field area to be extracted comprises: Performing a first preprocessing on the multispectral image and the panchromatic image respectively; the first preprocessing includes one or more combinations of the following: radiation calibration, atmospheric correction, orthorectification; The multispectral image after the first preprocessing and the panchromatic image are preliminarily fused to obtain a preliminary fused image including four multispectral bands; the four multispectral bands are red, green, blue and near infrared bands; Converting the DEM data to obtain a DEM raster image with the same resolution as the preliminary fused image; The DEM raster image is fused with the preliminary fused image to obtain a fused image of the terraced field area to be extracted, which includes four multispectral bands and a DEM band.

3. The method according to claim 1, characterized in that The post-processing of the terrace segmentation image based on the DEM data and the land cover data to correct the terrace segmentation image to obtain the final terrace segmentation image includes: The edge of the predicted terrace segmentation image is preliminarily corrected by the CRF model to obtain a preliminarily corrected image; Generate a terrace slope map based on the DEM data of the terrace to be extracted; Superimposing the land cover data, the terrace slope map and the preliminary correction map to obtain a terrace spatial distribution map; The terraced field spatial distribution map is subjected to mask processing to correct the non-terraced field area as the background, thereby obtaining a final terraced field segmentation image.

4. A training method for improving a semantic segmentation model, characterized in that: The training method comprises: Obtain the GF-2 image history data and DEM data of the target terrace area; The GF-2 image history data and the DEM data of the target terraced field area are fused to obtain a target fused image set; Performing a second preprocessing on the target fused image set to obtain training samples; the second preprocessing at least includes labeling and data expansion and enhancement processing; The training samples are input into the improved semantic segmentation model to be trained for iterative training until a preset training condition is met, wherein the preset training condition is to reach a preset total number of iterations or the loss of the model is minimized, wherein the parameters of the improved semantic segmentation model are adjusted by a preset loss function in each iterative training.

5. The method according to claim 4, characterized in that The performing second preprocessing on the target fused image set to obtain training samples comprises: Selecting a fused image set to be labeled from the target fused image set; Labeling the fused image set to be labeled to obtain a labeled image set; Data expansion and enhancement processing is performed on each frame image in the label image set and the fused image set to be labeled to obtain training samples; the data expansion and enhancement processing at least includes rotation, horizontal and vertical flipping, diagonal mirroring, salt and pepper and Gaussian noise enhancement.

6. The method according to claim 4, characterized in that The improved semantic segmentation model includes an encoder and a decoder. The encoder adopts ResNet50, including a feature extraction module, a convolution module, a maximum pooling module, a plurality of residual modules and a global pooling module connected in sequence; Each iterative training process of the improved semantic segmentation model includes: Extracting initial features of each image in the training sample based on the feature extraction module to obtain an initial feature image; Downsampling the initial feature image based on the convolution module and the maximum pooling module to obtain a first intermediate feature image; Calculating residual information of the first intermediate feature image based on the multiple residual modules, and adding the intermediate feature image to the residual information to obtain a second intermediate feature image; Performing global pooling processing on the second intermediate feature image based on the global pooling module to obtain a global feature image; The decoder decodes the global feature image and outputs a predicted terrace segmentation image.

7. The method according to claim 4, characterized in that The preset loss function consists of a Dice Loss function and a cross entropy loss function, and the expressions of each function are: L=λ·L Dice +(1-λ)·L CE Among them, L is the preset loss function; λ is a weight parameter between 0 and 1; L Dice is the Dice Loss function, which is used to measure the similarity between the predicted terrace segmentation image and the true label image; L CE is the cross entropy loss function, which is used to measure the error between the probability distribution of the predicted terrace segmentation image and the probability distribution of the true label image; Y pred is the terrace segmentation image predicted by the model; Y gt is the true label image; Y gt (c) is the one-hot representation of the true label image; Y pred (c) is the probability distribution diagram of the predicted terrace segmentation image.

8. A terrace extraction device based on an improved semantic segmentation model, characterized in that: The network structure of the improved semantic segmentation model consists of an encoder and a decoder, the encoder adopts ResNet50, and the device includes: An acquisition unit is used to acquire GF-2 images, DEM data and land cover data of the terraced field area to be extracted; the GF-2 images include multispectral images and panchromatic images; A fusion unit is used to fuse the multispectral image, the panchromatic image and the DEM data to obtain a fused image of the terrace area to be extracted; A prediction unit, used for inputting the fused image of the terraced field area to be extracted into the trained improved semantic segmentation model to perform prediction to obtain a terraced field segmentation image; A post-processing unit is used to post-process the terrace segmentation image based on the DEM data and the land cover data to correct the terrace segmentation image to obtain a final terrace segmentation image.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images