High-resolution remote sensing image farmland plot extraction method and system

Abstract

The invention relates to the technical field of remote sensing image processing, in particular to a high-resolution remote sensing image farmland plot extraction method, which comprises the following steps: collecting a high-resolution remote sensing satellite image, and carrying out data preprocessing and labeling to obtain a data set; the method comprises the following steps: dividing a data set into a training set and a test set, turning, rotating and zooming images of the training set, constructing a high-resolution remote sensing image farmland plot extraction model SED-DUNet based on a dual U-Net network with semantic-edge mutual guidance, and extracting a high-resolution remote sensing image farmland plot extraction model SED-DUNet by using the training set after data enhancement, so as to obtain a high-resolution remote sensing image farmland plot extraction model SED-DUNet; training samples with labels are input in batches into the SED-DUNet for training and network parameters are optimized, the network is finally converged after multiple times of training, and an optimal model is obtained; inputting the test set into the optimal model, and testing to obtain a prediction image of the semantics and the edge of the farmland; and fusing the semantic and edge prediction images of the farmland through post-processing to obtain a final farmland plot extraction result. The farmland plot extraction precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular, to a method and system for extracting farmland plots from high-resolution remote sensing images. Background Art

[0002] In the process of promoting China's agricultural modernization, the extraction of farmland plots provides key data support for agricultural meteorological forecasting, grain yield prediction, agricultural insurance, and scientific decision-making. Traditional farmland plot extraction methods mainly rely on manual operations, which not only consume a large amount of manpower and material resources but also are affected by human factors, resulting in limited reliability of the results. Remote sensing technology provides a new solution for farmland plot extraction. In particular, the continuous development of high-resolution remote sensing sensors makes it possible to efficiently and accurately extract large-scale farmland plots based on remote sensing data. Compared with medium- and low-resolution remote sensing images, high-resolution remote sensing images provide rich farmland radiation information, texture information, and boundary information, which can accurately reflect the spatial distribution and internal structure of plots. However, the complex changes in high-resolution images make plot extraction a challenging task. Therefore, how to efficiently and accurately extract farmland plot information from high-resolution remote sensing images has become an important research topic. In addition, the types of farmland plots are rich, the spatial structures are significantly different, and the planting patterns are diverse, such as dry land, paddy field, terraced field, sloping field, and greenhouse, etc. These all make the characteristics of farmland plots more complex and pose a huge challenge to high-precision extraction. Therefore, how to design a farmland plot extraction method suitable for China's complex planting structure has become a key problem to be solved urgently. Summary of the Invention

[0003] In order to solve the problem of low accuracy in extracting farmland plots in complex environments, the present invention provides a method for extracting farmland plots from high-resolution remote sensing images, which mainly includes:

[0004] S1: Collect high-resolution remote sensing satellite images, and perform data preprocessing and annotation on the images to obtain a dataset for extracting farmland plots from high-resolution remote sensing images;

[0005] S2: Divide the dataset into a training set and a test set for a farmland plot extraction model, and perform flipping, rotation, and scaling operations on the training set images to achieve data augmentation;

[0006] S3: Based on the dual U-Net network with semantic-edge mutual guidance, construct a high-resolution remote sensing image farmland plot extraction model SED-DUNet. Take the SED-DUNet model as the initial high-resolution remote sensing image farmland plot extraction model, including: two farmland plot extraction models SED-Net based on depth-supervised edge-semantic shared U-Net networks as the edge detection branch and the semantic segmentation branch respectively; add a multi-scale feature edge enhancement module SFM, a feature pyramid fusion module FEM for coupling edge enhancement, and an edge-assisted feature fusion semantic enhancement module EFM in the two parallel branches;

[0007] S4: Use the augmented training set, batch input the labeled training samples into the SED-DUNet model built in S3 for training and optimizing the network parameters. After multiple trainings, the network finally converges to obtain the best model;

[0008] S5: Input the test set into the best model obtained in S4, and obtain the predicted images of the semantics and edges of the farmland through testing;

[0009] S6: Fuse the predicted images of the semantics and edges of the farmland through post-processing to obtain the final farmland plot extraction result; where the post-processing is used to obtain the intersection difference between the semantic segmentation result and the edge detection result to obtain a high-precision plot extraction result with rich details.

[0010] A high-resolution remote sensing image farmland plot extraction system, the system includes:

[0011] A dataset generation module, used to collect the high-resolution remote sensing images to be detected, perform preprocessing operations, manual annotation and cropping into image blocks on the high-resolution remote sensing images to obtain a high-resolution remote sensing image farmland plot extraction dataset;

[0012] A dataset division module, used to divide the high-resolution remote sensing image farmland plot extraction dataset into a training set and a test set for the high-resolution remote sensing image farmland plot extraction model;

[0013] A model training module, used to construct a high-resolution remote sensing image farmland plot extraction model based on the dual U-Net network with semantic-edge mutual guidance, and use the training set to train the high-resolution remote sensing image farmland plot extraction model based on the dual U-Net network with semantic-edge mutual guidance; train the best model, and perform testing and threshold segmentation, and obtain the results of the semantic segmentation and edge detection branches at the same time;

[0014] Among them, the high-resolution remote sensing image farmland plot extraction model based on semantic-edge mutual guidance dual U-Net network includes: two farmland plot extraction models SED-Net based on depth-supervised edge semantic shared U-Net network, which are used as the edge detection branch and the semantic segmentation branch respectively; a multi-scale feature edge enhancement module SFM and a feature pyramid fusion module FEM for coupling edge enhancement; and an edge-assisted feature fusion semantic enhancement module EFM;

[0015] A model evaluation module, which is used to evaluate the training result according to the high-resolution remote sensing image farmland plot extraction test set, and obtain the final high-resolution remote sensing image farmland plot extraction model when the evaluation passes;

[0016] A farmland plot extraction module, which is used to fuse the predicted images of the semantics and edges of the farmland through post-processing to obtain the final farmland plot extraction result; where the post-processing is used to obtain the intersection difference between the semantic segmentation result and the edge detection result to obtain a high-precision plot extraction result with rich details.

[0017] A high-resolution remote sensing image farmland plot extraction device, which includes:

[0018] A high-resolution remote sensing image acquisition module, which is used to acquire the high-resolution remote sensing image to be detected;

[0019] A high-resolution remote sensing image farmland plot extraction result determination module, which is used to input the high-resolution remote sensing image to be detected into the trained high-resolution remote sensing image farmland plot extraction model, and at the same time obtain the results predicted by the semantic segmentation and edge detection branches, fuse the edge detection result and the semantic segmentation result, and finally obtain the high-resolution remote sensing image farmland plot extraction result.

[0020] An electronic device, characterized in that the electronic device includes:

[0021] At least one processor, which is used to call the program instructions in the memory to execute the cloud detection task, and

[0022] A memory communicatively connected to the at least one processor, which is used to store program instructions;

[0023] Among them, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the high-resolution remote sensing image farmland plot extraction method.

[0024] The beneficial effects brought by the technical solution provided by the present invention are as follows: The present invention collects high-resolution remote sensing images, and then performs preprocessing operations, manual annotation, and cropping into image blocks on the high-resolution remote sensing images in sequence to obtain a dataset for extracting farmland plots from high-resolution remote sensing images; divides the dataset for extracting farmland plots from high-resolution remote sensing images into a training set and a test set; constructs a model for extracting farmland plots from high-resolution remote sensing images based on a dual U-Net network with semantic-edge mutual guidance, and uses the training set to train the model for extracting farmland plots from high-resolution remote sensing images based on the initial dual U-Net network with semantic-edge mutual guidance; evaluates the training results according to the test set, and when the evaluation passes, obtains the best model for extracting farmland plots from high-resolution remote sensing images, realizing the improvement of the extraction accuracy of farmland plots from high-resolution remote sensing images and the improvement of the prediction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be further described below in conjunction with the drawings and embodiments. In the drawings:

[0026] Figure 1 is a flowchart of a method for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0027] Figure 2 is a schematic structural diagram of a system for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0028] Figure 3 is a schematic structural diagram of a device for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0029] Figure 4 is a schematic structural diagram of an electronic device for a method for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0030] Figure 5 is a schematic structural diagram of the SED-DUNet structure of a model for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0031] Figure 6 is a schematic structural diagram of the FLFF multi-scale skip link module in the edge extraction and semantic segmentation branches of the SED-DUNet model for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0032] Figure 7 is a schematic structural diagram of the HAF module in the edge extraction and semantic segmentation branches of the SED-DUNet model for extracting farmland plots from high-resolution remote sensing images in an embodiment of the present invention;

[0033] Figure 8It is a schematic structural diagram of the SFM module in the farmland plot extraction model SED-DUNet for high-resolution remote sensing images in the embodiments of the present invention;

[0034] Figure 9 It is a schematic structural diagram of the FEM module in the farmland plot extraction model SED-DUNet for high-resolution remote sensing images in the embodiments of the present invention;

[0035] Figure 10 It is a schematic structural diagram of the EFM module in the farmland plot extraction model SED-DUNet for high-resolution remote sensing images in the embodiments of the present invention;

[0036] Figure 11 It is a flowchart of a post-processing method for fusing edge detection results and semantic segmentation results in the embodiments of the present invention. Detailed implementation manners

[0037] For a clearer understanding of the technical features, objectives, and effects of the present invention, the specific implementation manners of the present invention will now be described in detail with reference to the accompanying drawings.

[0038] Embodiment 1

[0039] Please refer to Figure 1 , Figure 1 It is a flowchart of a method for extracting farmland plots from high-resolution remote sensing images in the embodiments of the present invention. This embodiment is applicable to the task of extracting farmland plots from high-resolution remote sensing images. This method can be executed by a device for extracting farmland plots from high-resolution remote sensing images based on a dual U-Net network with semantic-edge mutual guidance. The device for extracting farmland plots from high-resolution remote sensing images can be implemented in the form of hardware and / or software and can be configured in an electronic device such as a computer. The method specifically includes:

[0040] S1: Collect high-resolution remote sensing satellite images, and perform data preprocessing and annotation on the images to obtain a dataset for extracting farmland plots from high-resolution remote sensing images.

[0041] In the embodiments of the present invention, taking the data of the GF-2 satellite as the data source, to ensure the representativeness of the dataset, high-resolution remote sensing images containing different regions and different farmland types are selected, and the ENVI data processing software is used to preprocess the images, and the ArcMap is used to manually annotate the preprocessed images to construct a dataset for extracting farmland plots from high-resolution remote sensing images.

[0042] S1.1: Collect a dataset for extracting farmland plots from high-resolution remote sensing images containing different regions and different farmland types.

[0043] In the embodiments of the present invention, farmlands in different regions and of different types are selected to increase the diversity of samples, ensuring the representativeness of the dataset and the effectiveness of the farmland plot extraction model. Among them, Xinyang City, Henan Province, Zibo City, Shandong Province, and Ruian City, Zhejiang Province are respectively selected.

[0044] Most of the farmland plots in Xinyang City have regular shapes and appear as long and narrow rectangular structures on high-resolution remote sensing images. The area of each plot is small and they are densely distributed. Most of the farmland plots in Zibo City are flat and appear as large-scale regular rectangles on high-resolution remote sensing images, and the area of each plot is large. The agricultural planning in Ruian City features small-scale farmer farms, making most of the farmland plots in this area extremely irregular in shape on high-resolution remote sensing images, with extremely small areas and extremely dense distribution.

[0045] S1.2: Perform data preprocessing operations on the high-resolution remote sensing images.

[0046] In the embodiments of the present invention, the ENVI data image processing software is used to perform a series of preprocessing operations on each image to improve the image quality. Specifically, the steps include: first, performing RPC orthorectification on the 1-meter resolution image and the 4-meter resolution image, then fusing the multi-spectral band data and the panchromatic band data, and then performing fast atmospheric correction processing on the fused data.

[0047] S1.3: Annotate the high-resolution remote sensing images.

[0048] In the embodiments of the present invention, the ArcMap is used to perform manual annotation processing on the images, marking them into two categories: farmland plots and non-farmland plots. Among them, the pixels of "farmland plots" are assigned a value of 255, and the pixels of "non-farmland plots" are assigned a value of 0. And edge labels are generated, where the pixels of "edge" are assigned a value of 255, and the pixels of "non-edge" are assigned a value of 0.

[0049] S1.4: According to the high-resolution remote sensing image farmland plot extraction model based on the semantic-edge mutual guidance dual U-Net network, select, crop, and block the images and labels to obtain image patches with a size of 512×512.

[0050] S2: Divide the dataset for high-resolution remote sensing image farmland plot extraction into a training set and a test set for the farmland plot extraction model according to a ratio of 3:1. The dataset includes hyperspectral remote sensing image samples and labels, and perform flipping, rotation, and scaling operations on the training set images to achieve data augmentation; both the training set and the test set contain farmland plots of different regions and types to increase the representativeness of the dataset.

[0051] S3: Based on the dual U-Net network with semantic-edge mutual guidance, construct a high-resolution remote sensing image farmland plot extraction model SED-DUNet, use it as the initial high-resolution remote sensing image farmland plot extraction model, and train the initial high-resolution remote sensing image farmland plot extraction model with the training set.

[0052] The main network architecture part is as Figure 5 shown, including: two farmland plot extraction models SED-Net based on the edge semantic sharing U-Net network with deep supervision are used as the edge detection branch and the semantic segmentation branch respectively; a multi-scale feature edge enhancement module (Semantic-assistant Feature Module, SFM), a feature pyramid fusion module for coupling edge enhancement (Feature Enhance Module, FEM), and an edge-assisted feature fusion semantic enhancement module (Edge-assistant Feature Module, EFM) are designed and added in the middle of the two parallel branches.

[0053] Among them, the network part serving as the edge detection and semantic segmentation branches adopts the encoder-decoder network architecture. The encoder part uses the ResNet34 network pre-trained on the ImageNet dataset. The ResNet34 network adopts the residual block design, which can effectively extract the features of the image while solving the problem of gradient disappearance. By using the weights pre-trained on the ImageNet dataset, the ResNet34 network has relatively good initial transfer learning ability. Secondly, the middle part of the encoder and decoder adopts the Hybird Dilated Convolution (HDC) module to capture context information at different scales through multi-scale dilated convolution operations. Thirdly, the features of all levels of the encoder are fused through the FLFF (Full-level Feature Fusion, FLFF) module, as Figure 6 shown, and fed to the corresponding decoder levels. Then, five upsampling steps are performed using transposed convolution, and the corresponding side outputs of each decoder level are obtained to get the feature pyramid. Finally, a multi-scale feature fusion module based on the resolution attention mechanism (HierarchicalAttention Fusion, HAF), as Figure 7 shown, is used to adaptively fuse these side output results.

[0054] The Semantic-assistant Feature Module (SFM) is mainly designed to further reduce the generation of false edges to improve the accuracy of plot extraction. The module is designed to enhance the edges of farmland plots and mask false edges by introducing semantic information into the edge detection task. By using semantic information at different levels to define the farmland range for the edge information at the corresponding level, the correct target edge features within the farmland are enhanced, and the noise edge features outside the farmland are suppressed, greatly solving the problem of frequent occurrence of false edges inherent in traditional edge extraction networks. The specific structure of the SFM module is as shown in Figure 8 shown below.

[0055] In SFM, the edge feature f e at a certain level of the edge branch decoder and the semantic feature f e at the same level on the semantic branch decoder are used as the given inputs of SFM. First, f s is passed through a 1×1 convolution and the sigmoid function to generate the attention mask c s corresponding to the farmland range at this level. After multiplying the attention mask c i with the edge feature f i , the enhanced farmland edge feature is obtained e . This process is expressed as: This process is expressed as:

[0056] c i =σ(F conv1 (f s ))

[0057]

[0058] where σ represents the sigmoid function;

[0059] Subsequently, it is respectively input into the three-branch structure of horizontal pooling, vertical pooling, and the HDC module. The horizontal pooling branch consists of a horizontal pooling layer, a convolutional layer with a kernel size of 3×3, a BN layer, a non-linear activation layer ReLU, and an upsampling layer for restoring the input size of the image; the vertical pooling branch consists of a vertical pooling layer, a 3×3 convolutional layer, a BN layer, a non-linear activation layer ReLU, and an upsampling layer for restoring the input size of the image; the middle branch is the HDC module that uses hybrid dilated convolution operations to expand the receptive field. After splicing the outputs of the three branches together, a 1×1 convolution is performed on them to obtain the final edge feature f enhanced by semantic assistance: i :

[0060]

[0061] Among them, F conv1 represents a convolution operation with a convolution kernel size of 1, and F hpool represents the operation of the vertical pooling branch, and F wpool represents the operation of the horizontal pooling branch, and F hdc represents the operation of the HDC module, and concat represents the concatenation operation.

[0062] A Feature Pyramid Fusion Module for Edge Enhancement (FEM) is used, and its specific structure is as Figure 9 shown. Its design purpose is to improve the utilization rate of deep global features by transmitting deep edge information to the shallow layer in a top-down and gradually enhanced manner, thereby improving the final edge extraction result. Specifically:

[0063] First, the side output result obtained by performing a 1×1 convolution on a certain layer of features of the decoder in the edge branch is input into the EEB (Edge Enhance Block) block. Through a series of operations such as the sobel operator, the edge of the edge feature is obtained. Subsequently, it is added to the original edge feature to enhance the edge feature, and then a 1×1 convolution operation is performed on the result of the previous sum to obtain the edge feature of the corresponding layer enhanced by the EEB block. The process is expressed as:

[0064]

[0065] Among them, is the final side output result of the i-th layer of the decoder after enhancement, is the output result of the i-th layer passing through the EEB module, is the side output result of the (i - 1)-th layer passing through the EEB module.

[0066] Moreover, except that the bottom layer of the decoder directly uses the edge feature map obtained by the SFM block of the same layer and enhances it through the FEM block as the side output, the side outputs of other layers are all the fusion results of the edge feature map obtained by the SFM block of the same layer and the result obtained by the FEM block that transmits features from bottom to top as the side output of this layer of the edge branch decoder.

[0067] The FEM module continuously enhances the edge deep information and transmits it to the upper layer, while enabling the shallow layer of the model to also pay attention to the global features, so that the deeper, more advanced, and more robust global features have a more positive guiding effect on the entire process of model training, improving the feature discrimination ability and robustness of the model as a whole, and thus increasing the extraction accuracy of the model for farmland plots.

[0068] An Edge-Assisted Semantic Feature Fusion Enhancement Module (EFM), its specific structure is as Figure 10As shown, this module is located at the end of SED-DUNet. Its design purposes are, firstly, to achieve the general purpose of using edge information to improve the classification of feature edge pixels in semantic segmentation, and secondly, specifically for the farmland plot extraction task, the fusion with edge information can make the final output result of the semantic branch retain a certain number of edge details internally, which can help the broken edges in the edge extraction result to be closed to a certain extent during the post-processing process. By fusing the output result of the edge branch with the output result of the semantic branch, the classification of feature edge pixels in semantic segmentation is improved, thereby improving the accuracy of the model in pixel-level metrics for the farmland plot extraction task. At the same time, a certain number of edge details are added to the final semantic feature output of the network, which can help some broken edges to be closed during the post-processing process, thus improving the accuracy of the model in plot-level metrics.

[0069] S4: Batch input the labeled training sample dataset into the farmland plot extraction model built in step S3, set the loss function, train and optimize the network parameters, and after multiple trainings, achieve the final convergence of the network to obtain the best trained model.

[0070] In the embodiment of the present invention, the training set is input into the high-resolution remote sensing image farmland plot extraction model of the semantic-edge mutually guiding dual U-Net network constructed in step S3. The network weights are initialized at, and at the same time, the farmland plot prediction probability and edge prediction probability map of the training samples are output, and the network loss function is calculated and backpropagated.

[0071] In the embodiment of the present invention, the cross-entropy loss function is used to train the network, as shown in the formula:

[0072]

[0073] where, and q i respectively represent the predicted label and the true label of the i-th sample, and m represents the number of training samples.

[0074] After obtaining m side outputs, the method of summing and then averaging is used to obtain the loss of the side output results, as shown in the formula:

[0075]

[0076] where l side represents the average value of the sum of all side output losses, l BCE represents the cross-entropy loss function, Y represents the original image, represents the side output image of the i-th layer; w represents the width of the image, h represents the height of the image; L ij represents the true label corresponding to the i-th layer of the edge and the j-th layer of the semantics, Denote the output image of the i-th layer of the edge and the j-th layer of the semantics.

[0077] Subsequently, for the semantic segmentation task branch, first use the HAF module to fuse the m side outputs to obtain the preliminary fusion feature result, and then use the EFM to fuse the preliminary fusion feature result with the edge feature to obtain the final semantic feature result. Finally, calculate the loss of the fusion result with the ground truth, which can be expressed as:

[0078]

[0079] where denotes the fusion image loss, l BCE denotes the cross-entropy loss function, Y denotes the original image, denotes the fused image obtained through the HAF module.

[0080] Therefore, the final loss of the semantic segmentation branch can be expressed as:

[0081]

[0082] where w side and w fuse are the weight values corresponding to the losses. Referring to the general parameter settings, they are both set to 0.5 whether in the edge detection task or the semantic segmentation task.

[0083] For the edge detection task branch, the loss of the fusion result calculated with the ground truth using the feature result obtained by the FEM module can be expressed as:

[0084]

[0085] where denotes the fusion image loss, l BCE denotes the cross-entropy loss function, Y denotes the original image, denotes the final result obtained through the FEM module.

[0086] Therefore, the final loss of the edge detection task branch can be expressed as:

[0087]

[0088] where w side and w fem are the weight values corresponding to the losses. Referring to the general parameter settings, they are both set to 0.5 whether in the edge detection task or the semantic segmentation task.

[0089] Therefore, the final loss of SED-DUNet can be expressed as:

[0090]

[0091] Among them, represents the edge detection branch loss, represents the semantic segmentation branch loss, w1 is the weight of the edge detection branch, and w2 is the weight of the semantic segmentation branch. In this embodiment, w1 is set to 0.2, w1 is set to 0.8, and w2 is set to 0.2.

[0092] The Adam optimizer is selected for weight optimization, the batch size is set to 8, the initial learning rate is set to 0.0002, and the maximum number of training epochs is set to 200. During the training phase, the training weights at the lowest loss value are retained. When the training loss value does not decrease for 5 consecutive epochs, the learning rate is reduced to one-fifth of the current learning rate. When the training loss value does not decrease for 10 consecutive epochs, the training is terminated, and the best model is finally obtained.

[0093] S5: Input the test image into the best model obtained in step S4, and respectively obtain the semantic segmentation result and the edge detection result of the farmland plot through prediction.

[0094] In the embodiment of the present invention, the test image is input into the best model trained in step S4 for prediction, and finally the semantic segmentation result and the edge detection result are output.

[0095] S6: Fuse the prediction results of semantics and edges through post-processing to obtain the final farmland plot extraction result. The main purpose of the post-processing process is to obtain the intersection difference between the semantic segmentation result and the edge detection result to obtain a high-precision plot extraction result with rich details. The specific steps are as follows:

[0096] First, extract the centerline from the obtained edge detection result. The centerline extraction operation can refine the farmland edge, making the final extraction result more visually in line with the actual situation. In the subsequent process of making the intersection difference, using the single-pixel centerline to replace the original edge detection result to participate in the operation can also greatly reduce the loss of farmland pixels. Secondly, subtract the semantic segmentation result from the edge detection centerline extraction result. After this operation, there are three pixel values in the intermediate result: -255, 0, 255. Among them, -255 represents the edge outside the farmland area (red part), 0 represents the edge or background within the farmland area (black part), and 255 represents the farmland itself (white part). Finally, set -255 to 0, and then perform hole filling and removal of small fragments on the intermediate result to obtain the final result. Setting -255 to 0 discards the edge outside the farmland area. Hole filling and removal of small fragments are performed to improve the quality and accuracy of the final plot extraction effect. The flowchart is as Figure 11 shown.

[0097] Select the high-resolution remote sensing image farmland plot extraction dataset constructed in step S1 for the farmland plot extraction experiment, and select a variety of models as comparison methods.

[0098] Among them, the overall quantitative evaluation results of SED-DUNet and other farmland plot extraction methods at the pixel scale are shown in the following table:

[0099]

[0100] The overall quantitative evaluation results of SED-DUNet and other farmland plot extraction methods at the plot scale are shown in the following table:

[0101]

[0102] As can be seen from the table, the SED-DUNet proposed by the present invention is not only superior to other models in terms of the OA, Precision, F1-score and IOU indicators at the pixel level. Among them, Precision, OA, F1-score, and IOU are 93.63%, 91.26%, 93.01%, and 86.91% respectively, which are 0.53%, 0.81%, 0.6%, and 1.02% higher than the sub-optimal method HRNet-OCR. The indicators at the plot level are also superior to other models. At the plot level, SED-DUNet is 27.46% higher than the best-performing UNet-RCF model among other models in terms of Precision, 36.82% higher in Recall, and 32.11% higher in F1-score. This proves the effectiveness and superiority of the method proposed by the present invention.

[0103] The technical solution of this embodiment is to collect high-resolution remote sensing images, perform data preprocessing and manual annotation on the images to obtain a dataset for high-resolution remote sensing image farmland plot extraction; divide the dataset into a training set and a test set; construct a high-resolution remote sensing image farmland plot extraction model based on a dual U-Net network with semantic-edge mutual guidance; batch input training samples into the model, train and optimize the network parameters, and after multiple trainings, the network achieves final convergence to obtain the best model; input the test set into the trained model, and obtain the semantic segmentation probability image and the edge detection probability image through testing; set thresholds and perform binary threshold segmentation on the image semantic segmentation probability image and the edge detection probability respectively to obtain the semantic segmentation and edge detection prediction images, and through a series of post-processing, fuse the edge detection results and the semantic segmentation results to obtain the final farmland plot extraction result.

[0104] A high-resolution remote sensing image farmland plot extraction system includes a high-resolution remote sensing image farmland plot extraction model training device (such as Figure 2as shown) and a high-resolution remote sensing image farmland plot extraction model device (such as Figure 3 shown).

[0105] Embodiment 2

[0106] Figure 2 is a schematic structural diagram of a high-resolution remote sensing image farmland plot extraction system based on a semantic-edge mutually guiding dual U-Net network according to Embodiment 1 of the present invention. As shown in the figure, the system includes:

[0107] A dataset generation module, configured to collect high-resolution remote sensing images, perform preprocessing operations and manual annotation on the images, and obtain a dataset for high-resolution remote sensing image farmland plot extraction;

[0108] A dataset division module, configured to divide the high-spectral remote sensing image farmland plot extraction dataset into a training set and a test set;

[0109] A model training module, configured to construct a high-resolution remote sensing image farmland plot model based on a semantic-edge mutually guiding dual U-Net network, and batch input training set samples for model training, and continuously adjust the network model parameters to obtain an optimal network model.

[0110] A model evaluation module, inputting test set samples to test the trained model, and performing threshold segmentation on the predicted probability images of the farmland plots and edges obtained through the test to obtain the predicted results of the farmland plots and edges, fusing the semantic segmentation results and the edge detection results through post-processing to obtain the farmland plot prediction results, and performing accuracy evaluation on it. When the evaluation passes, the final high-resolution remote sensing image farmland plot extraction model is obtained.

[0111] Optionally, the model training module includes:

[0112] An edge extraction branch, composed of SED-Net, for extracting edge information;

[0113] A semantic segmentation branch, composed of SED-Net, for extracting plot semantic information;

[0114] A semantic-assisted multi-scale feature edge enhancement module, the SFM module, for introducing semantic information to improve edge detection;

[0115] A feature pyramid edge enhancement module, the FEM module, for fusing global features in edge detection;

[0116] An edge-assisted feature fusion semantic enhancement module, the EFM module, for introducing edge information to improve semantic segmentation;

[0117] Optionally, the model evaluation module includes:

[0118] An evaluation unit for evaluating preset indicators based on the training results of farmland plot extraction from a test set;

[0119] Among them, after testing with the test set, through the set segmentation threshold, binary threshold segmentation is performed on the predicted farmland plot image and the edge detection image, and through post-processing, they are fused to obtain the final farmland plot extraction result. Finally, the training result is evaluated by calculating preset indicators;

[0120] The preset indicators include pixel-level evaluation indicators and plot-level evaluation indicators. Among them, the pixel-level evaluation indicators include precision, accuracy, recall, F1-score, and intersection over union; the plot-level evaluation indicators are precision, recall, and F1-score. In the plot-level accuracy evaluation calculation, only when the IOU between the detected farmland plot area and the ground truth annotation area is greater than or equal to 0.5, it will be considered a correctly detected farmland plot. Conversely, if the IOU between the detected farmland plot area and the ground truth annotation area is less than 0.5, it will be considered a misdetected farmland plot.

[0121] The high-resolution remote sensing image farmland plot extraction system based on the semantic-edge mutual guidance dual U-Net network provided by the embodiments of the present invention can execute the high-spectral remote sensing image cloud detection model training method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0122] Embodiment 3

[0123] Figure 3 It is a schematic structural diagram of a high-resolution remote sensing image farmland plot extraction device. As Figure 3 shown, the device includes:

[0124] A high-resolution remote sensing image acquisition module for acquiring the high-resolution remote sensing image to be detected;

[0125] A high-resolution remote sensing image farmland plot extraction result determination module for inputting the high-resolution remote sensing image to be detected into the best farmland plot extraction model trained by the high-resolution remote sensing image farmland plot extraction method based on the semantic-edge mutual guidance dual U-Net network provided by any embodiment of the present invention, obtaining the results predicted by the semantic segmentation and edge detection branches at the same time, fusing the edge detection result and the semantic segmentation result, and finally obtaining the high-resolution remote sensing image farmland plot extraction result.

[0126] The high-resolution remote sensing image farmland plot extraction model device based on the semantic-edge mutual guidance dual U-Net network provided by the embodiments of the present invention can execute the high-resolution remote sensing image farmland plot extraction method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0127] Example 4

[0128] Figure 4 FIG. 1 is a schematic structural diagram of an electronic device 10 according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital assistants, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0129] As Figure 4 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0131] The processor 11 may be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the methods and processes described above.

[0132] In some embodiments, the method for extracting farmland plots based on the dual U-Net network with semantic-edge mutual guidance can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the above-described method for detecting clouds in hyperspectral images based on the spatial-spectral residual U-Net network can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the method for detecting clouds in hyperspectral remote sensing images in any other suitable manner (e.g., by means of firmware).

[0133] The various embodiments of the systems and techniques described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] In order to provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0137] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0138] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0139] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for extracting farmland plots from high-resolution remote sensing images, characterized in that, Including: S1: Collect high-resolution remote sensing satellite images, perform data preprocessing and annotation on the images, and obtain a dataset for high-resolution remote sensing image farmland plot extraction; S2: Divide the dataset into a training set and a test set for the farmland plot extraction model, and perform flipping, rotation, and scaling operations on the training set images to achieve data augmentation; S3: Construct a high-resolution remote sensing image farmland plot extraction model SED-DUNet based on the semantic-edge mutually guided dual U-Net network. Take the SED-DUNet model as the initial high-resolution remote sensing image farmland plot extraction model, including: two farmland plot extraction models SED-Net based on depth-supervised edge semantic sharing U-Net networks as the edge detection branch and the semantic segmentation branch respectively; add a multi-scale feature edge enhancement module SFM, a feature pyramid fusion module FEM for coupling edge enhancement, and an edge-assisted feature fusion semantic enhancement module EFM in the two parallel branches; S4: Use the data-augmented training set, batch input the labeled training samples into the SED-DUNet model built in S3 for training and optimize the network parameters. After multiple trainings, the network finally converges to obtain the best model; S5: Input the test set into the best model obtained in S4, and obtain the predicted images of the semantics and edges of the farmland through testing; S6: Fuse the predicted images of the semantics and edges of the farmland through post-processing to obtain the final farmland plot extraction result; where the post-processing is used to obtain the intersection difference between the semantic segmentation result and the edge detection result to obtain a high-precision plot extraction result with rich details.

2. The method for extracting farmland plots from high-resolution remote sensing images according to claim 1, characterized in that, In S1, the construction of the dataset includes the following steps: S1.1: Collect high-resolution remote sensing images containing different types of farmland plots in different regions; S1.2: Perform preprocessing on the images, including radiometric correction and fast atmospheric correction; S1.3: Use ArcMap to perform manual annotation processing on the preprocessed high-resolution remote sensing images, mark them as two categories: farmland plots and non-farmland plots, where the pixels of "farmland plots" are assigned a value of 255, and the pixels of "non-farmland plots" are assigned a value of 0; and generate edge labels, where the pixels of "edges" are assigned a value of 255, and the pixels of "non-edges" are assigned a value of 0; S1.4: According to the SED-DUNet model, select, crop, and block the images and labels to obtain image blocks with a size of 512×512, that is, obtain the dataset.

3. A method for extracting farmland plots from high-resolution remote sensing images according to claim 1, characterized in that, Including: In S3, multiple modules are designed between two parallel SED-Net network decoders to enable interaction between the two networks. First, through the Semantic-Assisted Multi-Scale Feature Edge Enhancement Module (SFM), semantic features at a certain level are used to enhance the edge features at the corresponding level. Subsequently, the edge features enhanced by SFM are fed into the Feature Pyramid Fusion Module with Coupled Edge Enhancement (FEM), and the enhanced edge features are continuously passed from the deep layer to the shallow layer. Finally, the semantic fusion features obtained through the Multi-Scale Feature Fusion Module (HAF) and the edge extraction features obtained from FEM are fed into the Edge-Assisted Feature Fusion Semantic Enhancement Module (EFM) to use the edge features to enhance the semantic features, and at the same time, the semantic segmentation result and the edge detection result are obtained.

4. A method for extracting farmland plots from high-resolution remote sensing images according to claim 3, characterized in that, In the SFM of S3, the edge feature f at a certain level of the edge branch decoder e and the semantic feature f at the same level on the semantic branch decoder e are used as the given input of the SFM; First, f s is passed through a 1×1 convolution and the sigmoid function to generate an attention mask c for the corresponding farmland range at this level s , and the attention mask c i is multiplied by the edge feature f i to obtain an enhanced farmland edge feature e This process is expressed as: This process is expressed as: c i = σ(F conv1 (f s )) Where σ represents the sigmoid function; Subsequently, are respectively input into a three-branch structure of a horizontal pooling, a vertical pooling, and an HDC module. The horizontal pooling branch consists of a horizontal pooling layer, a 3×3 convolutional layer, a BN layer, a non-linear activation layer ReLU, and an upsampling layer for restoring the input size of the image; the vertical pooling branch consists of a vertical pooling layer, a 3×3 convolutional layer, a BN layer, a non-linear activation layer ReLU, and an upsampling layer for restoring the input size of the image; the middle branch is an HDC module that uses hybrid dilated convolution operations to expand the receptive field; after splicing the outputs of the three branches together, a 1×1 convolution is performed on them to obtain the final edge feature f that uses semantic assistance for edge enhancement i : Among them, F conv1 represents a convolution operation with a convolution kernel size of 1, F hpool represents the operation of the vertical pooling branch, F wpool represents the operation of the horizontal pooling branch, F hdc represents the operation of the HDC module, and concat represents the concatenation operation.

5. A method for extracting farmland plots from high - resolution remote sensing images according to claim 4, characterized in that, In the FEM of S3, first, the side output result obtained by performing a 1×1 convolution on the feature at a certain level of the decoder in the edge branch is input into the EEB block to obtain the edge of the edge feature. Subsequently, the edge is added to the original edge feature to enhance the edge feature, and a 1×1 convolution operation is performed on the enhanced edge feature to obtain the edge feature at the corresponding level after being enhanced by the EEB block. The process is expressed as: Among them, is the final edge-side output result after enhancement of the i-th layer of the decoder, is the output result of the i-th layer passing through the EEB module, is the side output result of the (i - 1)-th layer passing through the EEB module; Except that the bottom layer of the decoder directly uses the edge feature map obtained by the SFM block at the same level and enhances it through the FEM block as the side output, the side outputs of other levels are the fusion results of the edge feature map obtained by the SFM block at the same level and the results obtained by the FEM block that transmits features from bottom to top as the side output of this level of the edge branch decoder. The EFM module is located at the end of the SED-DUNet. By fusing the output results of the edge branch and the semantic branch, it improves the classification of feature edge pixels in semantic segmentation, thereby improving the accuracy of the model in pixel-level metrics in the farmland plot extraction task. At the same time, it adds a certain number of edge details to the final semantic feature output of the network, which can help some broken edges close during the post-processing process, thus improving the accuracy of the model in plot-level metrics.

6. The method for extracting farmland plots from high-resolution remote sensing images according to claim 1, wherein In S4, the training sample data of high-resolution remote sensing images and the corresponding sample label data are batch-input into the high-resolution remote sensing image farmland plot extraction model constructed in S3, and the plot and edge prediction probabilities of the training samples are output. The loss function is calculated and backpropagated to modify the parameters. The loss function is expressed as follows: Among them, represents the edge detection branch loss, represents the semantic segmentation branch loss, w1 is the weight of the edge detection branch, and w2 is the weight of the semantic segmentation branch; Both the edge and semantic branches are trained using the cross-entropy loss function, and its formula is as follows: Among them, and q i respectively represent the predicted label and the true label of the i-th sample, and m represents the number of training samples; After obtaining m side outputs, the method of summing and then taking the average is used to obtain the loss of the side output result, as shown in the formula: Among them represents the average value of the sum of all side output losses, represents the cross-entropy loss function, Y represents the original image, represents the side output image of the i-th layer, w represents the width of the image, and h represents the height of the image; L ij represents the true label corresponding to the i-th layer of the edge and the j-th layer of semantics, represents the side output image of the i-th layer of the edge and the j-th layer of semantics; Regarding the semantic segmentation task branch, since the HAF module is first used to fuse the m side outputs to obtain the preliminary fusion feature result, and then the EFM is used to fuse the preliminary fusion feature result with the edge feature to obtain the final semantic feature result, and finally the loss of the fusion result is calculated with the ground truth. The loss is expressed as: Among them represents the fused image loss represents the cross-entropy loss function, Y represents the original image represents the fused image obtained through the HAF module; Therefore, the final loss of the semantic segmentation branch is expressed as: where w side and w fuse are the weight values corresponding to the losses; Regarding the edge detection task branch, the loss of the fusion result calculated from the feature result obtained using the FEM module and the ground truth is expressed as: Among them represents the fused image loss represents the cross-entropy loss function, Y represents the original image represents the final result obtained through the FEM module; Therefore, the final loss of the edge detection task branch is expressed as: where w side and w fem are the weight values corresponding to the losses.

7. A method for extracting farmland plots from high-resolution remote sensing images according to claim 1, characterized in that, In S6, centerline extraction is performed on the edge detection result, the difference is taken between the semantic segmentation result and the centerline extraction result of the edge detection, the result after taking the difference is reclassified, and finally hole filling and elimination of small fragments are performed. After fusion processing, a binary result for the extraction of farmland plots in high-resolution remote sensing images is obtained.

8. A high-resolution remote sensing image farmland plot extraction system, characterized in that, The system includes: A dataset generation module for collecting high-resolution remote sensing images to be detected, performing preprocessing operations, manual annotation, and cropping into image patches on the high-resolution remote sensing images to obtain a dataset for the extraction of farmland plots in high-resolution remote sensing images; A dataset division module for dividing the dataset for the extraction of farmland plots in high-resolution remote sensing images into a training set and a test set for the model for the extraction of farmland plots in high-resolution remote sensing images; A model training module for constructing a model for the extraction of farmland plots in high-resolution remote sensing images based on a dual U-Net network with semantic-edge mutual guidance, and training the model for the extraction of farmland plots in high-resolution remote sensing images based on the dual U-Net network with semantic-edge mutual guidance using the training set; training the best model, performing testing and threshold segmentation, and obtaining the results of the semantic segmentation and edge detection branches at the same time; Among them, the model for the extraction of farmland plots in high-resolution remote sensing images based on the dual U-Net network with semantic-edge mutual guidance includes: two farmland plot extraction models SED-Net based on deep-supervised edge-semantic shared U-Net networks as the edge detection branch and the semantic segmentation branch respectively; a multi-scale feature edge enhancement module SFM and a feature pyramid fusion module FEM for coupling edge enhancement; and an edge-assisted feature fusion semantic enhancement module EFM; A model evaluation module for evaluating the training results according to the test set for the extraction of farmland plots in high-resolution remote sensing images, and obtaining the final model for the extraction of farmland plots in high-resolution remote sensing images when the evaluation passes; A farmland plot extraction module for fusing the predicted images of the semantics and edges of the farmland through post-processing to obtain the final farmland plot extraction result; where the post-processing is used to obtain the intersection difference between the semantic segmentation result and the edge detection result to obtain a high-precision plot extraction result with rich details.

9. An apparatus for extracting farmland plots from high-resolution remote sensing images, characterized in that, The device includes: A high-resolution remote sensing image acquisition module for acquiring high-resolution remote sensing images to be detected; A module for determining the extraction result of farmland plots in high-resolution remote sensing images for inputting the high-resolution remote sensing images to be detected into the trained model for the extraction of farmland plots in high-resolution remote sensing images, obtaining the results predicted by the semantic segmentation and edge detection branches at the same time, and fusing the edge detection result and the semantic segmentation result to finally obtain the extraction result of farmland plots in high-resolution remote sensing images.

10. An electronic device, characterized in that, The electronic device includes: At least one processor for calling program instructions in the memory to execute the cloud detection task, and a memory communicatively connected to the at least one processor for storing program instructions; Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the high-resolution remote sensing image farmland plot extraction method according to any one of claims 1-7.

Images