Farmland identification method, device and storage medium based on dual-stream deep network

Abstract

The present invention provides a farmland identification method, device and storage medium based on a dual-stream deep network, the method comprising: obtaining a first training set and a second training set; constructing a farmland semantic segmentation network based on D-LinkNet; using the first training set to train the farmland semantic segmentation network to obtain a trained farmland semantic segmentation network; constructing a farmland edge detection network based on DexiNed; using the second training data set to train the farmland edge detection network to obtain a trained farmland edge detection network; inputting the remote sensing image of the farmland to be identified into the trained farmland semantic segmentation network and the trained farmland edge detection network respectively, performing full-image prediction based on an expansion prediction method, obtaining semantic segmentation results and edge detection results; performing intersection difference processing on the semantic segmentation results and the edge detection results, obtaining farmland plot identification results. The present invention improves the accuracy of farmland plot identification based on a dual-stream deep network, and realizes high-precision identification of farmland plots.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a farmland recognition method, device and storage medium based on a dual-stream deep network. Background Art

[0002] Traditional methods for identifying farmland plots are generally divided into pixel-based identification methods and object-oriented methods. Although the pixel-based identification method can quickly obtain farmland spatial distribution information in a large range, it does not consider the boundaries between the ridges of the plots and cannot meet the needs of land rights confirmation, cadastral management, and refined management of agricultural information. In addition, the pixel-based method does not fully consider the spatial context information of the image, and the recognition results are often interfered by "salt and pepper noise", and the problem of missing plots is serious, which makes it difficult to meet the application needs of precision agriculture. The object-oriented method uses objects as the basic unit, comprehensively utilizes spectral features, texture features, and topological information for farmland identification, and couples multi-scale information of farmland. Compared with the pixel-based method, it can effectively improve the accuracy of farmland identification, alleviate "salt and pepper noise", and greatly improve the quality of thematic mapping. It is currently the most widely used method. However, most of the above methods can only extract shallow spatial spectral features of farmland, and it is difficult to accurately describe the essential attributes of farmland plots. The robustness is insufficient in complex environments, and it is difficult to achieve refined identification of farmland plots. Summary of the invention

[0003] The main technical problem solved by the present invention is to achieve stable and refined recognition of shallow spatial spectrum characteristics and essential attributes of farmland plots in complex environments. The technical solution adopted by the present invention is to provide a farmland recognition method, device and storage medium based on a dual-stream deep network.

[0004] According to a first aspect of the present invention, a method for farmland recognition based on a dual-stream deep network is provided, comprising the following steps:

[0005] Obtain a first training set and a second training set;

[0006] Build a farmland semantic segmentation network based on D-LinkNet;

[0007] Using the first training set to train the D-LinkNet-based farmland semantic segmentation network, after the training is completed, a trained farmland semantic segmentation network is obtained;

[0008] Construct a farmland edge detection network based on DexiNed;

[0009] Using the second training data set to train the DexiNed-based farmland edge detection network, after the training is completed, a trained farmland edge detection network is obtained;

[0010] The remote sensing image of the farmland to be identified is input into the trained farmland semantic segmentation network and the trained farmland edge detection network respectively, and the whole image prediction is performed based on the expansion prediction method to obtain the semantic segmentation result and the edge detection result;

[0011] The semantic segmentation result and the edge detection result are subjected to intersection difference processing to obtain a farmland plot recognition result.

[0012] Furthermore, the step of obtaining the first training set and the second training set includes:

[0013] Obtain remote sensing images of farmland scenes;

[0014] Performing vector processing on the remote sensing image to obtain a vector layer;

[0015] Convert the vector layer into a raster layer and perform binarization;

[0016] Create a first data set based on the binarized raster layer;

[0017] Create a second data set based on the binarized raster layer;

[0018] Acquire a first training set from the first data set;

[0019] A second training set is obtained from the second data set.

[0020] Furthermore, the farmland semantic segmentation network based on D-LinkNet includes: an encoder, a hole convolution layer and a decoder;

[0021] The encoder is connected to the atrous convolutional layer;

[0022] The decoder is connected to the encoder and the hole convolution layer respectively;

[0023] The dilated convolution is used to expand the receptive field of the feature points.

[0024] Furthermore, the loss function of the farmland semantic segmentation network based on D-LinkNet is composed of a cross entropy loss function L and PointLoss, and the formula is as follows:

[0025]

[0026] Pointloss=L(Y PointID ,P PointID )

[0027] PointID=Max N (-1*|mask1-mask2|)

[0028] Where: yi Indicates the label of sample i, the positive class is 1, the negative class is 0, and p i represents the probability that sample i is predicted as the positive class, PointID represents the location of the point, mask1 represents the feature value predicted as farmland, mask2 represents the feature value predicted as non-farmland, Max N Indicates taking the first N largest numbers, Y PointID Indicates the label value of the PointID pixel position, P PointID Indicates the predicted probability of the PointID pixel location.

[0029] Furthermore, the farmland edge detection network based on DexiNed includes: Dexi sub-network and UB sub-network;

[0030] The Dexi sub-network consists of 6 main modules, which are used to extract the features of the image to obtain the feature map;

[0031] The UB subnetwork upsamples the feature maps obtained by each main module to the original image size, and achieves accurate extraction of boundary information by fusing feature maps obtained by upsampling at different levels.

[0032] Furthermore, the step of performing intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result includes:

[0033] Based on the intersection difference processing, the semantic segmentation result and the edge detection result are fused, and the intersection of the semantic segmentation result and the edge detection result is subtracted based on the semantic segmentation result to obtain the farmland plot recognition result.

[0034] Furthermore, after the step of performing intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result, the method further includes:

[0035] Establish an expert-based post-processing mechanism;

[0036] The post-processing mechanism is used to eliminate small blocks, eliminate holes, and simplify edges of the farmland plot identification results to obtain optimized farmland plot identification results.

[0037] According to a second aspect of the present invention, a farmland identification device based on a dual-stream deep network is provided, the farmland identification device is used to implement any of the farmland identification methods based on a dual-stream deep network, and the farmland identification device includes the following modules:

[0038] A training set acquisition module, used to acquire a first training set and a second training set;

[0039] Semantic segmentation network construction module, used to build a farmland semantic segmentation network based on D-LinkNet;

[0040] A semantic segmentation network training module, used to train the D-LinkNet-based farmland semantic segmentation network using the first training set, and after the training is completed, obtain a trained farmland semantic segmentation network;

[0041] Edge detection network building module, used to build a farmland edge detection network based on DexiNed;

[0042] An edge detection network training module is used to train the DexiNed-based farmland edge detection network using the second training data set, and after the training is completed, a trained farmland edge detection network is obtained;

[0043] A full-image prediction module is used to input the farmland remote sensing image to be identified into the trained farmland semantic segmentation network and the trained farmland edge detection network respectively, perform full-image prediction based on the expansion prediction method, and obtain semantic segmentation results and edge detection results;

[0044] The intersection difference processing module is used to perform intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result.

[0045] Furthermore, the farmland identification device further includes:

[0046] The post-processing module is used to establish an expert-based post-processing mechanism, and use the post-processing mechanism to eliminate small blocks, eliminate holes, and simplify edges of the farmland plot identification results to obtain optimized farmland plot identification results.

[0047] According to the third aspect of the present invention, a storage medium is also provided, which is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is run, it is used to implement the steps of any of the methods for farmland identification based on a dual-stream deep network.

[0048] The beneficial effects of the technical solution provided by the present invention are:

[0049] (1) The farmland parcel recognition task is transformed into a semantic segmentation problem and an edge detection problem. A two-stream deep network that couples the semantic segmentation and edge detection models is constructed for farmland parcel recognition. By designing the intersection difference operation, the farmland edge recognition results are used to assist the refinement and purification of the farmland parcels in the farmland semantic segmentation results, thereby achieving high-precision detection of farmland parcels.

[0050] (2) Taking into full consideration the characteristics of farmland plots, the D-LinkNet and DexiNed networks in the field of computer vision are applied to farmland semantic segmentation and edge detection, and transformed into a model suitable for farmland feature extraction and recognition. The advantages of the two models are combined to achieve fast and accurate recognition of farmland plots. In addition, referring to the central idea of ​​the PointRend network, a PointLoss is added on the basis of the cross entropy loss function used in the original D-LinkNet network to enhance the recognition effect of the boundary.

[0051] (3) In order to improve efficiency and avoid errors caused by cropping and then stitching the image, the full-image expansion prediction method is adopted. The original image can be directly input to obtain an output result that matches the original image. This not only saves many manual processing steps, but also improves the accuracy of the results.

[0052] (4) A complete set of post-processing mechanisms based on expert rules was established to further improve the accuracy of farmland plot identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0054] Figure 1 It is a technical roadmap of a farmland identification method based on a dual-stream deep network of the present invention;

[0055] Figure 2 It is a data example diagram of the present invention;

[0056] Figure 3 It is a network structure diagram of the farmland semantic segmentation network based on D-LinkNet of the present invention;

[0057] Figure 4 It is the application process of Pointloss of the present invention;

[0058] Figure 5 It is a schematic diagram of PointID of the present invention;

[0059] Figure 6 It is a network structure diagram of the farmland edge detection network based on DexiNed of the present invention;

[0060] Figure 7 It is an example of the output of each layer of the DexiNed network of the present invention;

[0061] Figure 8 It is a schematic diagram of expansion prediction of the present invention;

[0062] Fig. 9 It is a comparison diagram of expansion prediction of the present invention;

[0063] Fig.10is a flow chart of land parcel identification of the present invention;

[0064] Fig.11 This is an example diagram of eliminating fragmented land plots according to the present invention;

[0065] Fig.12 This is an example diagram of eliminating holes in the present invention;

[0066] Fig.13 It is an example diagram of the generalized results of the present invention;

[0067] Fig.14 This is a full picture display of the recognition results of a certain area in Funan, Anhui Province in the present invention;

[0068] Fig.15 This is a detailed display of the recognition results of a certain area in Funan, Anhui Province in the present invention;

[0069] Fig.16 This is a detailed comparison diagram of the remote sensing image of a certain area in Funan, Anhui Province and the binary classification results of the present invention;

[0070] Fig.17 It is the farmland identification result of the model corresponding to the verification area independently selected by the present invention;

[0071] Fig.18 This is a full picture display of the recognition results of a certain area in Rui'an, Zhejiang Province in the present invention;

[0072] Fig.19 This is a detailed display of the recognition results of a certain area in Rui'an, Zhejiang Province in the present invention;

[0073] Fig. 20 This is a detailed comparison diagram of the remote sensing image of a certain area in Rui'an, Zhejiang Province and the binary classification results of the present invention;

[0074] Fig.21 This is a greenhouse identification effect diagram of the present invention;

[0075] Fig. 22 It is a structural diagram of a farmland identification device based on a dual-stream deep network of the present invention. DETAILED DESCRIPTION

[0076] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0077] The present invention fully considers the complexity of farmland plots and constructs an efficient dual-stream farmland recognition deep network, including a farmland semantic segmentation network based on D-LinkNet and a farmland edge detection network based on DexiNed. Farmland semantic segmentation and boundary extraction learning are performed simultaneously. Based on the designed intersection difference operation, the farmland boundary detection results are used to assist in the refinement and purification of the farmland semantic segmentation results. An effective post-processing mechanism based on expert rules is established to further improve the accuracy of farmland plot recognition, thereby achieving high-precision recognition of farmland plots. The overall technical route is as follows: Figure 1 shown.

[0078] Step 1: Create a dataset

[0079] Using existing software, we vectorized the remote sensing images and outlined the farmland plots. The types of farmland should be fully covered. Then, we converted the vector layer into a raster layer and binarized it. According to the different requirements of the farmland semantic segmentation network based on D-LinkNet and the farmland edge detection network based on DexiNed for training data, we made two sets of experimental data sets. Finally, we cropped the raster layer to a suitable size according to the requirements of model training. The data set is divided into training set and validation set in an 8:2 ratio. The data example is as follows: Figure 2 shown. Figure 2 (a) is the original image, Figure 2 (b) is the true label of semantic segmentation. Figure 2 (c) is the true label of edge detection.

[0080] Step 2: Build the model

[0081] (1) Farmland semantic segmentation network based on D-LinkNet

[0082] Taking into full consideration the characteristics of farmland plot recognition, the plot recognition problem is simplified into the semantic segmentation problem of farmland and non-farmland areas. Combined with the shape and structure characteristics of farmland plots, the advanced road extraction network D-LinkNet in the field of computer vision is introduced to construct the farmland semantic segmentation network branch.

[0083] The farmland semantic segmentation network based on D-LinkNet is based on ResNet, which uses an encoder-decoder structure and adds dilated convolution as an intermediate link. This network is improved on LinkNet and dilated convolution is added to its central part. Dilated convolution is a powerful tool that can expand the receptive field of feature points without reducing the resolution of feature maps. Here, the D-LinkNet34 network based on the pytorch framework is selected for the experiment.

[0084] like Figure 3The figure shows the structure of the farmland semantic segmentation network based on D-LinkNet, where each rectangle represents a feature map of multiple channels. Part A is the encoder of D-LinkNet. Part C is the decoder of D-LinkNet. Part B is the hole convolution layer, which can expand the receptive field while retaining detailed spatial information. Except for the last convolution layer, which uses the sigmoid activation function, all other convolution layers use the ReLU activation function.

[0085] In the binary classification task, the model needs to predict only two results. For each category, we predict the probability of p and 1-p. The expression is shown in formula (1). The loss function of the original D-LinkNet network is the cross entropy loss function. Considering that the classification task has high requirements for boundary recognition, referring to the central idea of ​​the PointRend network, a PointLoss is added on the basis of the cross entropy loss function. The formula is as follows:

[0086]

[0087] PointID=Max N (-1*|mask1-mask2|) Formula (2)

[0088] Pointloss=L(Y PointID , P PointID ) Formula (3)

[0089] Where: y i ——The label of the sample, the positive class is 1 and the negative class is 0;

[0090] p i ——The probability that sample i is predicted to be a positive class;

[0091] PointID – the location of the point;

[0092] mask1——Eigenvalues ​​predicted to be farmland;

[0093] mask2——feature values ​​predicted as non-agricultural land;

[0094] Max N ——Take the first N largest numbers;

[0095] Y PointID ——The label value of the PointID pixel position;

[0096] P PointID ——The predicted probability of the PointID pixel location.

[0097] The application process of Pointloss is as follows Figure 4As shown, input image, extract image features, perform feature prediction, perform feature point prediction, and generate point feature prediction results. The PointID schematic diagram is as follows Figure 5 shown.

[0098] Using the divided training set, the D-LinkNet network model is trained according to the model parameters shown in Table 1, and the final model is saved after the training. The D-LinkNet network has the advantages of dilated convolution, skip connection, residual block and encoder-decoder architecture. The network is lightweight, fast and has excellent performance, and can well complete the recognition task of large areas of land.

[0099] Table 1 Model parameters

[0100] Parameters\Model D-LinkNet DexiNed batchsize 4 16 epoch 100 100 Learning rate 2*10-4 10-4

[0101] (2) Farmland edge detection network based on DexiNed

[0102] Considering the complexity of the boundaries of farmland plots, it is difficult to accurately determine the boundaries of plots inside the farmland by relying solely on semantic segmentation. To this end, the present invention introduces the currently advanced edge detection method, Dense Extreme Inception Network for Edge Detection (DexiNed), to construct an edge detection network branch to perform farmland plot edge detection, obtain the boundary information of farmland plots, and assist in the refinement of farmland semantic segmentation results. The farmland edge detection network based on DexiNed consists of two subnetworks: the Dense extreme inception network (Dexi) subnetwork and the Up-sampling Block (UB) subnetwork, wherein the Dexi subnetwork consists of six main modules for extracting rich features of the image, and the UB subnetwork upsamples the feature maps obtained by each main module in the Dexi subnetwork to the original image size, and by fusing the feature maps obtained by upsampling the main modules at different levels, accurate extraction of farmland boundary information is achieved.

[0103] The network structure of the farmland edge detection network based on DexiNed is as follows: Figure 6 As shown in , the encoder part consists of six main modules. Each main module is connected by a 1x1 convolution block. Each main module is composed of sub-modules whose outputs are closely connected to the output of the previous main module. The output of each main module passes through an upsampling layer and generates an intermediate result graph, as shown in Figure 7 As shown, the intermediate result graph is used to generate the final fusion result graph.

[0104] The processed sample data set is used to train the farmland edge detection network based on DexiNed. The model training parameters are shown in Table 1. After the training, the final model is saved. The farmland edge detection network model based on DexiNed has a simple structure, high network efficiency and good performance.

[0105] Step 3: Full-image prediction

[0106] Use the trained model to perform full-image predictions.

[0107] Since the size of the original image is too large, it is not suitable to be directly input into the network. However, if the original image is cropped into small images for prediction and then merged, there will be splicing traces, which will affect the detection results. To avoid this problem, we use the expansion prediction method for prediction. The main idea of ​​the expansion prediction is to fill the border twice to ensure that the central area is retained during the expansion prediction. When using the trained farmland semantic segmentation network (D-LinkNet) for full-image prediction, the sliding window size is 512*512, and the center 256*256 is retained each time, and the step size is set to 256. When using the trained farmland edge detection network (DexiNed) for full-image prediction, the sliding window size is 256*256, and the center 128*128 is retained each time, and the step size is set to 128. Through this expansion prediction, the predicted image is finally spliced ​​seamlessly, which improves the segmentation effect of large-sized images. The expansion prediction example is shown in the figure below. Figure 8 As shown in the figure, the expansion prediction comparison is as follows Fig. 9 As shown, Fig. 9 (a) is before using expansion prediction, Fig. 9 (b) After prediction using dilation.

[0108] After obtaining the prediction results of the two models for the whole image, the semantic segmentation results and edge detection results are fused based on the intersection difference operation. Taking the semantic segmentation results as the benchmark, the intersection of the semantic segmentation results and the edge detection results is subtracted to obtain the final farmland plot recognition result. Fig.10 shown.

[0109] Step 4: Post-processing

[0110] Using tools such as Python and GDAL library, we can perform a full set of post-processing operations. In the raster recognition results, there will be small and fragmented plots and holes in some large plots. According to various data analysis and the actual situation of the image area, we use the threshold setting method to remove small and fragmented plots with an area of ​​less than 45. The example of eliminating small and fragmented plots is shown in the figure below. Fig.11 As shown; Based on the idea that each plot should not have any gaps and should be a complete polygon, the holes in the plot are filled and the example of eliminating the holes is shown in Fig.12 shown.

[0111] Finally, according to the actual application requirements, the raster layer can be converted into a vector layer. According to the idea of ​​simplifying vector lines of Douglas Pugh method, the boundary line is simplified to reduce the data storage volume. The generalized result example is shown in the figure Fig.13 shown.

[0112] In order to verify the recognition effect of the present invention, in this embodiment, the data adopts the GF-2 remote sensing image, and the method of the present invention is used to extract farmland plots in a certain area of ​​Funan, Anhui and Rui'an, Zhejiang.

[0113] Example 1: Funan, Anhui

[0114] Most of the images in a certain area of ​​Funan County, Anhui Province are farmland. Fig.14 This is a full picture display of the recognition results of a certain area in Funan, Anhui. It can be seen that the model can successfully identify the farmland in the entire image. The recognition results are highly consistent with the actual image, and the overall recognition effect is basically consistent with the actual situation.

[0115] Fig.15 This is a detailed display of the recognition results for a certain area in Funan, Anhui. It can be seen that in the recognition results, the boundaries of farmland plots are clear and regular. The model can recognize large-area regular plots and small-area fragmented plots very well, and there are very few cases of misclassification of other land features such as buildings and water systems.

[0116] Fig.16 This is a comparison of the remote sensing image of a certain area in Funan, Anhui Province and the binary classification results. It can be seen that the model can well complete the classification of farmland and non-farmland in the remote sensing image, with clear boundaries and complete plots. At the same time, the model can also achieve good recognition results for farmland plots with complex shapes, colors, textures and structural compositions.

[0117] Step 5: Quantitative evaluation

[0118] (1) Calculation of evaluation indicators

[0119] F1-Score formula:

[0120] TP (True Positive): Identify the correct farmland plots

[0121] FP (False Positive): Non-agricultural land is identified as agricultural land.

[0122] FN (False Negative): Unrecognized farmland

[0123]

[0124]

[0125]

[0126] The formula for calculating the degree of conformity of plot area is:

[0127] S 农田 : The actual area of ​​the farmland plot

[0128] S 预测 : The area of ​​farmland plots predicted by the model

[0129]

[0130] (2) Independent evaluation and analysis

[0131] Fig.17 This is the farmland identification result of the model corresponding to the independently selected validation area. The validation set contains a total of four plots, which are marked as ①, ②, ③, and ④ for the convenience of explanation and evaluation.

[0132] F1-Score evaluation results:

[0133] Plot ①: TP=335, FP=48, FN=21

[0134] Precision=0.875,Recall=0.941

[0135] F1-Score ① =0.906

[0136] Plot ②: TP=318, FP=52, FN=29

[0137] Precision=0.859,Recall=0.916

[0138] F1-Score ② =0.887

[0139] Plot ③: TP=893, FP=76, FN=38

[0140] Precision=0.922,Recall=0.959

[0141] F1-Score ③ =0.940

[0142] Plot ④: TP=572, FP=79, FN=80

[0143] Precision=0.879,Recall=0.877

[0144] F1-Score ④ =0.878

[0145]

[0146] From the above results, it can be concluded that the F1-Score of the method for the first verification area is 90.6%, the F1-Score for the second verification area is 88.7%, the F1-Score for the third verification area is 94%, and the F1-Score for the fourth verification area is 87.8%. That is, the model has the best recognition effect in the third verification area, and the recognition effect in the fourth verification area is poor, which may be related to the distribution of training samples, but the overall F1-Score average of the model reaches more than 90%, which further shows that the performance of the model is good.

[0147] Evaluation results of land area consistency:

[0148] Plot ①: S 农田 =3109208.634997, S 预测 =3265915.631675

[0149] Land area matching ① =0.94959907324516

[0150] Plot ②: S 农田 =2633259.759682, S 预测 =2639132.720244

[0151] Land area matching ② =0.9977696994987273

[0152] Plot ③: S 农田 =2598064.769412, S 预测 =2664936.612565

[0153] Land area matching ③ =0.9742609022144838

[0154] Plot ④: S 农田 =2234748.962053, S 预测 =2208040.683033

[0155] Land area matching ④ =0.988048644624735

[0156] In summary:

[0157]

[0158] From the above results, it can be concluded that the area of ​​the land parcel identified by the model for the first verification area is 95.0%, the area of ​​the land parcel identified for the second verification area is 99.8%, the area of ​​the land parcel identified for the third verification area is 97.4%, and the area of ​​the land parcel identified for the fourth verification area is 98.8%. It can be seen that the area of ​​the land parcel identified by the model for each verification area can reach more than 95%, which shows that the model's results for land parcel identification are basically consistent with the actual situation, and the degree of consistency is very high.

[0159] Therefore, using the self-selected verification area for verification, the overall F1-Score average of the model can reach more than 90%, and the average matching degree of the overall land parcel identification area of ​​the model can reach more than 97%, which shows that the performance of the model is excellent. Using the land rights confirmation data for overall verification based on the above indicators, the overall accuracy of the recognition result is 84%.

[0160] Example 2: Ruian, Zhejiang

[0161] Fig.18 This is a full picture display of the recognition results of a certain area in Ruian, Zhejiang. It can be seen that there are more greenhouses and paddy fields in the farmland of Zhejiang Province compared with that of Anhui Province. At the same time, the farmland in the lower right corner is surrounded by water bodies, and the greenhouses in the upper left corner are relatively concentrated and larger in number. This method also has a good recognition effect for these two types of farmland.

[0162] Fig.19 This is a detailed display of the recognition results of a certain area in Ruian, Zhejiang. It can be seen that in the recognition results, the boundaries of farmland plots can be accurately and finely segmented, and the details are well expressed. At the same time, the area of ​​the identified plots is consistent with the actual area and has a high degree of consistency. Overall, it shows a high precision rate and a low error rate.

[0163] Fig. 20 This is a comparison of the remote sensing image and the binary classification results of a certain area in Rui'an, Zhejiang. It can be seen that this method accurately classifies farmland and non-farmland plots. The farmland plots in the recognition results are regular and basically consistent with the actual farmland range. At the same time, the boundary segmentation is relatively fine and the recognition accuracy is high.

[0164] Fig.21 This is the greenhouse recognition effect diagram. It can be seen that in the recognition results, the boundary segmentation of concentrated greenhouses is accurate and can be completely recognized. At the same time, the boundary segmentation of the complex situation of greenhouses and dry fields can also be relatively accurate. From the detailed diagram of greenhouse recognition, it can be seen that this method has good adaptability to vegetable greenhouses and high recognition accuracy.

[0165] In some embodiments, a farmland identification device based on a dual-stream deep network is provided, and the farmland identification device is used to implement any of the farmland identification methods based on a dual-stream deep network, such as Fig. 22 As shown, the farmland identification device includes the following modules:

[0166] A training set acquisition module 1 is used to acquire a first training set and a second training set;

[0167] Semantic segmentation network construction module 2, used to build a farmland semantic segmentation network based on D-LinkNet;

[0168] A semantic segmentation network training module 3 is used to train the farmland semantic segmentation network based on D-LinkNet using the first training set, and after the training is completed, a trained farmland semantic segmentation network is obtained;

[0169] Edge detection network construction module 4, used to build a farmland edge detection network based on DexiNed;

[0170] The edge detection network training module 5 is used to train the DexiNed-based farmland edge detection network using the second training data set, and after the training is completed, a trained farmland edge detection network is obtained;

[0171] The whole-image prediction module 6 is used to input the farmland remote sensing image to be identified into the trained farmland semantic segmentation network and the trained farmland edge detection network respectively, perform whole-image prediction based on the expansion prediction method, and obtain semantic segmentation results and edge detection results;

[0172] The intersection difference processing module 7 is used to perform intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result.

[0173] In some embodiments, the farmland identification device further includes:

[0174] The post-processing module 8 is used to establish an expert-based post-processing mechanism, and use the post-processing mechanism to eliminate small blocks, eliminate holes, and simplify edges of the farmland plot identification results to obtain optimized farmland plot identification results.

[0175] In some embodiments, a storage medium is also provided, which is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is run, it is used to implement the steps of any of the methods for farmland identification based on a dual-stream deep network.

[0176] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0177] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0178] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A farmland recognition method based on a dual-stream deep network, characterized in that: The following steps are involved: Obtain a first training set and a second training set; Build a farmland semantic segmentation network based on D-LinkNet; The farmland semantic segmentation network based on D-LinkNet includes: an encoder, a hole convolution layer and a decoder; The encoder is connected to the atrous convolutional layer; The decoder is connected to the encoder and the hole convolution layer respectively; The dilated convolution is used to expand the receptive field of the feature points; The loss function of the farmland semantic segmentation network based on D-LinkNet consists of the cross entropy loss function L and PointLoss, and the formula is as follows: Pointloss=L(Y PointID ,P PointID ) PointID=Max N (-1*|mask1-mask2|) Where: y i Indicates the label of sample i, the positive class is 1, the negative class is 0, and p i represents the probability that sample i is predicted as the positive class, PointID represents the location of the point, mask1 represents the feature value predicted as farmland, mask2 represents the feature value predicted as non-farmland, Max N Indicates taking the first N largest numbers, Y PointID Indicates the label value of the PointID pixel position, P PointID Indicates the predicted probability of the PointID pixel location; Using the first training set to train the D-LinkNet-based farmland semantic segmentation network, after the training is completed, a trained farmland semantic segmentation network is obtained; Construct a farmland edge detection network based on DexiNed; The DexiNed-based farmland edge detection network includes: a Dexi subnetwork and a UB subnetwork; The Dexi sub-network consists of 6 main modules, which are used to extract the features of the image to obtain the feature map; The UB sub-network upsamples the feature maps obtained by each main module to the original image size, and achieves accurate extraction of boundary information by fusing the feature maps obtained by upsampling at different levels; Using the second training data set to train the DexiNed-based farmland edge detection network, after the training is completed, a trained farmland edge detection network is obtained; The remote sensing image of the farmland to be identified is input into the trained farmland semantic segmentation network and the trained farmland edge detection network respectively, and the whole image prediction is performed based on the expansion prediction method to obtain the semantic segmentation result and the edge detection result; The semantic segmentation result and the edge detection result are subjected to intersection difference processing to obtain a farmland plot recognition result.

2. The farmland identification method based on dual-stream deep network according to claim 1 is characterized in that: The step of obtaining the first training set and the second training set includes: Obtain remote sensing images of farmland scenes; Performing vector processing on the remote sensing image to obtain a vector layer; Convert the vector layer into a raster layer and perform binarization; Create a first data set based on the binarized raster layer; Create a second data set based on the binarized raster layer; Acquire a first training set from the first data set; A second training set is obtained from the second data set.

3. The farmland identification method based on dual-stream deep network according to claim 1, characterized in that: The step of performing intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result includes: Based on the intersection difference processing, the semantic segmentation result and the edge detection result are fused, and the intersection of the semantic segmentation result and the edge detection result is subtracted based on the semantic segmentation result to obtain the farmland plot recognition result.

4. The farmland identification method based on dual-stream deep network according to claim 1, characterized in that: After the step of performing intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result, the method further includes: Establish an expert-based post-processing mechanism; The post-processing mechanism is used to eliminate small blocks, eliminate holes, and simplify edges of the farmland plot identification results to obtain optimized farmland plot identification results.

5. A farmland identification device based on a dual-stream deep network, characterized in that: The farmland identification device is used to implement the farmland identification method based on the dual-stream deep network as claimed in any one of claims 1 to 4, and the farmland identification device includes the following modules: A training set acquisition module, used to acquire a first training set and a second training set; Semantic segmentation network construction module, used to build a farmland semantic segmentation network based on D-LinkNet; A semantic segmentation network training module, used to train the D-LinkNet-based farmland semantic segmentation network using the first training set, and after the training is completed, obtain a trained farmland semantic segmentation network; Edge detection network building module, used to build a farmland edge detection network based on DexiNed; An edge detection network training module is used to train the DexiNed-based farmland edge detection network using the second training data set, and after the training is completed, a trained farmland edge detection network is obtained; A full-image prediction module is used to input the farmland remote sensing image to be identified into the trained farmland semantic segmentation network and the trained farmland edge detection network respectively, perform full-image prediction based on the expansion prediction method, and obtain semantic segmentation results and edge detection results; The intersection difference processing module is used to perform intersection difference processing on the semantic segmentation result and the edge detection result to obtain the farmland plot recognition result.

6. The farmland identification device based on the dual-stream deep network according to claim 5 is characterized in that: Also includes: The post-processing module is used to establish an expert-based post-processing mechanism, and use the post-processing mechanism to eliminate small blocks, eliminate holes, and simplify edges of the farmland plot identification results to obtain optimized farmland plot identification results.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is run, it is used to implement the steps of the farmland identification method based on the dual-stream deep network as described in any one of claims 1-4.

Images