A farmland plot extraction method based on multi-type dual-branch convolutional neural network

Abstract

The present invention provides a method for extracting farmland plots based on a multi-type dual-branch convolutional neural network. It creates a dual-branch multi-task network DBMNet based on a multi-type convolutional module, designs a dual-branch feature extraction module DBFE to replace the convolution layer of the original network, uses a dynamic linear constraint convolution module LCDC and a multi-scale deformable convolution feature extraction module MDFE to comprehensively extract the edge features of the image and the regional features of farmland plots of different sizes and shapes, and enhances the network's perception of information at different scales, thereby improving the network's segmentation accuracy. The network uses boundary detection and distance regression tasks to improve the accuracy of farmland plot mask extraction, thereby enabling the network to have a better generalization effect. The present invention applies a multi-type convolutional network to the task of farmland plot extraction. Experimental results show that it achieves the best segmentation performance compared to other state-of-the-art algorithms, can effectively extract the discriminative features of farmland plots, significantly improves segmentation accuracy, and accurately extracts farmland plots, showing great application potential and prospects.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image farmland plot segmentation, and in particular to a method for extracting farmland plot information through a deep convolutional network, constructing a multi-type convolutional network through an improved multi-scale deformable convolution module and a dynamic linear constraint convolution module, comprehensively and fully extracting the edge features and regional features of remote sensing image farmland plots, and accurately extracting remote sensing image farmland plots. Background Art

[0002] Arable land is the material basis for social development and human survival, and agricultural plots are the basic units for agricultural practice and application.

[0003] In such a large agricultural production context, using scientific and technological means to supervise and assist the transformation, development and production links of farmland will be a very important way to improve efficiency and increase production.

[0004] With the advancement of science and technology, more and more methods are available for determining field boundaries, and remote sensing technology has become widely popular as one of the most convenient and fastest means. Remote sensing is a non-contact method for acquiring information about distant targets, primarily using satellites, aircraft, drones, and other equipment for data collection. Remote sensing equipment can transmit and receive specific electromagnetic waves from ground targets at high altitudes, analyzing and applying them. Remote sensing imagery offers a wide range, dynamic monitoring capabilities, and real-time performance. Therefore, applying remote sensing technology to agricultural production can quickly and accurately determine the distribution, size, and spatial location of farmland plots, providing crucial basic data for land management and agricultural production.

[0005] With the continuous development of earth remote sensing observation platforms, repeatedly acquiring high-resolution ground imagery in a short period of time is no longer a problem. However, extracting valuable land parcel information from these high-quality images remains a challenge. Early methods for extracting farmland parcels from remote sensing images relied on manual experience to extract and classify information, which was inefficient and consuming a significant amount of manpower and resources. Furthermore, cultivated land is characterized by a wide variety of crop types, with significant variations in cultivation methods and maturity periods. There is also fallow land that remains uncultivated. These factors result in significant differences in the characteristics of cultivated land with different crop types in remote sensing images, leading to a significant phenomenon of different spectra for the same object, resulting in low accuracy rates for manual extraction methods. Therefore, given the vast amount of information and complex image features in remote sensing images, it is of great significance to leverage computer technology to automatically extract farmland parcels from remote sensing images. Automating this process can reduce manual intervention and improve the accuracy of parcel segmentation.

[0006] Machine learning methods are widely used for the automatic classification and extraction of remote sensing imagery. Traditional machine learning methods are highly effective for extracting cultivated land from low- and medium-resolution remote sensing imagery. However, these methods require manual parameter and feature selection and can only extract superficial features. Therefore, for high-resolution remote sensing imagery containing complex ground features, traditional methods are unable to effectively extract the deeper features of various ground features and accurately identify farmland plots. However, with the continuous development of digital agriculture and the advancement of remote sensing technology, satellite-collected remote sensing imagery is becoming clearer, with richer color, contour, and texture features, and even contains a large number of deep features such as spatial location features and information entropy changes. Therefore, the development of precision agriculture and smart agriculture is placing increasingly high demands on the data quality of farmland plots. The increasingly complex and extensive remote sensing imagery information sources and the increasingly stringent and high-standard requirements for farmland plot information have made the efficient and accurate extraction of farmland plots from high-resolution imagery a pressing challenge.

[0007] Applying deep learning technology to the semantic segmentation of remote sensing imagery plots allows for pixel-level identification and classification of plots. This automated segmentation of farmland plots in remote sensing images improves accuracy and processing speed, while also directly outputting visual segmentation results. In recent years, with the advancement of deep learning, convolutional neural network (CNN) technology has achieved remarkable results in image processing. CNNs can automatically learn and extract features from images and exhibit good robustness. Consequently, an increasing number of researchers are applying CNNs to high-resolution remote sensing image classification and extraction. However, when directly transferring CNNs from natural images to remote sensing imagery, it was discovered that, compared to natural images, remote sensing images contain not only red, green, and blue band data, but also more bands such as near-infrared and short-wave infrared. Therefore, fully leveraging the rich information advantages of remote sensing images is of great research significance. Furthermore, farmland plots in remote sensing images vary greatly in size and shape, resulting in significant feature diversity. However, existing CNN-based farmland plot extraction methods mostly use fixed-size traditional convolutional feature extraction, which fails to fully capture the diverse characteristics of farmland plots. Therefore, it is crucial to design a farmland plot extraction method based on convolutional neural networks that can more efficiently process complex high-resolution remote sensing images, extract complex and diverse farmland plot features, and significantly improve the plot extraction accuracy. Summary of the Invention

[0008] The purpose of this paper is to solve the problem of single feature extraction in existing methods. A dual-branch multi-task network DBMNet based on multi-type convolution modules is proposed. The network uses multi-type convolution to comprehensively and fully extract the diverse features of farmland plots in remote sensing images, thereby improving the segmentation effect of high-resolution remote sensing plot images.

[0009] The technical solution of the present invention provides a method for extracting farmland plots based on a multi-type dual-branch convolutional neural network, comprising the following steps:

[0010] Step 1: Build a dual-branch multi-task network (DBMNet) based on multi-type convolutional modules. Using the U-Net network as the baseline network, design a dual-branch feature extraction module (DBFE) to replace the convolutional layers of the original network. This enables the network to comprehensively extract edge features of the image and regional features of fields of different sizes and shapes, increasing the network depth while improving the network's ability to perceive information at different scales.

[0011] Step 2: Design edge feature extraction branches and regional feature extraction branches for the DBFE module in Step 1. For edge feature extraction, a dynamic linear constrained convolution module (LCDC) is designed to enhance the network's perception of slender linear structures and adaptively focus on slender and tortuous local structures to accurately capture edge features of field boundaries. For regional feature extraction, a multi-scale deformable convolution feature extraction module (MDFE) is designed. This module performs deformable convolution operations on input data at different scales, adaptively focusing on the shape structure of field plots, and accurately capturing regional feature information of field plots of different sizes and shapes.

[0012] In step 3, the network finally outputs the boundary, mask, and boundary distance of the field block respectively. Boundary detection and distance regression are used to improve the accuracy of field block mask extraction, so that the network has a better generalization effect.

[0013] Furthermore, in step 1, the multi-band input data x∈ extracted from the remote sensing image H×W×B , where H×W is the size of the input sample, and B is the number of band channels of the sample; after the data is processed by the first layer of 3×3 convolution module, it becomes H1, where H i ∈ h×w×b represents the feature map output by the i-th layer of the network, h×w is the feature map size, and b is the number of feature map channels; H1 is used as the input data of the first layer of DBFE and is further encoded to obtain H2, and so on until the last layer. The stacked DBFE module extracts multi-scale, shallow to deep deep farmland plot features from the original multi-band remote sensing image data and outputs them.

[0014] Furthermore, the stacked DBFE module includes m DBFE modules.

[0015] Furthermore, in step 2, the input data H of each layer of the DBFE module i, i = 1, 2, …, m-1 are fed into two different branches to extract edge features and regional features of farmland plots, respectively, where m is the number of DBFE modules. The edge feature extraction branch of DBFE consists of an LCDC module and a traditional 3×3 convolution. The LCDC module is a deformable convolution with arbitrary parameters N. The convolution considers the upper left corner as the sampling origin (0, 0). Let R denote the sampling grid. R is expressed as follows:

[0016] R={(-1,-1),(-1,0),…,(0,1),(1,1)} (1)

[0017] The initial sampling position p of the convolution n It can be customized or randomly generated, p0 is the output image h i,1 The value of each pixel in the output image is calculated as follows:

[0018]

[0019] Among them, w(p n ) indicates p n The size of the convolution kernel weight of the point. n +offset n ) represents p0+p n +offset n The eigenvalue of the point, offset n is the offset, but if the model learns the deformation offset completely freely, the receptive field will often deviate from the target, especially when dealing with slender linear structures. Therefore, consider a convolution kernel of size k, with the point in the upper left corner as the sampling origin C0, and the remaining k-1 sampling points adopt an iterative strategy when offsetting: each C i , i=1,2,...,k-1 points are relative to their previous point C i-1 ,i=1,2,...,k-1 adds an offset Δ={δ|δ∈[-1,1]}, and the offsets are accumulated in sequence to ensure that the convolution kernel conforms to the linear morphological structure. The formula is as follows:

[0020]

[0021] Since the offset is usually a decimal, bilinear interpolation is used to calculate the pixel value of the offset point. The formula is as follows:

[0022]

[0023] Where q represents the four nearest integer points around point p on the feature map, x(q) represents the pixel value of point q, and x(p) represents the calculated value of point p after the offset, p = p0 + p n +offset n .

[0024] Furthermore, in step 2, the regional feature extraction branch of DBFE is composed of MDFE modules. i , i=1,2,...,m, and evenly divide it into s feature map subsets h according to the channel i,j ,j=1,2,...,s. Except h i,1 Without going through the convolution layer, it is a quick connection, and the rest of h i,j ,j=2,3,...,s have corresponding 5×5 deformable convolution layers, using D i,j (),j=2,3,...,s represents, D i,j () is composed of 5×5 deformable convolution, BatchNormalization (BN) and ReLU activation function in sequence. i,j Indicates D i,j The output of (), the output of each branch j is the input h of the branch i,j Splice the output y of the previous branch i,j-1 After D i,j () is obtained, so y i,j can be written as:

[0025]

[0026] in, Represents the concatenation operation, each 5×5 deformable convolution operator D i,j () It is possible to split h from all features i,j ,j≤i receives feature information, so the receptive field of each branch output result increases successively. This paper obtains equivalent larger convolution and receptive field by repeatedly stacking small-sized convolutions. Moreover, since two convolution layers use one more activation function than one convolution layer, the nonlinear ability is enhanced.

[0027] Furthermore, in step 3, the mask and boundary prediction tasks are binary classification tasks, so the negative log-likelihood (NLL) loss is used, while the distance map estimation task is a regression task and the mean square error (MSE) loss is used.

[0028] Let L cls is the pixel classification loss (i.e., NLL), x is the pixel location in the image space R, and the loss function for the mask and boundary prediction task of N input instances is defined as:

[0029]

[0030] Among them, P mc (x; l mc (x)) represents the true label l activated by SoftMax mc The predicted probability of .

[0031] In order to obtain the MSE loss for the distance map estimation task (i.e., L reg ), this paper uses the predicted distance Calculate the deviation from the actual distance D(x):

[0032]

[0033] For multiple tasks, the loss values of different subtasks are aggregated to obtain the total loss:

[0034] L total =w1·L cls1 +w2·L cls2 +w3·L reg (8)

[0035] Among them, L cls1 、L cls2 、L reg Refers to the loss values of the three subtasks, corresponding to mask, boundary and distance prediction respectively, and w1, w2 and w3 are the corresponding weight coefficients.

[0036] Furthermore, the initial learning rate used in the training phase is 10 -2 and weight decay is 10 -4 The Apollo optimizer is used to train the mask prediction and distance map estimation tasks, and the initial learning rate is 10 -8 and weight decay is 10 -4 The stochastic gradient descent (SGD) optimizer is used to train the edge prediction task.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The proposed dual-branch multi-task network DBMNet, based on multi-type convolutional modules, builds on the existing U-Net network by replacing the convolutional layers with a dual-branch feature extraction module. This allows the network to comprehensively extract edge features from images as well as regional features of fields of varying sizes and shapes. This increases network depth while enhancing its ability to perceive information at different scales, thereby improving segmentation accuracy. The network ultimately outputs the field boundary, mask, and boundary distance. Boundary detection and distance regression improve the accuracy of field mask extraction, resulting in better generalization of the network.

[0039] The dual-branch feature extraction module DBFE proposed in the present invention is designed with edge feature extraction branches and regional feature extraction branches. For the extraction of edge features, a dynamic linear constrained convolution module LCDC is designed to enhance the network's perception of slender linear structures, adaptively focusing on slender and tortuous local structures to accurately capture the edge features of field boundaries. For the extraction of regional features, a multi-scale deformable convolution feature extraction module MDFE is designed to perform deformable convolution operations on input data at different scales, adaptively focusing on the shape structure of field plots, accurately capturing regional feature information of field plots of different sizes and shapes, and improving the generalization ability of the model. Therefore, the farmland plot extraction method based on multi-type dual-branch convolutional neural networks not only has very important academic value, but also has important practical significance and application potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall network structure diagram of the present invention.

[0041] Figure 2 It is the structural diagram of the dynamic linear constrained convolution module LCDC.

[0042] Figure 3 It is the structural diagram of the multi-scale deformable convolutional feature extraction module MDFE. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the following, in conjunction with the accompanying drawings, will further illustrate the farmland plot extraction method based on a multi-type dual-branch convolutional neural network with examples. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention.

[0044] The present invention mainly addresses the problem that traditional convolutional neural networks usually only extract edge features or regional features of farmland plots, use a fixed-size receptive field, and lack the mining of diversified features of farmland plots. A dual-branch multi-task network DBMNet based on multi-type convolution modules is constructed, and a dual-branch feature extraction module DBFE is designed: a multi-scale deformable feature extraction module, which mines multi-scale information of farmland plots in remote sensing images, and adaptively focuses on the shape structure of field plots, accurately capturing regional feature information of field plots of different sizes and shapes; a dynamic linear constraint convolution module, which enhances the network's perception of slender linear structures, adaptively focuses on slender and tortuous local structures, and accurately captures the edge features of field boundaries; and builds a multi-task network framework: the network finally outputs the boundary, mask and boundary distance of the field plot respectively, among which boundary detection and distance regression are conducive to improving the accuracy of farmland plot mask extraction, thereby making the network have a better generalization effect.

[0045] Step 1: Build a dual-branch multi-type convolutional network (DBMNet) based on a multi-type convolutional module. Using the U-Net network as the baseline network, we design a dual-branch feature extraction module (DBFE) to replace the convolutional layers of the original network. This enables the network to comprehensively extract edge features of the image and regional features of fields of different sizes and shapes, increasing the network depth while improving the network's ability to perceive information at different scales.

[0046] Step 2: Design edge feature extraction branches and regional feature extraction branches for the DBFE module in Step 1. For edge feature extraction, a Line Constrained Dynamic Convolution Module (LCDC) is designed to enhance the network's perception of slender linear structures and adaptively focus on slender and tortuous local structures to accurately capture the edge features of field boundaries. For regional feature extraction, a Multi-scale Deformable Feature Extraction Module (MDFE) is designed to perform deformable convolution operations on input data at different scales, adaptively focusing on the shape structure of field plots and accurately capturing regional feature information of field plots of different sizes and shapes.

[0047] In step 3, the network finally outputs the boundary, mask, and boundary distance of the field block respectively. Boundary detection and distance regression are used to improve the accuracy of field block mask extraction, so that the network has a better generalization effect.

[0048] Furthermore, in step 1, the multi-band input data x∈ extracted from the remote sensing image H×W×B , where H×W is the size of the input sample, and B is the number of band channels of the sample; after the data is processed by the first layer of 3×3 convolution module, it becomes H1, where H i ∈ h×w×b represents the feature map output by the i-th layer of the network, h×w is the feature map size, and b is the number of feature map channels; H1 is used as the input data of the first layer of DBFE and is further encoded to obtain H2, and so on until the last layer. The stacked DBFE module extracts multi-scale, shallow to deep deep farmland plot features from the original multi-band remote sensing image data and outputs them.

[0049] Furthermore, in this embodiment, the stacked DBFE module includes m DBFE modules, and the input data H of each layer of the DBFE module is i, i=1,2,...,m-1 are input into two different branches to extract the edge features and regional features of farmland plots respectively, where m is the number of DBFE modules. The final output of the module is H i+1 =h i,1 +h i,2 ,i=1,2,...,m, where h i,1 ,i=1,2,...,m represents the final output of the edge feature extraction branch, h i,2 , i = 1, 2, ..., m represents the final output of the regional feature extraction branch. The edge feature extraction branch of DBFE consists of an LCDC module and a traditional 3 × 3 convolution. The LCDC module is a deformable convolution with arbitrary parameters N. The convolution considers the upper left corner as the sampling origin (0, 0). Let R represent the sampling grid. R is expressed as follows:

[0050] R={(-1,-1),(-1,0),...,(0,1),(1,1)} (1)

[0051] The initial sampling position p of the convolution n It can be customized or randomly generated, p0 is the output image h i,1 The value of each pixel in the output image is calculated as follows:

[0052]

[0053] Among them, w(p n ) indicates p n The size of the convolution kernel weight of the point. n +offset n ) represents p0+p n +offset n The eigenvalue of the point, offset n is the offset, but if the model learns the deformation offset completely freely, the receptive field will often deviate from the target when processing slender linear structures. Therefore, consider a convolution kernel of size k, with the point in the upper left corner as the sampling origin C0, and the remaining k-1 sampling points adopt an iterative strategy when offsetting: each C i , i=1,2,...,k-1 points are relative to their previous point C i-1 ,i=1,2,...,k-1 adds an offset Δ={δ|δ∈[-1,1]}, and the offsets are accumulated in sequence to ensure that the convolution kernel conforms to the linear morphological structure. The formula is as follows:

[0054]

[0055] Since the offset is usually a decimal, bilinear interpolation is used to calculate the pixel value of the offset point. The formula is as follows:

[0056]

[0057] Where q represents the four nearest integer points around point p on the feature map, x(q) represents the pixel value of point q, and x(p) represents the calculated value of point p after the offset, p = p0 + p n +offset n .

[0058] Furthermore, the regional feature extraction branch of DBFE is composed of MDFE modules. i , i=1,2,...,m, and evenly divide it into s feature map subsets h according to the channel i,j ,j=1,2,...,s. Except h i,1 Without going through the convolution layer, it is a quick connection, and the rest of h i,j ,j=2,3,...,s have corresponding 5×5 deformable convolution layers, using D i,j (),j=2,3,...,s represents, D i,j () is composed of 5×5 deformable convolution, BatchNormalization (BN) and ReLU activation function in sequence. i,j Indicates D i,j The output of (), the output of each branch j is the input h of the branch i,j Splice the output y of the previous branch i,j-1 After D i,j () is obtained, so y i,j can be written as:

[0059]

[0060] in, Represents the concatenation operation, each 5×5 deformable convolution operator D i,j () It is possible to split h from all features i,j ,j≤i receives feature information, so the receptive field of each branch output result increases successively. This paper obtains equivalent larger convolution and receptive field by repeatedly stacking small-sized convolutions. Moreover, since two convolution layers use one more activation function than one convolution layer, the nonlinear ability is enhanced.

[0061] Furthermore, in step 3, the mask and boundary prediction tasks are binary classification tasks, so the negative log-likelihood (NLL) loss is used, while the distance map estimation task is a regression task and the mean square error (MSE) loss is used.

[0062] Let L cls is the pixel classification loss (i.e., NLL), x is the pixel location in the image space R, and the loss function for the mask and boundary prediction task of N input instances is defined as:

[0063]

[0064] Among them, P mc (x; l mc (x)) represents the true label l activated by SoftMax mc The predicted probability of .

[0065] In order to obtain the MSE loss for the distance map estimation task (i.e., L reg ), this paper uses the predicted distance Calculate the deviation from the actual distance D(x):

[0066]

[0067] For multiple tasks, the loss values of different subtasks are aggregated to obtain the total loss:

[0068] L total =w1·L cls1 +w2·L cls2 +w3·L reg (8)

[0069] Among them, L cls1 、L cls2 、L reg Refers to the loss values of the three subtasks, corresponding to mask, boundary and distance prediction respectively, w1, w2, w3 are the corresponding weight coefficients. The initial learning rate used in the training phase is 10 -2 and weight decay is 10 -4 The Apollo optimizer is used to train the mask prediction and distance map estimation tasks, and the initial learning rate is 10 -8 and weight decay is 10 -4 The stochastic gradient descent (SGD) optimizer is used to train the edge prediction task.

[0070] The present invention can be implemented by computer software technology. Figure 1 、 Figure 2 and Figure 3 The specific steps of implementing the farmland plot extraction method based on multi-type dual-branch convolutional neural network are described in detail.

[0071] Step 1: Build a dual-branch multi-type convolutional network (DBMNet) based on a multi-type convolutional module. Using the U-Net network as the baseline network, we design a dual-branch feature extraction module (DBFE) to replace the convolutional layers of the original network. This enables the network to comprehensively extract edge features of the image and regional features of fields of different sizes and shapes, increasing the network depth while improving the network's ability to perceive information at different scales.

[0072] In the example, Figure 1As shown in the network diagram, DBMNet consists of two parts: an encoder and a decoder. The encoder consists of two layers of traditional 3×3 convolutional blocks and two layers of DBFE modules, while the decoder consists of two layers of traditional 3×3 convolutional blocks and one layer of DBFE modules. The network input data size is (4, 320, 320), and each sample contains 4 channels of size (320, 320). After the first layer of 3×3 convolution in the encoder, the output feature map size is (32, 320, 320), meaning the number of channels is reduced to 32. The feature map output by the first layer is further used as the input of the second-layer DBFE module, and the new feature map output size is (64, 320, 320). After the 2×2 maximum pooling operation, the output feature map is (64, 160, 160). This operation is repeated until the fourth-layer DBFE module, and the output feature map is (256, 80, 80). After the 2×2 maximum pooling operation, the output feature map is (256, 40, 40). After that, the output feature map obtained by the 2×2 upsampling operation is (256, 80, 80), which is spliced with the output feature map of the third 3×3 convolutional layer in the encoder, and the resulting feature map is (384, 80, 80), which is used as the input of the first 3×3 convolutional layer in the decoder, and the new feature map output size is (128, 80, 80). After that, it is subjected to a 2×2 upsampling operation, and the output feature map is (128, 160, 160), which is spliced with the output feature map of the second layer DBFE module in the encoder, and the resulting feature map is (192, 160, 160), which is used as the input of the second layer DBFE module in the decoder, and the new feature map output size is (64, 160, 160). This operation is repeated until the third layer DBFE module in the decoder, and the output feature map is (32, 320, 320). Then, 1×1 convolution is used to obtain the output feature maps corresponding to the three tasks of mask, boundary prediction, and distance regression. The feature map size corresponding to mask and boundary prediction is (2, 320, 320), which is passed through the softmax function to obtain the predicted label. The feature map size corresponding to the distance regression task is (1, 320, 320), which is passed through the sigmoid function to obtain the regression result. The initial learning rate used in the training phase is 10 -2 and weight decay is 10 -4 The Apollo optimizer is used to train the mask prediction and distance map estimation tasks, and the initial learning rate is 10 -8 and weight decay is 10 -4 The stochastic gradient descent (SGD) optimizer is used to train the edge prediction task. The training batch size is 2 and the training lasts for 150 iterations.

[0073] Step 2: The DBFE module in step 1 consists of two branches. The edge feature extraction branch consists of a dynamic linear constrained convolution module (LCDC) and a traditional 3×3 convolution module. The region feature extraction branch consists of a multi-scale deformable feature extraction module (MDFE). The input feature map of the DBFE module is input into the two branches respectively, and features are extracted by the LCDC module and the MDFE module respectively. Finally, the output feature maps of the two branches are added together to obtain the output result of the DBFE module.

[0074] In the example, Figure 2 As shown in the structure diagram, the LCDC module is used to enhance the network's perception of slender linear structures, adaptively focusing on slender and tortuous local structures to accurately capture the edge features of the field boundary. The LCDC module is a deformable convolution with a parameter of 9. The convolution considers the point in the upper left corner as the sampling origin (0, 0). The initial sampling position of the convolution is p n is set to be along the diagonal position, i.e. p n ={(0,0),(1,1),...,(8,8)}. Consider a convolution kernel of size 9, with the upper left corner as the sampling origin C0, and the remaining 8 sampling points adopt an iterative strategy when offsetting: each C i , i=1,2,..., 8 points are relative to their previous point C i-1 , i = 1, 2, ..., 8, an offset Δ = {δ|δ∈[-1, 1]} is added. This means that each point can be offset within a 3×3 range around its previous point, and the offsets are accumulated sequentially to ensure that the convolution kernel conforms to the linear morphological structure. Since the offset is usually a decimal, bilinear interpolation is used to calculate the pixel value of the offset point.

[0075] In the example, Figure 3 As shown in the structure diagram, MDFE is used to perform deformable convolution operations on the input data at different scales, adaptively focusing on the shape structure of the field blocks, and accurately capturing the regional feature information of field blocks of different sizes and shapes. The MDFE module evenly divides the input data H into 4 feature map subsets h according to the channel. i ,i=1,2,3,4. Except h1 which does not pass through the convolution layer and is a shortcut connection, the other h i , i=2,3,4 all have corresponding 5×5 deformable convolution layers. i Represents the output of each branch, then the output y i , i=3,4 are all input h of this branch iSplice the output y of the previous branch i Finally, the outputs of the four branches are concatenated to obtain the final output y of the MDFE module.

[0076] In this example, the loss function weight coefficients of the three tasks are all 1, and the initial learning rate of the mask prediction and distance map estimation tasks is 10 -2 , using a weight decay of 10 -4 The Apollo optimizer is trained with an initial learning rate of 10 for edge prediction tasks. -8 , using a weight decay of 10 -4 The training is performed using the stochastic gradient descent (SGD) optimizer. The training batch size is 2 and the training lasts for 150 iterations.

[0077] Those skilled in the art will understand that the present invention, for the first time, looks at the problem of cross-regional crop mapping from the perspective of deep feature distribution. It effectively learns the feature distribution of crop samples in different phenological regions without using target domain labels, effectively aligns the deep phenological features corresponding to crop samples in different regions, makes the invalid deep model discriminative again, significantly improves the mapping accuracy, and has the potential to embed multi-scale feature alignment modules into other deep networks for cross-phenological crop mapping.

[0078] It should be noted and understood that various modifications and improvements can be made to the present invention described in detail above without departing from the spirit and scope of the present invention as required by the appended claims. Therefore, the scope of the technical solution claimed is not limited by any specific exemplary teaching given.

Claims

1. A farmland plot extraction method based on a multi-type dual-branch convolutional neural network, characterized in that: The following steps are involved: Step 1: Build a dual-branch multi-task network (DBMNet) based on multi-type convolutional modules. Using the U-Net network as the baseline network, design a dual-branch feature extraction module (DBFE) to replace the convolutional layers of the original network. This enables the network to comprehensively extract edge features of the image and regional features of fields of different sizes and shapes, increasing the network depth while improving the network's ability to perceive information at different scales. Step 2: For the DBFE module in step 1, we design edge feature extraction branches and regional feature extraction branches. For edge feature extraction, we design a dynamic linear constrained convolution module (LCDC) to enhance the network's perception of slender linear structures and adaptively focus on slender and tortuous local structures to accurately capture the edge features of field boundaries. For regional feature extraction, we design a multi-scale deformable convolution feature extraction module (MDFE) to perform deformable convolution operations on input data at different scales, adaptively focusing on the shape structure of field plots, and accurately capturing the regional feature information of field plots of different sizes and shapes. Step 3: The network finally outputs the boundary, mask and boundary distance of the field block respectively. Boundary detection and distance regression are used to improve the accuracy of field block mask extraction, so that the network has a better generalization effect.

2. The farmland plot extraction method based on a multi-type dual-branch convolutional neural network according to claim 1, characterized in that: In step 1, the multi-band input data extracted from the remote sensing image Where H×W is the size of the input sample, B is the number of band channels of the sample; the data is processed by the first layer of 3×3 convolution module to become H1, where represents the feature map output by the i-th layer of the network, h×w is the feature map size, and b is the number of feature map channels; H1 is used as the input data of the first layer of DBFE and is further encoded to obtain H2, and so on until the last layer. The stacked DBFE module extracts multi-scale, shallow to deep deep farmland plot features from the original multi-band remote sensing image data and outputs them.

3. The farmland plot extraction method based on a multi-type dual-branch convolutional neural network according to claim 1, characterized in that: The stacked DBFE module includes m DBFE modules.

4. The farmland plot extraction method based on a multi-type dual-branch convolutional neural network according to claim 1, characterized in that: In step 2, the input data H of each layer of the DBFE module i , i=1,2,...,m-1 are input into two different branches to extract the edge features and regional features of farmland plots respectively, where m is the number of stacked DBFE modules. The final output of the module is H i+1 =h i,1 +h i,2 ,i=1,2,...,m, where h i,1 ,i=1,2,...,m represents the final output of the edge feature extraction branch, h i,2 , i = 1, 2, ..., m represents the final output of the regional feature extraction branch; the edge feature extraction branch of the DBFE module consists of an LCDC module and a traditional 3 × 3 convolution. For the LCDC module, it is a deformable convolution with an arbitrary parameter N. The convolution regards the point in the upper left corner as the sampling origin (0, 0). Let R represent the sampling grid. R is expressed as follows: R＝{(-1,-1),(-1,0),...,(0,1),(1,1)} (1) The initial sampling position p of the convolution n It can be customized or randomly generated, p0 is the output image h i,1 The value of each pixel in the output image is calculated as follows: Among them, w(p n ) indicates p n The size of the convolution kernel weight of the point, x(p0+p n +offset n ) represents p0+p n +offset n The eigenvalue of the point, offset n is the offset, but if the model learns the deformation offset completely freely, the receptive field will often deviate from the target when processing slender linear structures. Therefore, consider a convolution kernel of size k, with the point in the upper left corner as the sampling origin C0, and the remaining k-1 sampling points adopt an iterative strategy when offsetting: each C i , i=1,2,...,k-1 points are relative to their previous point C i-1 ,i=1,2,...,k-1 adds an offset Δ={δ|δ∈[-1,1]}, and the offsets are accumulated in sequence to ensure that the convolution kernel conforms to the linear morphological structure. The formula is as follows: Since the offset is usually a decimal, bilinear interpolation is used to calculate the pixel value of the offset point. The formula is as follows: Where q represents the four nearest integer points around point p on the feature map, x(q) represents the pixel value of point q, and x(p) represents the calculated value of point p after the offset, p = p0 + p n +offset n .

5. The farmland plot extraction method based on a multi-type dual-branch convolutional neural network according to claim 1, characterized in that: In step 2, the regional feature extraction branch of the DBFE module is composed of a multi-scale deformable convolutional feature extraction module MDFE. i , i=1,2,...,m, and evenly divide it into s feature map subsets h according to the channel i,j ,j=1,2,...,s, except h i,1 Without going through the convolution layer, it is a quick connection, and the rest of h i,j ,j=2,3,...,s have corresponding 5×5 deformable convolution layers, using D i,j (·),j=2,3,…,s means, D i,j (·) is composed of 5×5 deformable convolution, BatchNormalization (BN) and ReLU activation function in sequence, with y i,j Indicates D i,j (·) output, the output of each branch j is the input h of the branch i,j Splice the output y of the previous branch i,j-1 After D i,j (·) is obtained, so y i,j can be written as: in, Represents the concatenation operation, each 5×5 deformable convolution operator D i,j (·) It is possible to split h from all features i,j ,j≤i receives feature information, so the receptive field of each branch output result increases successively, and a larger convolution and receptive field are equivalently obtained by repeatedly stacking small-sized convolutions. Moreover, since two convolution layers use one more activation function than one convolution layer, the nonlinear ability is enhanced.

6. The method for extracting farmland plots based on a multi-type dual-branch convolutional neural network according to claim 1, characterized in that: In step 3, the mask and boundary prediction tasks are binary classification tasks, so negative log-likelihood loss is used, while the distance map estimation task is a regression task, and mean square error loss is used; let L cls is the pixel classification loss, x is the pixel location in the image space R, and the loss function for the mask and boundary prediction task of N input instances is defined as: Among them, P mc (x; l mc (x)) represents the true label l activated by SoftMax mc The predicted probability of distance map estimation task is to obtain the MSE loss L reg , according to the predicted distance Calculate the deviation from the actual distance D(x): For multiple tasks, the loss values of different subtasks are aggregated to obtain the total loss: L total =w1·L cls1 +w2·L cls2 +w3·L reg (8) Among them, L cls1 、L cls2 、L reg Refers to the loss values of the three subtasks, corresponding to mask, boundary and distance prediction respectively, and w1, w2 and w3 are the corresponding weight coefficients.

7. The farmland plot extraction method based on a multi-type dual-branch convolutional neural network according to claim 1, characterized in that: The initial learning rate used in the training phase is 10 -2 and weight decay is 10 -4 The Apollo optimizer is used to train the mask prediction and distance map estimation tasks, and the initial learning rate is 10 -8 and weight decay is 10 -4 The stochastic gradient descent optimizer is used to train the edge prediction task, the training batch is selected as 2, and the training lasts for 150 iterations.

Images