Depth learning method and device for remote sensing extraction of cultivated land parcels guided by boundary features

Abstract

The invention discloses a cultivated land remote sensing extraction deep learning method guided by boundary features, and the method comprises the steps: constructing a CTHBNet model which employs an encoder, a conversion encoder and a decoder structure, and constructing a CNN-Transform fusion module for carrying out the feature fusion of the information of corresponding levels in the encoder and the conversion encoder, global and local information of the cultivated land parcels is effectively fused; a hierarchical information fusion module is adopted in a decoder module to recover the shape of the land parcel with fine granularity, so that the model not only can fully reserve the integrity of a large-scale land parcel, but also can accurately restore the boundary of a small-scale, long and narrow land parcel; a cultivated land feature enhancement module guided by boundary information is constructed at the end of the decoder so as to enhance the boundary information of the cultivated land parcel and improve the definition of the land parcel boundary; the method effectively solves the problems of adhesion of extraction results and serious extraction omission caused by different areas and shapes of cultivated land parcels.

Description

Technical Field

[0001] This invention belongs to the field of farmland plot extraction technology from remote sensing images, specifically involving a deep learning method for remote sensing extraction of farmland plots guided by boundary features, and also involves a computer device. Background Technology

[0002] With the continuous growth of the global population, especially in developing countries, the food security problem faced by all countries is becoming increasingly severe. To meet the huge food demand, on the one hand, arable land areas have been expanded, and on the other hand, agricultural production has gradually become more mechanized and intensive. However, the increase in arable land area often comes at the cost of forests, threatening biodiversity and mitigating global climate change. To address this threat, reasonable policies are crucial, and arable land plot information is an important data support. In addition, understanding the level of agricultural intensification and mechanization, crop diversity, and crop rotation, and providing effective decision-making for agricultural production and management, also requires arable land plot data. Therefore, there is an urgent need for accurate mapping of arable land plots in order to grasp the extent and location information of arable land in a timely and accurate manner.

[0003] Traditional methods of acquiring information about land features rely heavily on field surveys, which consume significant human, material, and financial resources and are often time-consuming, making it difficult to extract information about large-scale farmland and crop planting conditions. Remote sensing technology, on the other hand, is a large-scale, long-distance Earth observation technology with advantages such as large coverage area, high timeliness, and rich information content, making it the primary technical means for acquiring information about farmland plots. With the rapid development of satellite sensor technology, the spatial resolution of remote sensing images has gradually improved, reaching meter-level or even sub-meter-level resolution. The land feature characteristics presented in the images are more refined, providing important data support for acquiring information on crop planting structure at the plot scale.

[0004] Real-time and dynamic monitoring of arable land parcel information is crucial for optimizing agricultural resource allocation, precision agriculture management, and agricultural disaster assessment. Currently, arable land parcel products both domestically and internationally are mainly based on medium-resolution remote sensing imagery, such as the Cropland Data Layer product created by the US Department of Agriculture and the One SoilMap product produced by One Soil in Belarus. However, these products lack sufficient parcel detail, necessitating the exploration of precise arable land parcel extraction methods based on high-resolution data. Traditional methods for arable land parcel information extraction, such as edge detection, region segmentation, and traditional machine learning, can only extract shallow features from remote sensing images, easily resulting in the "salt and pepper" effect. Furthermore, they are significantly influenced by prior knowledge, threshold parameter settings, and classifier selection, leading to poor parcel integrity and boundary accuracy. In recent years, deep learning technologies, represented by convolutional neural networks (CNNs), have been widely applied in remote sensing due to their ability to automatically acquire typical features from images for image classification. Examples include road network extraction, land use / cover classification, and crop identification, all achieving considerable success. However, CNNs are highly sensitive to... Images exhibit strong robustness to noise and variations, maintaining high accuracy and generalization across diverse agricultural landscapes. Therefore, they have been used as an alternative to traditional image segmentation methods for extracting farmland information. However, farmland plots vary in size and shape, a characteristic that becomes even more pronounced in areas with complex agricultural landscapes. This results in blurred boundaries, complex morphologies, and significant scale variations in high-resolution remote sensing images. Furthermore, farmland is susceptible to interference from surrounding environmental features such as roads, water bodies, and woodlands. These factors not only increase the difficulty of plot extraction but also affect the model's accurate identification of farmland boundaries. Against this backdrop, effectively enhancing the boundary information of farmland plots and acquiring their comprehensive, multi-scale features has become a key issue for the accurate remote sensing extraction of farmland plots. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems in the existing technology by providing a deep learning method for remote sensing extraction of cultivated land plots guided by boundary features. It also provides a computer device that solves problems such as blurred boundaries of cultivated land plots, difficulty in identifying narrow or small plots, and adhesion of adjacent plots, demonstrating superior performance in cultivated land plot extraction.

[0006] The above-mentioned objectives of the present invention are achieved by the following technical means: A deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features includes the following steps: Step 1: Obtain a high-resolution remote sensing image set of the study area and preprocess it into a multispectral image set; Step 2: Perform manual visual interpretation of the multispectral image set, draw the corresponding black-and-white binary map set, boundary map set, and distance map set of the cultivated land plots, and crop them to a set size to obtain the multispectral sub-image set, black-and-white binary sub-map set, boundary sub-map set, and distance sub-map set. Construct each multispectral sub-image, black-and-white binary sub-map, boundary sub-map, and distance sub-map as the input sample, range ground truth label, boundary ground truth label, and distance ground truth label in each sample of the sample set. Expand the sample set and divide it into training set, test set, and validation set. Step 3: Construct the CTHBNet model and its corresponding composite loss function; Step 4: Input the training set into the CTHBNet model for training. After training is complete, save the model parameters to obtain the trained CTHBNet model. Step 5: Preprocess the high-resolution remote sensing image to be processed into a multispectral image, then crop the multispectral image into multiple multispectral sub-images of a set size, and then input the multispectral sub-images to be processed into the trained CTHBNet model in sequence for prediction to obtain the prediction results of the cultivated land plots corresponding to each multispectral sub-image.

[0007] As described above, the CTHBNet model includes an input layer, an encoder module, a transformation encoder module, a feature fusion module, and a decoder module; The multispectral sub-images are input into the input layer, and the output of the input layer is input into the first level of the encoder module; The encoder module includes I sequentially connected layers. When i=1~I-1, the output of the i-th layer in the encoder module is input to the (i+1)-th layer. The conversion encoder module includes I sequentially connected levels. The multispectral sub-image is also input into the first level of the conversion encoder module. When i=1~I-1, the output of the i-th level in the conversion encoder module is input into the (i+1)-th level. The feature fusion module includes I CNN-Transformer fusion modules connected in sequence. The outputs of the I levels of the encoder module and the outputs of the corresponding levels in the transformation encoder are also input into the corresponding CNN-Transformer fusion modules of the feature fusion module for feature fusion. The decoder module includes J hierarchical information fusion modules, where J = 1-1. The output of the (j+1)th hierarchical information fusion module is input into the j-th hierarchical information fusion module. The outputs of the J CNN-Transformer fusion modules of the feature fusion module are respectively input into the hierarchical information fusion modules of the corresponding numbers of the decoder module. The output of the (j+1)th CNN-Transformer fusion module of the feature fusion module is also input into the j-th hierarchical information fusion module of the decoder module. The decoder module also includes a boundary information-guided farmland feature enhancement module and a segmentation head. The input of the boundary information-guided farmland feature enhancement module is the feature map FUP1 and multispectral sub-image output by the first-level information fusion module. The feature map output by the boundary information-guided farmland feature enhancement module is input into the segmentation head. The segmentation head outputs the prediction results of the CTHBNet model for farmland plots. The prediction results include range prediction labels, boundary prediction labels, and distance prediction labels.

[0008] As described above, the i-th level output feature map Fc of the encoder module i The FT output feature map of the i-th level of the transform encoder i Feature map Fc i and feature map FT i The input feature map is fed into the i-th CNN-Transformer fusion module, and the i-th CNN-Transformer fusion module performs the following processing steps on the input feature map: Step 3.1.1: FC the feature map i After calculation using a gating mechanism, a nonlinear transformation is performed using the Sigmoid activation function to obtain the feature map FC1. i ; Step 3.1.2: FC the feature map i After a 7×7 convolution operation, average pooling is performed, followed by two 3×3 convolution operations, then the Sigmoid activation function is calculated, and finally bilinear interpolation is used to obtain the feature map FC2. i ; Step 3.1.3: FC the feature map i Feature map FC1 i and feature map FC2 i After multiplication, it is then compared with the feature map FT. i The stacking operation is performed, and then the result is passed through a CBR module with a convolutional kernel size of 1×1 to obtain the feature map FK. i ; Step 3.1.4: Transfer the feature map FK iMax pooling and average pooling are performed separately along the channel dimension, and then the features are summed after passing through a multi-level perceptron module. Finally, the Sigmoid activation function is calculated and then combined with the feature map FK. i Multiply to obtain the feature map FS i ; Step 3.1.5: FS of the feature map i After performing max pooling and average pooling in the spatial dimension, the layers are stacked in the channel dimension, then processed by a 1×1 convolution kernel and a sigmoid activation function, and finally combined with FS. i Multiply the results to obtain the feature map FR output by the i-th CNN-Transformer fusion module. i .

[0009] When j = 1 to J-1, the input of the j-th level information fusion module is the feature map FR. j The feature map FUP output by the (j+1)th level information fusion module j+1 When j=J, the input to the j-th level information fusion module is the feature map FR. j and feature map FR j+1 ; The j-th level information fusion module specifically includes the following processing steps for the input feature map: Step 3.2.1: Split the high-dimensional feature map in terms of spatial dimensions to obtain H×W feature maps FR1 j+1 Among them, the high-dimensional feature map is the feature map FR. j+1 or feature map FUP j+1 ; Step 3.2.2: Using a learnable convolutional kernel FL and H×W feature maps FR1 j+1 Perform convolution operations separately to obtain H×W feature maps FR2 j+1 ; Step 3.2.3: Convert the H×W feature maps FR2 j+1 It is divided into four equal parts along the channel dimension, resulting in 4×H×W feature maps FR2. j+1 ′; Step 3.2.4: FR2 of the 4×H×W feature maps j+1 Reorganize them spatially, grouping them into sets of four and arranging them in a 2×2 format, resulting in H×W feature maps FR3. j+1 ; Step 3.2.5: Convert the H×W feature maps FR3 j+1 After permutation and combination, the feature map FR4 is obtained. j+1 ; Step 3.2.6: Transfer the feature map FR4 j+1 and low-dimensional feature maps FRj The features map FUP is obtained by stacking the features along the channel dimension and passing them through a CBR module with a 1×1 convolution kernel. j .

[0010] As described above, the farmland feature enhancement module guided by boundary information specifically includes the following processing steps for the input feature map: Step 3.3.1: Perform Sobel operator operations on the multispectral sub-images in the horizontal and vertical directions respectively, and fuse the two results using the sum of squares to obtain a feature map F' that includes the boundary information of the multispectral sub-images; Step 3.3.2: After calculating the Sigmoid activation function, the feature map F' is multiplied by the feature map FUP1 to obtain the feature map F1; Step 3.3.3: After passing feature map F1 through linear transformation, ReLU activation function, and linear transformation in sequence, multiply it with itself to obtain feature map F2; Step 3.3.4: Perform two 7×7 convolution operations on feature map F2, calculate the Sigmoid activation function, and then multiply it with itself to obtain the output feature map FZ of the farmland feature map enhancement module guided by boundary information.

[0011] As described above, the composite loss function is calculated based on the following formula: ; In the formula, For composite loss function, For range loss function, For the boundary loss function, This is the distance loss function.

[0012] Range loss function Using binary classification cross-entropy loss function Calculated based on the following formula: ; In the formula, For the range of true labels, Predict labels for the range; Boundary loss function Using the Dice loss function Calculated based on the following formula: ; In the formula, For boundary true labels, Predict labels for boundaries, For the intersection operation, To take the absolute value; Distance loss function Using the Jaccard distance loss function Calculated based on the following formula: ; In the formula, For distance from the real label, To predict labels from distance.

[0013] As described above, the input layer includes a CBR module and a max pooling layer; Each level of the encoder module uses a Bottleneck module; Each level of the transformation encoder module uses a Swing Transformer module, and a Patch Merging module is inserted between each level of the transformation encoder module. A linear embedding module is set before the first Swing Transformer module.

[0014] As mentioned above, both the encoder module and the conversion encoder module include four levels. The four levels of the encoder module include 3, 4, 6, and 3 sequentially connected Bottleneck modules, respectively. The four levels of the conversion encoder module include 2, 2, 6, and 2 sequentially connected Swing Transformer modules, respectively.

[0015] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the deep learning method for remote sensing extraction of cultivated land plots guided by boundary features as described above.

[0016] Compared with the prior art, the present invention has the following advantages: (1) The CTHBNet model constructed in this invention adopts an encoder, a transformation encoder and a decoder structure, and constructs a CNN-Transformer fusion module to effectively fuse the global and local information of the extracted farmland plots, thereby enhancing the CTHBNet model's ability to perceive the multi-scale and complex boundary shapes of farmland plots.

[0017] (2) The CTHBNet model of the present invention also introduces a hierarchical information fusion module in the decoder module to restore the shape of the plot in a fine-grained manner, so that the model can fully preserve the integrity of large-scale plots and accurately restore the boundaries of small-scale, narrow plots.

[0018] (3) The CTHBNet model of the present invention adds a farmland feature enhancement module guided by boundary information at the end of the decoder module to strengthen the boundary information of farmland plots, improve the clarity of plot boundaries, and reduce the problem of plot adhesion.

[0019] (4) The CTHBNet model of the present invention can effectively solve the problem of the serious problem of the extraction results sticking and missing due to the different areas and shapes of cultivated land plots. The method of the present invention can not only extract the boundaries of cultivated land plots better, but also enhance the integrity of the plot extraction results, significantly improve the extraction accuracy, and has good robustness. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of the CTHBNet model of the present invention; Figure 2 This is a schematic diagram of the CNN-Transformer fusion module of the present invention (where C1, C2, and Cr are the number of channels, and H and W are the height and width of the feature map, respectively). Figure 3 This is a schematic diagram of the hierarchical information fusion module of the present invention; Figure 4 This is a schematic diagram of the structure of the hierarchical information fusion module of the present invention, which converts one of the split 2C×1×1 feature maps into a C×2×2 feature map; Figure 5 This is a schematic diagram of the structure of the farmland feature enhancement module guided by boundary information of the present invention; Figure 6 These are comparison results of the CTHBNet model of the present invention with thirteen other models in Example 2; wherein, Figure 6 (a1) and Figure 6 (a2) represents the first image to be processed and its corresponding ground truth. Figure 6 (a3) to (a16) are the prediction results of SAM, UNet, ResUNet, DeepLabv3+, PSPNet, HRNet, SwinUNet, UNetFormer, CMTFNet, PsiNet, BsiNet, SEANet, DBBANet, and the CTHBNet model of the present invention for the first image to be processed. Figure 6 (b1) and Figure 6 (b2) represents the second image to be processed and its corresponding real label. Figure 6 (b3) to (b16) are the prediction results of SAM, UNet, ResUNet, DeepLabv3+, PSPNet, HRNet, SwinUNet, UNetFormer, CMTFNet, PsiNet, BsiNet, SEANet, DBBANet, and the CTHBNet model of the present invention for the second image to be processed. Figure 6 (c1) and Figure 6 (c2) represents the third image to be processed and its corresponding real label. Figure 6 (c3) to (c16) are the prediction results of SAM, UNet, ResUNet, DeepLabv3+, PSPNet, HRNet, SwinUNet, UNetFormer, CMTFNet, PsiNet, BsiNet, SEANet, DBBANet, and the CTHBNet model of the present invention for the third image to be processed. Figure 7 This is an error graph showing the experimental results of a comparative experiment on the effectiveness of the three improved parts of the CTHBNet model in Example 2 (CNN-Transformer fusion module, hierarchical information fusion module, and boundary information-guided farmland feature enhancement module). Red indicates regions that were missed, and blue indicates regions that were misclassified. Figure 7 (a1) and Figure 7 (a2) represents the fourth image to be processed and its corresponding real label. Figure 7 (a3) to (a10) are the error graphs of the experimental results of models I to VIII in Table 1 on the fourth image to be processed; Figure 7 (b1) and Figure 7 (b2) shows the fifth image to be processed and its corresponding real label. Figure 7 (b3) to (b10) are the error graphs of the experimental results of models I to VIII in Table 1 on the fourth image to be processed; Figure 7 (c1) and Figure 7 (c2) represents the sixth image to be processed and its corresponding real label. Figure 7 (c3) to (c10) are the error graphs of the experimental results of models I to VIII in Table 1 on the fourth image to be processed. Detailed Implementation

[0021] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to embodiments. The embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0022] Example 1: A deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features includes the following steps: Step 1: Obtain a high-resolution remote sensing image set of the existing study area, and preprocess the high-resolution remote sensing image set to obtain a multispectral image set with a spatial resolution of 1m.

[0023] The preprocessing process is as follows: First, radiometric calibration, atmospheric correction, and geometric correction are performed on the multispectral bands of each high-resolution remote sensing image in the high-resolution remote sensing image set; second, radiometric calibration and orthorectification are performed on the panchromatic bands of the high-resolution remote sensing images; finally, image fusion methods (such as panchromatic sharpening NNDiffuse) are used to fuse the processed panchromatic bands and multispectral bands to obtain a multispectral image with a resolution of 1m. All multispectral images are constructed into a multispectral image set.

[0024] In this embodiment, the images of the study area were obtained from the land observation satellite data service platform of the China Resources Satellite Application Center using Gaofen-2 images of the study area (with a spatial resolution of 1m for the panchromatic band and 4m for other bands). The Gaofen-2 images were preprocessed using ENVI software to obtain Gaofen-2 multispectral images.

[0025] Step 2: Perform manual visual interpretation of the multispectral image set, and draw corresponding black-and-white binary maps, boundary maps, and distance maps for the corresponding farmland plots. Crop these maps to a set size to obtain multispectral sub-image sets, black-and-white binary sub-map sets, boundary sub-map sets, and distance sub-map sets. Construct each multispectral sub-image, black-and-white binary sub-map, boundary sub-map, and distance sub-map into the input sample, range ground truth label, boundary ground truth label, and distance ground truth label in the sample set, respectively. Expand the sample set and divide it into training, testing, and validation sets. This includes the following steps: Step 2.1: Based on the multispectral image set obtained after preprocessing in Step 1, use the vector surface construction method in ArcMap software to draw the vector dataset of cultivated land plots corresponding to the multispectral image set. Non-cultivated land areas are not drawn.

[0026] Step 2.2: Based on the cultivated land plot vector dataset constructed in Step 2.1, use the vector-to-raster tool in ArcMap software to convert the cultivated land plot vector data into a black and white binary map, with cultivated land plots in white and non-cultivated land plots in black, to obtain the black and white binary map set corresponding to the multispectral image set.

[0027] Step 2.3: Based on the black-and-white binary image set obtained in Step 2.2, the boundary map and distance map corresponding to each black-and-white binary image are calculated using the Canny function and Distance Transform function in the cv2 library, and the black-and-white binary image set, boundary map set, and distance map set corresponding to the multispectral image set are constructed.

[0028] Step 2.4: To facilitate model input, based on the binary range, boundary, and distance maps obtained in Step 2.3, each multispectral image in the multispectral image set is cropped into multiple 256*256 multispectral sub-images (Patchpartitions). The Patchpartitions are then cropped based on the black-and-white binary map set, boundary map set, and distance map set to obtain the corresponding 256*256 black-and-white binary sub-image, boundary sub-image, and distance sub-image for each multispectral sub-image. Step 2.5: Use each multispectral sub-image as the input sample in each sample, and use the black-and-white binary sub-image, boundary sub-image, and distance sub-image corresponding to each multispectral sub-image as the range true label, boundary true label, and distance true label in the corresponding sample, respectively. All samples constitute the sample set.

[0029] Step 2.6: To prevent model overfitting and improve model robustness, based on the sample set in Step 2.5, data augmentation processing methods such as image rotation, image flipping, and random noise addition are used to expand the sample set. The expanded sample set is then divided into training set, test set, and validation set according to an 8:2 sample ratio.

[0030] Step 3, as follows Figure 1 As shown, the CTHBNet model is constructed as follows: The CTHBNet model includes an input layer, an encoder module (CNNencoder), a transform encoder module (Transformer encoder), a feature fusion module (feature fusion), and a decoder module (Decoder). Multispectral sub-images are input into the input layer, and the output of the input layer is input into the first layer of the encoder module. In this embodiment, the input layer includes a CBR module with a kernel size of 7×7 and a number of convolutions of C, and a max pooling layer. Multispectral sub-images are input into the CBR module.

[0031] The encoder module comprises I sequentially connected layers. When i = 1 to I-1, the output of the i-th layer in the encoder module is input to the (i+1)-th layer. In this embodiment, I = 4, and the four layers of the encoder module each include 3, 4, 6, and 3 sequentially connected Bottleneck blocks, respectively. The feature map output from the max pooling layer is input to the first Bottleneck block in the first layer of the encoder module. The feature map output from the i-th layer of the encoder module is denoted as feature map FC. i .

[0032] The multispectral sub-image is also input into a transformation encoder module, which includes I sequentially connected layers. When i=1 to I-1, the output of the i-th layer in the transformation encoder module is input into the (i+1)-th layer. In this embodiment, I=4, and the four layers of the transformation encoder module include 2, 2, 6, and 2 sequentially connected Swin Transformer modules (sliding window Transformer blocks). The feature map output from the max pooling layer is output to the first Bottleneck block in the first layer of the encoder module. Patch Merging modules are also inserted between the layers of the transformation encoder module. A linear embedding module is set before the first Swin Transformer module. The feature map output from the i-th layer of the transformation encoder module is denoted as the feature map FT. i .

[0033] The feature fusion module includes I sequentially connected CNN-Transformer fusion modules (CTF-Module). The outputs of I layers in the encoder module and the outputs of the corresponding layers in the transformation encoder are also input into the corresponding CNN-Transformer fusion module of the feature fusion module for feature fusion.

[0034] The decoder module includes J hierarchical information fusion modules (HIF-Modules). The output of the (j+1)th hierarchical information fusion module is input into the j-th hierarchical information fusion module. The outputs of the J CNN-Transformer fusion modules of the feature fusion module are respectively input into the corresponding hierarchical information fusion modules of the decoder module. The output of the (j+1)th CNN-Transformer fusion module of the feature fusion module is also input into the j-th hierarchical information fusion module of the decoder module. The feature map output by the j-th hierarchical information fusion module is the feature map FUP. j That is, when j is 1, 2, ..., J-1, the input of the j-th level information fusion module is the feature map FR output by the i-th CNN-Transformer fusion module. j The feature map FR output by the (j+1)th CNN-Transformer fusion module i+1 The output of the j-th level information fusion module is the feature map FUP. j+1 When j is J, the input to the j-th level information fusion module is the feature map FR output by the j-th CNN-Transformer fusion module. j The feature map FR output by the (j+1)th CNN-Transformer fusion module j+1 .

[0035] The decoder module also includes a boundary information-guided farmland feature enhancement module (BGFE-Module) and a segment head. The input to the boundary information-guided farmland feature enhancement module is the feature map FUP1 and multispectral sub-image output by the first-level information fusion module. The feature map FZ output by the boundary information-guided farmland feature enhancement module is input into the segment head. The segment head is used to segment and output the prediction results of the CTHBNet model for farmland plots from the feature map FZ. The prediction results include range prediction labels, boundary prediction labels, and distance prediction labels. As one implementation method, the segment head can use a convolution operation with a kernel size of 1.

[0036] Where i and j are both serial numbers, i takes the values ​​1, 2, ..., I, j takes the values ​​1, 2, ..., J, and J = I-1.

[0037] Step 3.1, as follows Figure 2 As shown, a CNN-Transformer fusion module is constructed. The i-th CNN-Transformer fusion module performs FC processing on the input feature map. i and feature map FT i Specifically, the processing steps include the following: Step 3.1.1: FC the feature map i After calculation using a gating mechanism, a nonlinear transformation is performed using the Sigmoid activation function to obtain the feature map FC1. i .

[0038] Step 3.1.2: FC the feature map i After a 7×7 convolution (Conv) operation, average pooling (Avgpool) is performed, followed by two 3×3 convolution operations, then the Sigmoid activation function is calculated, and finally the feature map FC2 is obtained through bilinear interpolation. i .

[0039] Step 3.1.3: FC the feature map i Feature map FC1 i and feature map FC2 i After multiplication, it is then compared with the feature map FT. i The stacking operation is performed, and then the result is passed through a CBR module with a convolutional kernel size of 1×1 to obtain the feature map FK. i .

[0040] Step 3.1.4: Transfer the feature map FK iMax pooling and average pooling are performed separately along the channel dimension, and then the features are summed after passing through a multilayer perceptron (MLP) module. Finally, the sigmoid activation function is calculated and then combined with the feature map FK. i Multiply to obtain the feature map FS i .

[0041] Step 3.1.5: FS of the feature map i After performing max pooling and average pooling in the spatial dimension, the layers are stacked in the channel dimension, then processed by a 1×1 convolution kernel and a sigmoid activation function, and finally combined with FS. i Multiply the results to obtain the feature map FR output by the i-th CNN-Transformer fusion module. i .

[0042] The CNN-Transformer fusion module effectively fuses the extracted global and local information of farmland plots, enhancing the CTHBNet model's ability to perceive multi-scale and complex boundary shapes of farmland plots.

[0043] Step 3.2, as follows Figure 3 and Figure 4 As shown, a hierarchical information fusion module is constructed. The j-th hierarchical information fusion module specifically includes the following processing steps for the input feature map: Step 3.2.1: Split the high-dimensional feature map in terms of spatial dimensions to obtain H×W feature maps FR1 j+1 Wherein, when j=J, the high-dimensional feature map of the j-th level information fusion module is the feature map FR output by the (J+1)-th CNN-Transformer fusion module. j+1 When j=1~j-1, the high-dimensional feature map of the j-th level information fusion module is the feature map FUP output by the (j+1)-th level information fusion module. j+1 .

[0044] Step 3.2.2: Using a learnable convolutional kernel FL and H×W feature maps FR1 j+1 Perform convolution operations separately to obtain H×W feature maps FR2 j+1 .

[0045] Step 3.2.3: Convert the H×W feature maps FR2 j+1 The channel dimension is divided into four equal parts (Split), resulting in 4×H×W feature maps FR2. j+1 ′.

[0046] Step 3.2.4: FR2 of the 4×H×W feature maps j+1Reorganize them spatially, grouping them into sets of four and arranging them in a 2×2 format, resulting in H×W feature maps FR3. j+1 .

[0047] Step 3.2.5: Convert the H×W feature maps FR3 j+1 After permutation and combination, the feature map FR4 is obtained. j+1 .

[0048] like Figure 3 The flowchart shown illustrates the processing of the input feature map by the hierarchical information fusion module, which converts the 2C×H×W feature map FR1... j+1 The C×2H×2W feature map FR4 is obtained by sequentially splitting, decoding upsampling, and recombining. j+1 (Rearrangement includes splitting and reorganizing.) Step 3.2.6: Transfer the feature map FR4 j+1 and low-dimensional feature maps FR j The features map FUP is obtained by stacking the features along the channel dimension and passing them through a CBR module with a 1×1 convolution kernel. j .

[0049] The hierarchical information fusion module fuses the high semantic information of deep features with the high positional information of shallow features to restore the features of cultivated land plots in a fine-grained manner. This allows the CTHBNet model to fully preserve the integrity of large-scale plots while accurately restoring the boundaries of small-scale, elongated plots.

[0050] Step 3.3, as follows Figure 5 As shown, a boundary information-guided farmland feature enhancement module is constructed. This module specifically processes the input feature map FUP1 and multispectral shadow using the following steps: Step 3.3.1: Perform Sobel operator operations on the multispectral sub-images in the horizontal and vertical directions respectively, and fuse the two results using the sum of squares to obtain a feature map F' that includes the boundary information of the multispectral sub-images.

[0051] Step 3.3.2: After calculating the Sigmoid activation function, the feature map F' is multiplied by the feature map FUP1 to obtain the feature map F1.

[0052] Step 3.3.3: After passing feature map F1 through linear transformation, ReLU activation function, and linear transformation in sequence, multiply it with itself to obtain feature map F2.

[0053] Step 3.3.4: Perform two 7×7 convolution operations on feature map F2, calculate the Sigmoid activation function, and then multiply it with itself to obtain the output feature map FZ of the farmland feature map enhancement module guided by boundary information.

[0054] The boundary information-guided farmland feature enhancement module extracts texture features from the original image in the vertical and horizontal directions, multiplies and fuses them with farmland information, emphasizes the boundary features between farmland areas and other land features, and enhances the features from a global perspective so that it can focus on the extent of farmland.

[0055] Step 3.4: Construct a composite loss function Composite loss function (Right now Figure 1 L_total in the formula is calculated based on the following formula: (1); In the formula, For composite loss function, For range ( Loss function (i.e.) Figure 1 L_mask in For the boundary ( Loss function (i.e.) Figure 1 L_boundary in ( Distance ( Loss function (i.e.) Figure 1 Considering the final goal (L_distance), all tasks are equally important, so the model's total loss function is the sum of these three loss functions.

[0056] In this embodiment, the range loss function adopts the binary classification cross-entropy loss function. Calculated based on the following formula: (2); In the formula, Indicates the range of true labels, Indicates range prediction label.

[0057] In this embodiment, the boundary loss function Using the Dice loss function Calculated based on the following formula: (3); In the formula, Indicates the true label of the boundary. Indicates the boundary prediction label, For the intersection operation, To take the absolute value.

[0058] In this embodiment, the distance loss function Using the Jaccard distance loss function Calculated based on the following formula: (4); In the formula, Indicates the distance from the real label. This indicates the distance prediction label.

[0059] Step 4: Use the training set to build and train the CTHBNet model, and then input the test set into the CTHBNet model to extract land parcels and evaluate the results. This includes the following steps: Step 4.1: The initial learning rate of the CTHBNet model is lr; the learning rate decay method is cosine annealing; the training batch size of the CTHBNet model is set to N; and the number of model iterations is set to R.

[0060] Step 4.2: Input the training dataset into the CTHBNet model for training, use the weights of the last iteration as the weights of the final prediction, save the model parameters after training, and obtain the trained CTHBNet model.

[0061] Step 4.3: Make predictions on the test dataset based on the trained CTHBNet model and output the prediction results.

[0062] Step 4.4: Based on the prediction results of Step 4.3, calculate the difference between them and the corresponding actual labels, and count the true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN) of the farmland plot extraction results. Calculate the precision, recall, F1 score, and crossover ratio (CUP), and evaluate the model accuracy based on these four indicators.

[0063] Precision, Recall, Overall Precision (OA), and Mean Intersection over Union (MIOU) are calculated based on the following formulas: (5); (6); (7); (8); Step 5: Preprocess the high-resolution remote sensing image to be processed into a multispectral image, then crop the multispectral image into multiple 256*256 multispectral sub-images, and then input each multispectral sub-image into the trained CTHBNet model for prediction to obtain the prediction results of the cultivated land plots corresponding to each multispectral sub-image.

[0064] Example 2: This embodiment uses multiple domestically produced Gaofen-2 remote sensing images to conduct the experiment, covering the entire study area. All images were acquired in 2023, and after preprocessing, four-band images with a resolution of 1m were obtained, specifically near-infrared, red, green, and blue. Based on the distribution of cultivated land in the study area, four 5120*5120 pixel images and four 2560*2560 pixel images were ultimately selected.

[0065] Step 2 in Example 1 was performed on three 5120*5120 pixel images and three 2560*2560 pixel images respectively, and the resulting expanded sample set included 4500 samples.

[0066] Perform step 3 in Example 1, and divide the 4500 samples into a 3600 training dataset and a 900 validation dataset in an 8:2 ratio.

[0067] To ensure the correctness of the model test, step 2 in Example 1 was performed on the remaining 5120*5120 pixel image and 2560*2560 pixel image respectively, and the resulting expanded test dataset included 1500 samples.

[0068] In this embodiment, the parameters for executing step 5 of embodiment 1 are set as follows: This embodiment uses the PyTorch framework on an Ubuntu system with a single Nvidia GeForce RTX 3060 graphics card (12GB VRAM). The initial learning rate was set to 0.0001, the number of iterations to 100, and the batch size to 8. The Adam optimizer was used, and a multi-step decay strategy was employed for the learning rate. The start and end intervals for learning rate updates were set to [50, 85].

[0069] Perform step 5 in Example 1, input the training dataset of this example into the CTHBNet model for training, use the weight parameters of the last iteration as the final weight parameters of the CTHBNet model, save the final weight parameters, and obtain the trained CTHBNet model of this example. Then input each input sample in the test set into the trained CTHBNet model for prediction, and obtain the prediction results of each input sample in the test set.

[0070] In addition, to verify that the present invention is superior to basic deep learning models in extracting farmland plots, thirteen models, including UNet, ResUNet, DeepLabv3+, PSPNet, HRNet, SwinUNet, UNetFormer, CMTFNet, PsiNet, BsiNet, SEANet, DBBANet, and SAM, were compared with the extraction results of the CTHBNet model of the present invention. All models used the same dataset and experimental parameters.

[0071] To verify the effectiveness of the three parts of the CTHBNet model improvement in this invention (the designed CNN-Transformer fusion module, the hierarchical information fusion module, and the farmland feature enhancement module guided by boundary information), a comparative experiment was conducted using various combinations of the improved parts (Table 1). The parameters and datasets for each experiment were the same.

[0072] Table 1 compares the different combinations of the three improved modules to form the model: Accuracy evaluation and analysis of experimental results: To verify the accuracy of the CTHBNet model in extracting farmland plots, the farmland plot extraction model proposed in this invention was trained on the sample data. The training weights of the model were then used to predict the test dataset, and the test results were qualitatively and quantitatively compared with the manually labeled farmland plot sample data of the corresponding area.

[0073] Qualitative comparison of experimental results: Figure 6 and Figure 7 Experimental results for all comparative models on cultivated land plots, as well as experimental results for model improvements, are presented. Figure 6 Comparing the results of each model in extracting farmland plots in a portion of the test dataset helps to compare the extraction results of each model. Figure 7 The experimental results comparison chart of the three parts of the CTHBNet model improvement can more intuitively show the extraction results of the model by each improved module.

[0074] from Figure 6As can be seen from (a3)~(a16), (b3)~(b16), and (c3)~(c16), the SAM model performs the worst because the plots are largely stuck together, and a large portion of farmland remains unidentified. Among general semantic segmentation models, UNet, ResUNet, DeepLabv3+, and PSPNet exhibit significant oversegmentation and plot sticking, failing to accurately depict bounding boundaries. HRNet benefits from multi-scale information fusion, alleviating some of the sticking problem, but still faces difficulties in extracting plots from complex regions. For CNN-Transformer models, CMTFNet and UNetFormer... Figure 6 (c1) shows that the regular plot area performs well, but... Figure 6 As shown in (a1 and a2), significant adhesion issues still exist in irregular regions. Surprisingly, despite utilizing multi-task collaboration, SEANet, BsiNet, and PsiNet still exhibit significant plot adhesion and insufficient boundary refinement, possibly due to boundary blurring in specific test areas. DBBANet achieves better segmentation performance and effectively distinguishes plots. However, some adhesion issues remain in regions with irregular shapes and blurred boundaries. In contrast, the proposed CTHBNet model demonstrates superior performance across all scenarios. CTHBNet accurately depicts farmland boundaries in different and complex regions, yielding results that better reflect the actual spatial distribution of farmland plots.

[0075] from Figure 7 As can be seen from (a3)~(a10), (b3)~(b10), and (c3)~(c10), after the addition of the three modules constructed in this invention (i.e., Model VIII in Table 1, which is also the model of this invention), the misclassification and omission situations are significantly improved, making it more in line with the actual situation and highlighting the superior effect of their combined effect in farmland plot extraction. At the same time, compared with (Model I~Model IV), the misclassification and omission areas are reduced, and the boundary integrity of the extraction results is also improved, indicating that each module has a promoting effect on farmland plot extraction. Furthermore, compared... Figure 7 (a4), (a7), (a8), (a10). Figure 7 (b4), (b7), (b8), (b10), and Figure 7 (c4), (c7), (c8), and (c10), after being fused through the CNN-Transformer fusion module (Model II, Model V, Model VI, Model VIII), the model demonstrates excellent recognition of large-area cultivated land plots. This is primarily due to the combination of global and local information. (Comparison) Figure 7 (a1)~(a4) Figure 7 (b1)~(b4), and Figure 7 After the boundary-guided feature enhancement (Model IV) of (c1) to (c4), the boundary information of farmland plots is further enhanced, making the boundary of the result more complete.

[0076] Quantitative comparison of experimental results: The quantitative evaluation of the experimental results of each model is shown in Table 2. The indicators in the table are, in order, overall accuracy, precision, recall, F1 score, and mean intersection-over-union ratio (MIOU). As can be seen from Table 2, the CTHBNet network model proposed in this invention has the highest accuracy in farmland plot extraction, specifically with an overall accuracy of 95.56%, precision of 94.83%, recall of 94.51%, F1 score of 94.67%, and MIOU of 91.28%.

[0077] Table 2 presents a quantitative evaluation of the farmland plot extraction results of thirteen models and the CTHBNet model of this invention, using five indicators: The quantitative evaluation of the experimental results of the three improved modules is shown in Table 3. As can be seen from Table 3, after introducing the CNN and Transformer fusion module (II), the hierarchical information fusion module (III), and the farmland feature enhancement module guided by boundary information (IV), the overall accuracy, precision, recall, F1 score, and Miou of the farmland plot extraction results are improved. Overall, in the CTHBNet model proposed in this invention, the farmland plot extraction results of all three improved modules are better than those of two improved modules, and better than those of one improved module, indicating that the three improved modules all have a promoting effect on the extraction of farmland plots.

[0078] Table 3 presents a quantitative evaluation table of five indicators for the experimental results of models I to VIII in Table 1 of the comparative experiment of the improved modules: In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0079] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0080] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0081] It should be noted that the embodiments described in this invention are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains can make various modifications or additions to the described embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features, characterized in that, Includes the following steps: Step 1: Obtain a high-resolution remote sensing image set of the study area and preprocess it into a multispectral image set; Step 2: Perform manual visual interpretation of the multispectral image set, draw the corresponding black-and-white binary map set, boundary map set, and distance map set of the cultivated land plots, and crop them to a set size to obtain the multispectral sub-image set, black-and-white binary sub-map set, boundary sub-map set, and distance sub-map set. Construct each multispectral sub-image, black-and-white binary sub-map, boundary sub-map, and distance sub-map as the input sample, range ground truth label, boundary ground truth label, and distance ground truth label in each sample of the sample set. Expand the sample set and divide it into training set, test set, and validation set. Step 3: Construct the CTHBNet model and its corresponding composite loss function; Step 4: Input the training set into the CTHBNet model for training. After training is complete, save the model parameters to obtain the trained CTHBNet model. Step 5: Preprocess the high-resolution remote sensing image to be processed into a multispectral image, then crop the multispectral image into multiple multispectral sub-images of a set size, and then input the multispectral sub-images to be processed into the trained CTHBNet model in sequence for prediction to obtain the prediction results of the cultivated land plots corresponding to each multispectral sub-image.

2. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 1, characterized in that, The CTHBNet model includes an input layer, an encoder module, a transformation encoder module, a feature fusion module, and a decoder module; The multispectral sub-images are input into the input layer, and the output of the input layer is input into the first level of the encoder module; The encoder module includes I sequentially connected layers. When i=1~I-1, the output of the i-th layer in the encoder module is input to the (i+1)-th layer. The conversion encoder module includes I sequentially connected levels. The multispectral sub-image is also input into the first level of the conversion encoder module. When i=1~I-1, the output of the i-th level in the conversion encoder module is input into the (i+1)-th level. The feature fusion module includes I CNN-Transformer fusion modules connected in sequence. The outputs of the I levels of the encoder module and the outputs of the corresponding levels in the transformation encoder are also input into the corresponding CNN-Transformer fusion modules of the feature fusion module for feature fusion. The decoder module includes J hierarchical information fusion modules, where J = 1-1. The output of the (j+1)th hierarchical information fusion module is input into the j-th hierarchical information fusion module. The outputs of the J CNN-Transformer fusion modules of the feature fusion module are respectively input into the hierarchical information fusion modules of the corresponding numbers of the decoder module. The output of the (j+1)th CNN-Transformer fusion module of the feature fusion module is also input into the j-th hierarchical information fusion module of the decoder module. The decoder module also includes a boundary information-guided farmland feature enhancement module and a segmentation head. The input of the boundary information-guided farmland feature enhancement module is the feature map FUP1 and multispectral sub-image output by the first-level information fusion module. The feature map output by the boundary information-guided farmland feature enhancement module is input into the segmentation head. The segmentation head outputs the prediction results of the CTHBNet model for farmland plots. The prediction results include range prediction labels, boundary prediction labels, and distance prediction labels.

3. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 2, characterized in that, The i-th level output feature map Fc of the encoder module i The FT output feature map of the i-th level of the transform encoder i Feature map Fc i and feature map FT i The input feature map is fed into the i-th CNN-Transformer fusion module, and the i-th CNN-Transformer fusion module performs the following processing steps on the input feature map: Step 3.1.1: FC the feature map i After calculation using a gating mechanism, a nonlinear transformation is performed using the Sigmoid activation function to obtain the feature map FC1. i ; Step 3.1.2: FC the feature map i After a 7×7 convolution operation, average pooling is performed, followed by two 3×3 convolution operations, then the Sigmoid activation function is calculated, and finally bilinear interpolation is used to obtain the feature map FC2. i ; Step 3.1.3: FC the feature map i Feature map FC1 i and feature map FC2 i After multiplication, it is then compared with the feature map FT. i The stacking operation is performed, and then the feature map FK is obtained by passing it through a CBR module with a convolutional kernel size of 1×1. i ; Step 3.1.4: Transfer the feature map FK i Max pooling and average pooling are performed separately along the channel dimension, and then the features are summed after passing through a multi-level perceptron module. Finally, the Sigmoid activation function is calculated and then combined with the feature map FK. i Multiply to obtain the feature map FS i ; Step 3.1.5: FS feature map i After performing max pooling and average pooling in the spatial dimension, the layers are stacked in the channel dimension, then processed by a 1×1 convolution kernel and a sigmoid activation function, and finally combined with FS. i Multiply the results to obtain the feature map FR output by the i-th CNN-Transformer fusion module. i .

4. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 3, characterized in that, When j = 1 to J-1, the input of the j-th level information fusion module is the feature map FR. j The feature map FUP output by the (j+1)th level information fusion module j+1 When j=J, the input to the j-th level information fusion module is the feature map FR. j and feature map FR j+1 ; The j-th level information fusion module specifically includes the following processing steps for the input feature map: Step 3.2.1: Split the high-dimensional feature map in terms of spatial dimensions to obtain H×W feature maps FR1. j+1 Among them, the high-dimensional feature map is the feature map FR. j+1 or feature map FUP j+1 ; Step 3.2.2: Using a learnable convolutional kernel FL and H×W feature maps FR1 j+1 Perform convolution operations separately to obtain H×W feature maps FR2. j+1 ; Step 3.2.3: Convert the H×W feature maps FR2 j+1 It is divided into four equal parts along the channel dimension, resulting in 4×H×W feature maps FR2. j+1 ′; Step 3.2.4: FR2 of the 4×H×W feature maps j+1 Reorganize them spatially, grouping them into sets of four and arranging them in a 2×2 format, resulting in H×W feature maps FR3. j+1 ; Step 3.2.5: Convert the H×W feature maps FR3 j+1 After permutation and combination, the feature map FR4 is obtained. j+1 ; Step 3.2.6: Transfer the feature map FR4 j+1 and low-dimensional feature maps FR j The features map FUP is obtained by stacking the features along the channel dimension and passing them through a CBR module with a 1×1 convolution kernel. j .

5. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 4, characterized in that, The farmland feature enhancement module guided by boundary information specifically includes the following processing steps for the input feature map: Step 3.3.1: Perform Sobel operator operations on the multispectral sub-images in the horizontal and vertical directions respectively, and fuse the two results using the sum of squares to obtain a feature map F' that includes the boundary information of the multispectral sub-images; Step 3.3.2: After calculating the Sigmoid activation function, the feature map F' is multiplied by the feature map FUP1 to obtain the feature map F1; Step 3.3.3: After passing feature map F1 through linear transformation, ReLU activation function, and linear transformation in sequence, multiply it with itself to obtain feature map F2; Step 3.3.4: Perform two 7×7 convolution operations on feature map F2, calculate the Sigmoid activation function, and then multiply it with itself to obtain the output feature map FZ of the farmland feature enhancement module guided by boundary information.

6. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 2, characterized in that, The composite loss function is calculated based on the following formula: ； In the formula, For composite loss function, For range loss function, For the boundary loss function, This is the distance loss function.

7. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 6, characterized in that, Range loss function Using binary classification cross-entropy loss function Calculated based on the following formula: ； In the formula, For the range of true labels, Predict labels for the range; Boundary loss function Using the Dice loss function Calculated based on the following formula: ； In the formula, For boundary real labels, Predict labels for boundaries, For the intersection operation, To take the absolute value; Distance loss function Using the Jaccard distance loss function Calculated based on the following formula: ； In the formula, For distance from the real label, To predict labels from distance.

8. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 2, characterized in that, The input layer includes a CBR module and a max pooling layer; Each level of the encoder module uses a Bottleneck module; Each level of the transformation encoder module uses a Swing Transformer module, and a Patch Merging module is inserted between each level of the transformation encoder module. A linear embedding module is set before the first Swing Transformer module.

9. The deep learning method for remote sensing extraction of cultivated land parcels guided by boundary features according to claim 8, characterized in that, Both the encoder module and the conversion encoder module include four levels. The four levels of the encoder module include 3, 4, 6, and 3 sequentially connected Bottleneck modules, respectively. The four levels of the conversion encoder module include 2, 2, 6, and 2 sequentially connected Swing Transformer modules, respectively.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the deep learning method for remote sensing extraction of cultivated land plots guided by boundary features as described in any one of claims 1 to 9.

Images