Image cultivated land information extraction method based on multi-scale global-local information aggregation

Abstract

The application discloses a kind of remote sensing image cultivated land information extraction methods based on multi-scale global-local information aggregation, first builds and trains multi-scale global-local information aggregation network, so that it can accurately identify background and cultivated land in remote sensing image after convergence;Then the large-area remote sensing image is overlapped sliding window cutting, then the cut image slice is input to the trained multi-scale global-local information aggregation network according to batch, so as to output the label value corresponding to background and cultivated land in each image slice, finally, according to the prediction result of each image slice, remapping is carried out, and the visual large-area cultivated land information extraction map is formed.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and more specifically, relates to a method for extracting farmland information from remote sensing images based on a multi-scale global-local information aggregation network. Background Technology

[0002] Arable land is the foundation of human food production and is crucial for global food security, sustainable agricultural development, and ecological balance. Arable land includes not only farmland but also pastureland and grassland. With population growth and socio-economic transformation, the protection and rational utilization of arable land resources face severe challenges. Global food security is also threatened by international market supply shortages, regional conflicts, extreme weather events, and the impact of pandemics. These factors make ensuring a balance between food production and demand and food security an urgent task for national development. Therefore, strengthening the acquisition and protection of arable land information, improving food production efficiency, and promoting the scientific use of arable land have become important strategic measures to address the global food crisis. A prominent challenge in implementing these measures is the acquisition and monitoring of arable land information. my country has a vast territory and abundant but unevenly distributed arable land resources; traditional methods of acquiring arable land information are time-consuming and costly, making the rapid and accurate acquisition of arable land information difficult. Faced with this challenge, rapidly developing modern remote sensing and satellite technologies provide important data sources for arable land extraction from high-resolution remote sensing images.

[0003] In recent years, compared with traditional methods of manually designing features to extract farmland, research on farmland extraction from high-resolution remote sensing images based on deep learning has achieved more outstanding results. This technology allows relevant departments to monitor farmland use status and assess farmland quality in real time, providing a scientific basis for farmland protection and rational utilization. However, current research in this field is mainly limited by two factors: the development of farmland datasets and the design of deep learning algorithms. From the perspective of dataset development, existing remote sensing farmland datasets lack image data and pixel labels from multiple image sources, across resolutions, and with high generalization. Compared with other remote sensing land cover types, farmland has rich and diverse appearances, and the semantic features of farmland in different regions exhibit differences and irregularities. Currently proposed datasets typically use remote sensing images from the same geographical area and a single image source, lacking data diversity from real-world scenarios. Deep learning network models built on such datasets have severely insufficient generalization ability, making them difficult to apply to large-area farmland extraction tasks, seriously hindering the research and application of high-resolution remote sensing image farmland extraction methods. From the perspective of algorithm design, current deep learning-based remote sensing farmland extraction still faces several key challenges. First, farmland exhibits varying scale, color, and texture characteristics in remote sensing images, requiring deep learning methods to combine local features with global contextual information. Current research lacks methods that can simultaneously consider both types of features. Second, some land cover categories besides farmland may also display similar features in images. Furthermore, the uneven distribution of samples across different land cover types in the dataset makes existing deep learning methods susceptible to interference from other categories during training, limiting further improvements in farmland extraction accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for extracting farmland information from remote sensing images based on multi-scale global-local information aggregation. This method utilizes a multi-scale global-local information aggregation network to automatically learn the features of farmland and other typical land features from remote sensing images, thereby achieving intelligent extraction of farmland and other typical land feature elements with high extraction accuracy.

[0005] To achieve the above-mentioned objectives, this invention provides a method for extracting farmland information from images based on multi-scale global-local information aggregation, characterized by comprising the following steps:

[0006] (1) Construct a remote sensing image dataset;

[0007] (1.1) Download multiple remote sensing images and use a semantic segmentation and annotation tool to perform polygon annotation on the background and farmland in the remote sensing images to obtain semantic labels;

[0008] (1.2) Set the pixel values ​​corresponding to the background and cultivated land to 0 and 1 respectively to generate a binary semantic label image, wherein the pixel value of the background area is set to 0 and the pixel value of the cultivated land is set to 1.

[0009] (1.3) Each remote sensing image and its corresponding semantic label image are cropped into H×W patches, where the j-th remote sensing image patch after cropping from the i-th remote sensing image is denoted as image. i,j The j-th remote sensing image patch after cropping the i-th semantic label image is the label. i,j Where i represents the image number and j represents the image patch number after cropping; image i,j and label i,j The image size is H×W;

[0010] (1.4) Transfer the remote sensing image patch i,j With the corresponding semantic label image patch label i,j As a set of training data, thus forming the training dataset Dataset self ;

[0011] (2) Construct and train a multi-scale global-local information aggregation network;

[0012] (2.1) From the training dataset Dataset self Select a set of training data images i,j and label i,j and the remote sensing image patch i,j As input to a multi-scale global-local information aggregation network;

[0013] (2.2) The multi-scale global-local information aggregation network starts with the encoder, which includes two branches: ResNet and SegFormer. The ResNet branch contains one downsampling module and four cascaded residual block units. The downsampling module includes a 7x7 convolutional layer and a max pooling layer. Each cascaded residual block unit includes two sets of residual blocks, each set of residual blocks contains two 3×3 convolutional layers, two batch normalization layers, and two ReLU activation functions. The SegFormer branch contains four encoding stages. Each encoder stage contains one overlapping image patch embedding layer and several Transformer modules. The number of Transformer modules corresponding to the four stages in the SegFormer branch are [3, 4, 6, 3], respectively. Each Transformer module consists of one efficient self-attention layer, two hybrid feedforward network layers, and two layer normalization layers.

[0014] After the remote sensing image undergoes encoding operations through two branches, the multi-scale local coding features F are obtained by the ResNet branch. k L The multi-scale global encoding feature F is obtained from the SegFormer branch. k G Then F k L and F k G The input is fed into the feature aggregation attention module; the multi-scale local encoded features and multi-scale global encoded features are processed by the feature aggregation attention module on a scale-by-scale basis to obtain the multi-scale joint feature F. k U Then F k U The input is fed into the decoder for decoding; where k = 1, 2, 3, 4 represents different feature map scales; for F1 L F1 G and F1 U The feature map size is H / 4×W / 4×C1; for F2 L F2 G and F2 U The feature map size is H / 8×W / 8×C2; for F3 L F3 G and F3 U The feature map size is H / 16×W / 16×C3; for F4 L F4 G and F4 U The feature map size is H / 32×W / 32×C4;

[0015] Once the encoding operation is complete, the decoding operation begins. The decoder consists of three Global-Local Information Mining Modules (GLM Blocks) numbered GLM1, GLM2, and GLM3, three Learnable Progressive Upsampling Units (LPUs) numbered LPU1, LPU2, and LPU3, and a Feature Refinement Residual Segmentation Head (FRRSH). In addition, only during the training phase, an Auxiliary Prediction Module (ASH) is introduced into the decoder to calculate the auxiliary prediction loss.

[0016] F4 with feature map size of H / 32×W / 32×C4 U The input is fed into GLM3, resulting in a feature map with dimensions of H / 32×W / 32×C. d Multi-scale decoding feature F3 D and F2 U The feature maps are obtained by passing them through LPU3 together.

[0017] The input is fed into GLM2, resulting in a feature map with dimensions H / 16×W / 16×C. d Multi-scale decoding features F2 D and F2 U The feature maps are obtained by passing them through LPU2 together.

[0018] The input is fed into GLM1, resulting in a feature map with dimensions H / 8×W / 8×C. d Multi-scale decoding features F1 D and F1 U The feature map is obtained by passing through LPU1 together.

[0019] The data is input into FRRSH, and the final output is the farmland information extraction result map. i,j Its size is the same as the input remote sensing image patch. i,j Same dimensions; feature map dimensions are H / 32×W / 32×C d Multi-scale decoding feature F3 D Perform a 4x upsampling for feature map sizes of H / 16×W / 16×C. d Multi-scale decoding features Perform a 2x upsampling, then sum the feature map sizes to H / 8×W / 8×C. d Multi-scale decoding features The channels are summed, and then the result is input into the ASH (Help-Assisted Prediction) module to obtain the output-auxiliary farmland information extraction result image. i,j Its size is the same as the input remote sensing image patch. i,j Same size;

[0020] Output the final generated farmland information extraction result map. i,j and auxiliary cultivated land information extraction result map output-aux i,j With the corresponding semantic label image patch label i,j Perform loss function value Loss total Calculation:

[0021] Loss total =Loss primary +0.4*Loss auxilary

[0022] Among them, Loss primary The output is the result map extracted from cultivated land information. i,j and corresponding semantic label image patch label i,j Calculate the main loss function value, Lossprimary The specific calculation formula is as follows:

[0023]

[0024] Where Z is the normalization factor, ξ(t) is a cosine annealing function that monotonically decreases with the number of training steps t, annealing_step is the predetermined total number of training steps, and p l Output the results of farmland information extraction. i,j The predicted probability value of the l-th pixel, y l The corresponding semantic label image patch label i,j The true probability value of the l-th pixel, L dice It is the dice coefficient loss function, L soft_ce It is the soft cross-entropy loss function;

[0025] Loss auxilary It uses auxiliary cultivated land information to extract the output-aux map. i,j and corresponding semantic label image patch label i,j Calculated auxiliary loss function value; Auxiliary loss function Loss auxilary The soft cross-entropy loss function is used for calculation, and the specific calculation formula is as follows:

[0026]

[0027] Where, p l 'output-aux' is used to extract farmland information. i,j The predicted probability value of the l-th pixel, y l The corresponding semantic label image patch label i,j The true probability value of the l-th pixel;

[0028] Finally, based on the loss function value Loss total The parameters of the multi-scale global-local information aggregation network are updated using gradient descent, and then the training is repeated according to steps (2.1)-(2.2) until the multi-scale global-local information aggregation network converges.

[0029] (3) Extraction of large-area cultivated land information based on central belief;

[0030] (3.1) Let the overlap ratio of sliding window cropping be ρ. Then, for a large-area remote sensing image of size M×N, a window of size H×W is used for sliding cropping. The actual horizontal movement distance of each window is (1-ρ)×W, and the actual vertical movement distance is (1-ρ)×H. All the obtained overlapping sliding window cropped image slices are stored in memory in order according to their positions in the original large-area remote sensing image.

[0031] (3.2) Perform batch prediction on image slices cropped by overlapping sliding windows;

[0032] A batch of image slices is extracted sequentially from memory and input into the trained multi-scale global-local information aggregation network, thereby outputting the label values ​​0 and 1 corresponding to the background and cultivated land in each image slice;

[0033] (3.3) First, generate a binary prediction map of size H×W based on the label value of each image slice. Then, select the central region of size H(1-ρ)×W(1-ρ) as the effective remapping area. Finally, obtain a large-area farmland information extraction map of size M×N. Then, map the label value to the color range to form a visualized large-area farmland information extraction image.

[0034] The objective of this invention is achieved as follows:

[0035] This invention relates to a method for extracting farmland information from remote sensing images based on multi-scale global-local information aggregation. First, a multi-scale global-local information aggregation network is built and trained, enabling it to accurately identify the background and farmland in remote sensing images after convergence. Then, a large-area remote sensing image is cropped using an overlapping sliding window method. The cropped image slices are then batched and input into the trained multi-scale global-local information aggregation network, thereby outputting the label values ​​corresponding to the background and farmland in each image slice. Finally, remapping is performed based on the prediction results of each image slice to form a visualized large-area farmland information extraction map.

[0036] Meanwhile, the method for extracting farmland information from remote sensing images based on multi-scale global-local information aggregation also has the following beneficial effects:

[0037] (1) Based on the encoder-decoder structure, the present invention combines convolutional neural networks and self-attention networks to construct a bilateral encoder and a global-local information mining module, combining global features and local information to improve the accuracy of farmland information extraction, reduce the mixing with other land cover categories, and realize the intelligent extraction of farmland and other typical land cover elements.

[0038] (2) In view of the problem that farmland targets in remote sensing images are usually small in area, fragmented, and widely distributed, this invention introduces a bilateral encoder and a global-local information mining module to efficiently extract global features and local information and fuse the two to form a model.

[0039] (3) To address the issue of farmland scale variation caused by the inherent image resolution differences of multi-source remote sensing images and the rich geographical and topographical conditions, a multi-scale progressive upsampling structure is proposed. This structure focuses on fusing the features of multi-scale encoders and decoders, progressively restoring the resolution of the feature map layer by layer, and providing high-precision pixel-level farmland and other land cover category extraction results in the final stage of the network.

[0040] (4) In view of the large differences in similar land features and class ratios of farmland in remote sensing images, this invention introduces a loss function constraint based on a difficult sample mining mechanism to encourage the network to focus on difficult samples that cannot be accurately extracted due to confusion between different classes and low class ratios during the training process, thereby improving the accuracy of farmland information extraction from remote sensing.

[0041] (5) To address the problem of missing context and obvious splicing traces in the predicted slice edge region of farmland extraction results in large-area remote sensing images, this invention proposes an optimized sliding window prediction method based on center confidence, which mainly includes two improvements: overlapping sliding window and edge ignoring, thereby achieving efficient and high-precision extraction of large-area farmland information. Attached Figure Description

[0042] Figure 1 This is an overall structural diagram of the multi-scale global-local information aggregation network of the present invention;

[0043] Figure 2 This is a ResNet architecture diagram;

[0044] Figure 3 This is a diagram of the SegFormer structure;

[0045] Figure 4 This is a structural diagram of the global-local information mining module;

[0046] Figure 5 This is a diagram of the feature-refined residual segmentation head structure;

[0047] Figure 6 This is a structural diagram of the auxiliary prediction module;

[0048] Figure 7 This is a flowchart of the process for extracting information on large-area cultivated land based on central confidence.

[0049] Figure 8 This is a map showing the results of farmland extraction in a certain region;

[0050] Figure 9This is a map showing the results of farmland extraction in two specific regions;

[0051] Figure 10 This is a comparison chart of the farmland extraction results of this invention with those of other methods. Detailed Implementation

[0052] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0053] Example

[0054] For ease of description, the relevant technical terms appearing in the specific implementation method will be explained first:

[0055] ResNet (Residual Neural Network): A residual neural network;

[0056] GLM Block (Global LocalMining Block): Global-local information mining module;

[0057] LPU (Learnable Progressive Upsampling): Learnable progressive upsampling module;

[0058] ASH (Auxiliary Segmentation Head): Auxiliary prediction module;

[0059] FRRSH (Feature Refinement Residual Segmentation Head): Feature refinement residual segmentation head;

[0060] GLCANet (Global Local Context Aggregation Network): A multi-scale global-local context aggregation network.

[0061] In this embodiment, the present invention provides a method for extracting farmland information from remote sensing images based on multi-scale global-local information aggregation, comprising the following steps:

[0062] (1) Construct a remote sensing image dataset;

[0063] (1.1) Download multiple remote sensing images and use a semantic segmentation and annotation tool to perform polygon annotation on the background and farmland in the remote sensing images to obtain semantic labels;

[0064] (1.2) Set the pixel values ​​corresponding to the background and cultivated land to 0 and 1 respectively to generate a binary semantic label image, wherein the pixel value of the background area is set to 0 and the pixel value of the cultivated land is set to 1.

[0065] (1.3) Each remote sensing image and its corresponding semantic label image are cropped into H×W patches, where the j-th remote sensing image patch after cropping from the i-th remote sensing image is denoted as image. i,j The j-th remote sensing image patch after cropping the i-th semantic label image is the label. i,j Where i represents the image number and j represents the image patch number after cropping; image i,j and label i,j The image size is H×W;

[0066] (1.4) Transfer the remote sensing image patch i,j With the corresponding semantic label image patch label i,j As a set of training data, thus forming the training dataset Dataset self ;

[0067] (2) Construct and train a multi-scale global-local information aggregation network;

[0068] (2.1) From the training dataset Dataset self Select a set of training data images i,j and label i,j and the remote sensing image patch i,j As input to a multi-scale global-local information aggregation network;

[0069] (2.2) As Figure 1 As shown, the multi-scale global-local information aggregation network starts with the encoder, which contains two branches: a ResNet branch and a SegFormer branch; as Figure 2 As shown in (a), the ResNet branch contains one downsampling module and four cascaded residual block units; the downsampling module includes a 7x7 convolutional layer and a max-pooling layer; each cascaded residual block unit includes two sets of residual blocks, as shown in (a). Figure 2 As shown in (b), each residual block contains two 3×3 convolutional layers, two batch normalization layers, and two ReLU activation functions; Figure 3As shown in (a), the SegFormer branch contains four encoding stages; each encoder stage contains one overlapping image patch embedding layer and several Transformer modules. The number of Transformer modules corresponding to the four stages in the SegFormer branch are [3, 4, 6, 3], respectively; where... Figure 3 As shown in (b), each Transformer module consists of one efficient self-attention layer, one hybrid feedforward network layer, and two layer normalization layers;

[0070] After the remote sensing image undergoes encoding operations through two branches, the multi-scale local coding features F are obtained by the ResNet branch. i L For i = 1, 2, 3, 4, the multi-scale global coding features F are obtained from the SegFormer branch. i G i = 1, 2, 3, 4, then F i L i = 1, 2, 3, 4 and F i G The values ​​i = 1, 2, 3, 4 are input into the feature aggregation attention module; the multi-scale local encoded features and multi-scale global encoded features are processed by the feature aggregation attention module on a scale-by-scale basis to obtain the multi-scale joint feature F. i U i = 1, 2, 3, 4; then F i U The inputs i = 1, 2, 3, 4 are fed into the decoder for decoding; where i represents different feature map scales; for F1 L F1 G and F1 U The feature map size is H / 4×W / 4×C1; for F2 L F2 G and F2 U The feature map size is H / 8×W / 8×C2; for F3 L F3 G and F3 U The feature map size is H / 16×W / 16×C3; for F4 L F4 G and F4 U The feature map size is H / 32×W / 32×C4;

[0071] Once the encoding operation is complete, the decoding operation begins. The decoder consists of three Global-Local Information Mining Modules (GLM Blocks) numbered GLM1, GLM2, and GLM3, three Learnable Progressive Upsampling Units (LPUs) numbered LPU1, LPU2, and LPU3, and a Feature Refinement Residual Segmentation Head (FRRSH). In addition, only during the training phase, an Auxiliary Prediction Module (ASH) is introduced into the decoder to calculate the auxiliary prediction loss.

[0072] Among them, such as Figure 4 As shown in (a), each Global-Local Information Mining Block (GLM Block) consists of two regularization layers, a hybrid feedforward network layer, and a global-local attention layer; the structure of the global-local attention layer is as follows. Figure 4 As shown in (b), the input feature map is processed through a global branch and a local branch to obtain global and local context information, respectively, and then the two branches are summed. The global branch contains a window-based multi-head self-attention layer and a global connection compression module. The local branch contains three parallel branches of convolutional layers and batch normalization layers (Batch Norm), with kernel sizes of 1×1, 3×3, and 5×5, respectively. The feature map after summing the global and local branches is processed by an 8×8 deep convolutional layer, a batch normalization layer (Batch Norm), and a ReLU6 activation function to obtain the output of the global-local attention layer.

[0073] like Figure 5 As shown, the structure of the Feature Refinement Residual Segmentation Head (FRRSH) is as follows: First, the input feature map... After processing by a 3×3 convolutional layer, and following spatial and channel branches, the attention features generated by these two paths are fused through a summation operation. The fused feature map is then processed by a deep convolutional layer, a batch normalization layer (BatchNorm), and a ReLU6 activation function, and then summed and fused with the residual preservation branch. Next, it is processed by a 1×1 convolutional layer, a batch normalization layer (BatchNorm), a ReLU6 activation function, and a semantic segmentation head to obtain the final farmland information extraction result. The semantic segmentation head includes a 3×3 convolutional layer, a batch normalization layer (BatchNorm), a ReLU6 activation function, a 1×1 convolutional layer, and a 4x upsampling layer.

[0074] The feature map size is H / 32×W / 32×C4 The input is fed into GLM3, resulting in a feature map with dimensions of H / 32×W / 32×C. d Multi-scale decoding features and The feature maps are obtained by passing them through LPU3 together.

[0075] The input is fed into GLM2, resulting in a feature map with dimensions H / 16×W / 16×C. d Multi-scale decoding features and The feature maps are obtained by passing them through LPU2 together.

[0076] The input is fed into GLM1, resulting in a feature map with dimensions H / 8×W / 8×C. d Multi-scale decoding features and The feature map is obtained by passing through LPU1 together.

[0077] The data is input into FRRSH, and the final result is an image of the extracted farmland information. output Its size is the same as the input remote sensing image patch. i,j The dimensions are the same, h×w×3;

[0078] In this embodiment, Figure 6 The output of the auxiliary prediction module for generating auxiliary farmland information extraction results is described. i,j The process: For feature maps with dimensions H / 32×W / 32×C d Multi-scale decoding features Perform a 4x upsampling for feature map sizes of H / 16×W / 16×C. d Multi-scale decoding features Perform a 2x upsampling, then sum the feature map sizes to H / 8×W / 8×C. d Multi-scale decoding features The channels are summed, and then the result is input into the ASH (Help-Assisted Prediction) module to obtain the output-auxiliary farmland information extraction result image. i,j Its size is the same as the input remote sensing image patch. i,j Same size;

[0079] Output the final generated farmland information extraction result map. i,j Output-aux diagram of auxiliary cultivated land information extraction results i,j With the corresponding semantic label image patch label i,j Perform loss function value Loss total Calculation:

[0080] Loss total =Loss primary +0.4*Loss auxilary

[0081] Among them, Loss primary The output is the result map extracted from cultivated land information. i,j and corresponding semantic label image patch label i,j Calculate the main loss function value, Loss primary The specific calculation formula is as follows:

[0082]

[0083]

[0084] Where Z is the normalization factor, ξ(t) is a cosine annealing function that monotonically decreases with the number of training steps t, annealing_step is the predetermined total number of training steps, and p l Output the results of farmland information extraction. i,j The predicted probability value of the l-th pixel, y l The corresponding semantic label image patch label i,j The true probability value of the l-th pixel, L dice It is the dice coefficient loss function, L soft_ce It is the soft cross-entropy loss function;

[0085] Loss auxilary It uses auxiliary cultivated land information to extract the output-aux map. i,j and corresponding semantic label image patch label i,j Calculated auxiliary loss function value; Auxiliary loss function Loss auxilary The soft cross-entropy loss function is used for calculation, and the specific calculation formula is as follows:

[0086]

[0087] Where, p l 'output-aux' is used to extract farmland information. i,j The predicted probability value of the l-th pixel, y l The corresponding semantic label image patch label i,j The true probability value of the l-th pixel;

[0088] Finally, based on the loss function value Loss total The parameters of the multi-scale global-local information aggregation network are updated using gradient descent, and then the training is repeated according to steps (2.1)-(2.2) until the multi-scale global-local information aggregation network converges.

[0089] (3) Generation of information extraction results for large-area cultivated land based on central confidence;

[0090] (3.1) For large-area remote sensing images that require farmland information extraction, an overlapping sliding window cropping is performed. When cropping the sliding window, a certain proportion of overlapping area is designed to retain sufficient image context information.

[0091] Let the overlap ratio during sliding window cropping be ρ. Then, for a large-area remote sensing image of size M×N, a window of size H×W is used for sliding cropping. The actual horizontal movement distance of each window is (1-ρ)×W, and the actual vertical movement distance is (1-ρ)×H. All the obtained overlapping sliding window cropped image slices are stored in memory in order according to their positions in the original large-area remote sensing image.

[0092] (3.2) Perform batch prediction on image slices cropped by overlapping sliding windows;

[0093] A batch of image slices is extracted sequentially from memory and input into the trained multi-scale global-local information aggregation network, thereby outputting the label values ​​0 and 1 corresponding to the background and cultivated land in each image slice;

[0094] (3.3) Remap the processed prediction results back to the corresponding positions in the original remote sensing image to generate a complete large-area farmland information extraction map;

[0095] This invention proposes a strategy based on center confidence to remap the processed prediction results back to the corresponding locations in the original remote sensing image, thereby generating a complete large-area farmland information extraction map, such as... Figure 7 As shown, specifically, this invention ignores the low-confidence prediction results in the edge regions and only uses the central region of the prediction results for remapping.

[0096] Therefore, a binary prediction map of size H×W is first generated based on the label values ​​of each image slice. Figure 7 The region is denoted as region A; then, the central region with size H(1-ρ)×W(1-ρ) is selected as the effective remapping area. Figure 7 The area is represented as region a; finally, a large-area farmland information extraction map with a size of M×N is obtained, and then the label values ​​are mapped to the color range to form a visualized large-area farmland information extraction image.

[0097] In this embodiment, as Figure 8 and Figure 9 As shown, Figure 8 The following is a map showing the results of farmland extraction in a certain region in this embodiment. Figure 9 The following are images showing the farmland extraction results in two specific regions according to this embodiment; Figure 10The document presents detailed visualization results of farmland extraction in different areas, demonstrating that this invention significantly improves the accuracy of farmland information extraction from remote sensing imagery. As can be seen from the first column of remote sensing images, there are significant differences in the scale and morphology of farmland across different images. For example... Figure 10 The farmland plots in rows (c) and (d) are relatively small and fragmented, while the farmland plots in row (e) are much larger in scale than those in the other rows. However, compared to other networks, GLCANet performs better in extracting farmland from multi-source images and can adapt to farmland scenes with diverse scales and features. Figure 10 In rows (a) and (d), other networks misclassified vegetation categories with similar characteristics to cultivated land, while GLCANet achieved accurate differentiation by efficiently extracting global context and local features. Figure 10 In the middle (b) row, there are both cultivated lands where crops are being planted and cultivated lands that have been harvested, and GLCANet accurately extracts all cultivated lands.

[0098] In summary, this invention addresses the multiple challenges of extracting farmland information from remote sensing images by proposing a method based on multi-scale global-local information aggregation. This method utilizes a bilateral encoder combining a ResNet convolutional neural network and a SegFormer self-attention network, along with a global-local information mining module, significantly improving the extraction accuracy of farmland and other typical land features. Specifically, in the encoder-decoder structure, this invention employs a bilateral encoder and a global-local information mining module, combining global features with local information to improve the extraction accuracy of farmland information, reduce contamination with other land feature categories, and achieve intelligent extraction of farmland and other typical land feature elements. To address the issue that farmland targets in remote sensing images are typically small, fragmented, and widely distributed, an efficient feature extraction and fusion modeling mechanism is introduced. Through a multi-scale progressive upsampling structure, the resolution of the feature map is restored layer by layer, solving the problem of scale variations in farmland and ensuring high-precision pixel-level extraction results. Furthermore, this invention introduces a loss function based on a hard sample mining mechanism, prompting the network to focus on difficult samples that are hard to extract accurately due to out-of-class confusion and low class ratios during training, thereby improving the accuracy of remote sensing farmland information extraction. Addressing the issues of missing context and obvious splicing traces in the predicted slice edges of farmland extraction results in large-area remote sensing images, an optimized sliding window prediction method based on center confidence is proposed. Through improvements in overlapping sliding windows and edge ignoring, efficient and high-precision extraction of large-area farmland information is achieved.

[0099] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A method for extracting farmland information from images based on multi-scale global-local information aggregation, characterized in that, Includes the following steps: (1) Construct a remote sensing image dataset; (1.1) Download multiple remote sensing images and use a semantic segmentation and annotation tool to perform polygon annotation on the background and farmland in the remote sensing images to obtain semantic labels; (1.2) Set the pixel values ​​corresponding to the background and cultivated land to 0 and 1 respectively to generate a binary semantic label image, wherein the pixel value of the background area is set to 0 and the pixel value of the cultivated land is set to 1. (1.3) Cropping each remote sensing image and its corresponding semantic label image into... A tile of a certain size, where is denoted as the first. The first remote sensing image after cropping One remote sensing image patch is The corresponding number The first semantic tag image after cropping One remote sensing image patch is ;in Indicates the number of the remote sensing image. Indicates the number of the cropped remote sensing image patch; and The image size is ; (1.4) Patch the remote sensing image With corresponding semantic label image patches This serves as a set of training data, thus forming the training dataset. ; (2) Construct and train a multi-scale global-local information aggregation network; (2.1) From the training dataset Select a set of training data and and the remote sensing image patch As input to a multi-scale global-local information aggregation network; (2.2) The multi-scale global-local information aggregation network starts with the encoder, which contains two branches: a ResNet branch and a SegFormer branch. The ResNet branch contains one downsampling module and four cascaded residual block units. Each downsampling module includes a 7x7 convolutional layer and a max-pooling layer. Each cascaded residual block unit includes two sets of residual blocks, each set containing two 3x3 convolutional layers, two batch normalization layers, and two ReLU activation functions. The SegFormer branch contains four encoding stages. Each encoder stage contains one overlapping image patch embedding layer and several Transformer modules. The number of Transformer modules corresponding to the four stages in the SegFormer branch are as follows: Each Transformer module consists of one efficient self-attention layer, two hybrid feedforward network layers, and two layer normalization layers. After the remote sensing image undergoes encoding operations through two branches, the ResNet branch yields multi-scale local encoded features. Multi-scale global encoded features are obtained from the SegFormer branch. Then and The input is fed into the feature aggregation attention module; the multi-scale local encoded features and multi-scale global encoded features are processed by the feature aggregation attention module on a scale-by-scale basis to obtain the multi-scale joint features. Then The input is fed into the decoder for decoding; where, Representing different feature map scales; for , and The feature map size is ;for , and The feature map size is ;for , and The feature map size is ;for , and The feature map size is ; Encoding operation complete, proceed to decoding operation; the decoder consists of three numbers... , and The Global-Local Information Mining Module (GLM Block) consists of three modules numbered as follows: , and The learnable progressive upsampling LPU and the feature refinement residual segmentation head FRRSH; in addition, only during the training phase, an auxiliary prediction module ASH is introduced into the decoder to calculate the auxiliary prediction loss; Feature map size is of Input to In the process, the feature map size is obtained as Multi-scale decoding features and Together , to obtain feature map ; Input to In the process, the feature map size is obtained as Multi-scale decoding features and Together , to obtain feature map ; Input to In the process, the feature map size is obtained as Multi-scale decoding features and Together , to obtain feature map ; Input to In the end, the resulting map of extracted farmland information was obtained. Its size is the same as the input remote sensing image patch. Same size; for feature map size Multi-scale decoding features Perform 4x upsampling on the feature map with a size of Multi-scale decoding features Perform a 2x upsampling, then sum the feature map size to... Multi-scale decoding features The channels are summed, and then the result is input into the ASH (Auxiliary Prediction) module to obtain the auxiliary farmland information extraction result image. Its size is the same as the input remote sensing image patch. Same size; The final generated farmland information extraction result map And auxiliary farmland information extraction results map With corresponding semantic label image patches Perform loss function value Calculation: ； in, It is a map extracted using farmland information. and corresponding semantic label image patches Calculate the value of the main loss function; It is a map extracted using auxiliary cultivated land information. and corresponding semantic label image patches The calculated value of the auxiliary loss function; The main loss function value The specific calculation formula is as follows: ； ； ； ； Where Z is the normalization factor. It depends on the number of steps during the training process. The cosine annealing function is monotonically decreasing. It is the predetermined total number of training steps. Image of farmland information extraction results Middle The predicted probability value of each pixel. It is the corresponding semantic label image patch Middle The true probability value of each pixel. It is the dice coefficient loss function. It is the soft cross-entropy loss function; The auxiliary loss function The soft cross-entropy loss function is used for calculation, and the specific calculation formula is as follows: ； in, To assist in the extraction of farmland information result map Middle The predicted probability value of each pixel. It is the corresponding semantic label image patch Middle The true probability value of each pixel; Finally, based on the loss function value The parameters of the multi-scale global-local information aggregation network are updated using gradient descent, and then the training is repeated according to steps (2.1)-(2.2) until the multi-scale global-local information aggregation network converges. (3) Extraction of large-area cultivated land information based on central confidence; (3.1) Let the overlap ratio during sliding window clipping be... So for a size of Large-area remote sensing images, using a size of The windows are slid-trimmed, and the actual horizontal movement distance of each window is... The actual distance traveled in the vertical direction is All the acquired overlapping sliding window cropped image slices are stored in memory in order according to their positions in the original large-area remote sensing image; (3.2) Perform batch prediction on image slices cropped by overlapping sliding windows; A batch of image slices is extracted sequentially from memory and input into the trained multi-scale global-local information aggregation network, thereby outputting the label values ​​0 and 1 corresponding to the background and cultivated land in each image slice; (3.3) First, generate a size based on the label value of each image slice. The binary prediction map, and then select the one with size. The central region is the effective remapping area, resulting in a final size of The large-area farmland information extraction map is then mapped to the color range to form a visualized large-area farmland information extraction image.

Images