Method and device for extracting farmland distribution information based on remote sensing images

By automatically generating prompt points and combining image encoding and decoding technology, the complexity of farmland extraction in remote sensing images is solved, and efficient and accurate farmland recognition is achieved, which is suitable for large-scale agricultural monitoring.

CN118262251BActive Publication Date: 2024-11-22BEIJING NORMAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410251737.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-05
Publication Date
2024-11-22
Estimated Expiration
2044-03-05

AI Technical Summary

Technical Problem

The prior art is difficult to accurately process complex land objects, especially soft boundary features and diverse crop planting patterns when extracting farmlands in remote sensing images, and deep learning methods have insufficient generalization capabilities and efficiency, especially on large-scale or complex terrain. In case.

Method used

By automatically generating prompt points with appropriate number and correct locations, combining image encoder, prompt point generation module and mask decoder, the prompt point position is optimized by using gradient values, category homogeneity index and planting index to realize farmland in remote sensing images Accurate splitting of plots.

Benefits of technology

提高了农田识别的准确性和效率,特别是在复杂种植农田的处理上,提供更精确的边界提取和空间分辨率,适用于大规模农业监测和精准农业,且无需训练样本。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118262251B_ABST
    Figure CN118262251B_ABST
Patent Text Reader

Abstract

Disclosed are a method and device for extracting farmland distribution information based on remote sensing images. In the method, a remote sensing image containing a farmland plot to be measured is obtained; the remote sensing image is processed, a prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, a point positioning optimization module optimizes the candidate prompt points to obtain optimized prompt points and their prompt information, a prompt encoder encodes the prompt information of the prompt points, and then performs joint encoding of embedded representation and prompt point prompt encoding; a mask decoder generates a segmentation mask representing the position and boundary of the farmland plot in the remote sensing image according to the embedded representation and prompt information provided by the prompt encoder; all the field plot vector diagrams extracted from each individual image are superimposed together, and then the field plots generated due to geometric registration errors in the generated vector diagrams are removed to ensure that the smallest field plot in the time series is extracted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing satellites, and in particular to a method and a device for extracting farmland distribution information based on remote sensing images. Background Art

[0002] With the rapid development of remote sensing technology, high-resolution satellite images have been widely used around the world, especially in the fields of agricultural monitoring, environmental protection and land resource management. As the basic unit of agricultural production, the accurate identification and extraction of farmland is of great significance for crop monitoring, yield estimation, land use planning and environmental change monitoring. However, existing farmland extraction technologies face many challenges and limitations. Early farmland extraction technologies mainly relied on traditional image processing methods, such as edge detection, region growing and texture analysis. These methods often rely on the reflectivity characteristics of specific bands and have limited recognition effects on complex objects. Due to the lack of effective processing of internal heterogeneity of farmland, traditional methods are not effective in processing planted farmlands (PFs) with soft boundary features. In addition, manual interpretation has long been the main means of farmland extraction, but it is time-consuming and inefficient, and highly dependent on the experience and expertise of the interpreter. For farmland extraction over large areas or complex terrain, the accuracy and consistency of manual interpretation are difficult to guarantee. In recent years, with the development of deep learning technology, image segmentation technology based on convolutional neural network (CNN) has begun to be applied to farmland extraction. Although deep learning methods have shown superior performance in some applications, these methods usually require a large amount of annotated data and face challenges in processing multi-temporal and multi-source remote sensing data. In addition, since deep learning methods are usually data-driven, they may perform poorly when generalizing to new areas or new conditions.

[0003] Compared with traditional ground-based collection tasks, farmland fields (PFs) extracted from remote sensing images have more complex spatial features, such as irregular boundaries and diverse internal structures. Traditional methods and early deep learning methods often fail to achieve ideal results when extracting PFs with diverse crop planting patterns. Boundary recognition of PFs is important for crop type determination and yield estimation, but existing technologies often fail to accurately distinguish the subtle differences between different fields. The outstanding advantages of SegmentAnything Model (SAM) are: Visual Transformer (ViT) and attention mechanism, which ensures the capture of local and global features of the image and spatial contextual relevance. The largest training dataset to date (SA-1B), including 11 million different high-resolution images and 1.1 billion segmentation masks, ensures the universality of the model, that is, SAM's performance on zero-shot (i.e., no additional sample training required) transfer tasks is usually comparable to or even better than that of task-specific training models. The ability to handle various types of prompts, including points, boxes, or text, enables the model to handle a variety of user-defined image segmentation tasks.

[0004] It remains challenging to directly apply SAM to remote sensing segmentation tasks. The first influencing factor is that SAM is trained on natural images (RGB) rather than remote sensing images. The SA-1B training dataset used for the SAM model mainly consists of natural images, which usually only capture the red, green, and blue bands, while remote sensing images usually contain a wider range of spectral bands (such as near-infrared, red edge, and short-wave infrared bands). Therefore, how to effectively select and utilize the rich and useful spectral information in remote sensing images becomes a key issue in directly applying SAM to remote sensing segmentation tasks. In addition, the performance of SAM depends largely on the quality of the hints. If the hints provided are inaccurate or ambiguous, the segmentation results of the model will be affected, especially in terms of the location and number of hints. In dense remote sensing image segmentation tasks, manually selecting hint points may be impractical.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention proposes a method for extracting farmland distribution information based on remote sensing images, which is used to extract the planting field PF from single-time or multi-time remote sensing images, and automatically generate prompt points with appropriate number and correct positions to accurately extract the spatial distribution of fields in agricultural remote sensing.

[0007] The purpose of the present invention is to achieve this through the following technical solutions: a method for extracting farmland distribution information based on remote sensing images comprises:

[0008] Step 1, obtaining a remote sensing image containing the farmland to be measured;

[0009] Step 2, processing the remote sensing image, wherein the remote sensing image is subjected to image cropping, denoising, and contrast enhancement, and the remote sensing image is processed by an image encoder to generate an embedded representation of the remote sensing image compression and abstract representation;

[0010] Step 3: the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, and the point positioning optimization module optimizes the candidate prompt points to obtain optimized prompt points and their prompt information.

[0011] Step 4, the prompt encoder encodes the prompt information of the prompt point, and then performs joint encoding of the embedded representation and the prompt point prompt encoding;

[0012] Step 5, the mask decoder decodes the joint code output by the hint encoder to obtain the embedded representation of the image compressed and abstractly represented by the remote sensing image and the hint code of the hint point and its hint information, and the mask decoder generates a segmentation mask representing the position and boundary of the farmland plot in the remote sensing image according to the embedded representation and hint information provided by the hint encoder;

[0013] Step 6, obtain each field image in the single-date image, convert them all into vector graphics, and then perform an overlay operation to overlay all the field vector graphics extracted from each individual image. Then, remove the fields generated by geometric registration errors in the generated vector graphics to ensure that the smallest field in the time series is extracted.

[0014] In the method for extracting farmland distribution information based on remote sensing images, in step 3, the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, calculates the gradient value to represent the edge position, and uses the Sobel filter to calculate the gradient map so that the prompt point is far away from the PF edge. img =sobel(x), where x represents the input remote sensing image, Grad img Represents the extracted gradient map.

[0015] In the method for extracting farmland distribution information based on remote sensing images, the category homogeneity index HI is used to compare the category of each pixel with that of adjacent pixels to calculate the category homogeneity of each pixel to obtain a homogeneity map. Among them, I kIt is the category value of each adjacent pixel in a moving window composed of m pixels. When the category value of the kth pixel is the same as that of the central pixel i, the category value is set to 1; otherwise, it is set to 0. The category homogeneity index HI value ranges from 0 to 1. The larger the category value, the more uniform the category value. By combining the category homogeneity index HI and the gradient, a candidate cue point is generated at a position where the HI value is large and the gradient value is small.

[0016] In the method for extracting farmland distribution information based on remote sensing images, the combination of the category homogeneity index HI and the gradient is realized through a weighted index - the planting index PI to obtain a planting index map.

[0017] PI i =ε·(1-G rad i )+(1-ε)·HI i ,

[0018] Among them, the ε variable represents the gradient weight, (1-ε) represents the HI weight, and the calculated value of each pixel is sorted from small to large. The pixels with values ​​greater than the threshold are regarded as candidate pixels of the prompt point.

[0019] In the method for extracting farmland distribution information based on remote sensing images, the point positioning optimization module optimizes the candidate prompt points, creates an expansion window around each candidate prompt point, selects all prompt points within the expansion window, extracts the cultivated farmland PF from all selected prompt points, and evaluates the extraction results based on the prediction score, selects the prompt point with the highest prediction score within the expansion window, retains the position of the prompt point, and removes other prompt points.

[0020] In the method for extracting farmland distribution information based on remote sensing images, all the PFs of planted farmland are arranged in descending order according to the prediction scores, and the PF of planted farmland with the highest score is selected. The non-maximum suppression (NMS) is used to calculate the union intersection (IoU) between the PF with the highest score and each of the remaining PFs in the candidate list to quantify the overlap between the two fields.

[0021] Among them, f1 and f2 represent the two cultivated farmland PFs to be analyzed, area is the overlapping area between the two PFs calculated using the intersection and union operators, and if the joint intersection IoU value of the fields is greater than the threshold, the overlapping fields are deleted.

[0022] In the method for extracting farmland distribution information based on remote sensing images, the prompt includes a point, a box or text.

[0023] In the method for extracting farmland distribution information based on remote sensing images, in step 5,

[0024] Seg img =Vmask (U img (x), U pro (p point )),

[0025] Among them, U img represents the image encoder, U pro Corresponding prompt encoder, V mask refers to the mask decoder, x represents the remote sensing image to be processed, P point Is the prompt information entered by the user, Seg img The corresponding predicted mask.

[0026] In the method for extracting farmland distribution information based on remote sensing images, the prompt point generation module generates a gradient map, a cluster map and a homogeneity map based on the remote sensing image, and then obtains a planting index map based on the gradient map, the cluster map and the homogeneity map, and obtains candidate field prompt points based on the planting index map.

[0027] In the method for extracting farmland distribution information based on remote sensing images, in step 6, for remote sensing images from different time points, an edge overlapping module is used to process multi-temporal data, and farmland information reflecting seasonal changes and dynamic characteristics of farmland at different time points is integrated.

[0028] In addition, the present invention also discloses a farmland distribution information extraction device based on remote sensing images, which includes:

[0029] The first unit is used to obtain a remote sensing image containing a farmland plot to be measured;

[0030] A second unit is used to process the remote sensing image, wherein the remote sensing image is subjected to image cropping, denoising, and contrast enhancement, and the remote sensing image is processed through an image encoder to generate an embedded representation of an image that compresses and abstractly represents the remote sensing image;

[0031] The third unit is used for the prompt point generation module to generate candidate prompt points and prompt information based on the remote sensing image, and the point positioning optimization module optimizes the candidate prompt points and prompt information to obtain optimized prompt points and prompt information.

[0032] A fourth unit is used for encoding the cue information of the cue point through a cue encoder, and then performing joint encoding of the embedded representation and the cue point and its cue information;

[0033] The fifth unit is used to decode the joint code output by the hint encoder through the mask decoder to obtain the embedded representation of the image compressed and abstractly represented by the remote sensing image and the hint code of the hint point and its hint information, and then the mask decoder generates a segmentation mask representing the position and boundary of the farmland plot in the remote sensing image according to the embedded representation and hint information provided by the hint encoder;

[0034] The sixth unit is used to obtain all the field images in the single-date remote sensing image, convert them all into vector graphics, and then perform an overlay operation to overlay all the vector graphics extracted from each remote sensing image, and then remove the fields generated by geometric registration errors in the generated vector graphics to ensure that the smallest field in the time series is extracted.

[0035] Compared with the prior art, the present invention has the following advantages: the present invention can use a single date remote sensing image or a time series image to automatically generate appropriate and accurate prompts based on the PF and remote sensing image. The zero-shot capability of SAM in the high-intensity task of PF segmentation across the study area or across images of different spatial resolutions is further improved without reducing efficiency. The automated, efficient and accurate farmland extraction scheme, with special emphasis on the effective processing of PFs with soft boundaries, can effectively improve the accuracy and efficiency of farmland identification, especially for complex planted farmlands (PFs). The method can provide more accurate boundary extraction and better spatial resolution, and has important application value for large-scale agricultural monitoring and precision agriculture. No training samples are required, but higher accuracy is achieved than the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] By reading the detailed description of the preferred specific embodiments below, various other advantages and benefits of the present invention will become clear to those of ordinary skill in the art. The drawings in the specification are only for the purpose of illustrating the preferred embodiments and are not considered to be limitations of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work. Moreover, the same reference numerals are used to represent the same components throughout the drawings.

[0037] In the attached picture:

[0038] Figure 1 A schematic diagram of the process of the present invention is shown;

[0039] Figure 2 A schematic diagram showing the results of comparing the spatial distribution of the field plots with the distribution of the actual samples;

[0040] Figure 3 A schematic diagram showing the process of automatically generating cue points;

[0041] Figure 4 A schematic diagram of the point positioning optimization process is shown.

[0042] Figure 5 Schematic diagram showing the visual comparison of field extraction results generated by IAFRes and STAMP for four study areas.

[0043] Figure 6 Schematic diagram showing fields extracted from multi-temporal images using STAMP (ae) and the field distribution obtained by the edge overlap module (f), where the sub-region (gi) highlights the contrast between the spatial extent of the fields extracted at different time points and the field distribution obtained by the edge overlap module.

[0044] The present invention is further explained below in conjunction with the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0045] The specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the specific embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0046] It should be noted that certain words are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. This specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of the components as the criterion for distinction. As mentioned throughout the specification and claims, "including" or "comprising" is an open term, so it should be interpreted as "including but not limited to". The subsequent description of the specification is a preferred embodiment of the present invention, but the description is based on the general principles of the specification and is not intended to limit the scope of the present invention. The scope of protection of the present invention shall be determined by the attached claims.

[0047] To facilitate understanding of the embodiments of the present invention, further explanation will be given below by taking specific embodiments as examples in conjunction with the accompanying drawings, and each of the accompanying drawings does not constitute a limitation on the embodiments of the present invention.

[0048] In one embodiment, Figures 1 to 6 As shown, a method for extracting farmland distribution information based on remote sensing images includes:

[0049] Step 1, obtaining a remote sensing image containing the farmland to be measured;

[0050] Step 2, processing the remote sensing image, wherein the remote sensing image is subjected to image cropping, denoising, and contrast enhancement, and the remote sensing image is processed through an image encoder to generate an embedded representation of the compressed and abstract representation of the remote sensing image, wherein the image encoder first divides the input image into multiple fixed-size image patches (Patches). Each image patch is mapped to an embedded representation through a linear layer (fully connected layer). Then, position encoding is added to each embedded vector to retain the spatial information between image patches. After being processed by a Transformer encoding layer, these encoding layers use a regularization layer (Normal Layer) and a multi-head attention mechanism layer (Multi-Head Attention Layer), as well as a multi-layer fully connected layer (MLP) to encode these representations;

[0051] Step 3: the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, and the point positioning optimization module optimizes the candidate prompt points to obtain optimized prompt points.

[0052] Step 4: The prompt encoder encodes the prompt information of the prompt point, and then performs joint encoding of the embedding representation and the prompt point prompt encoding. The prompt encoder and the image encoder have the same structure, but the input prompt point needs to be converted into a vector in a high-dimensional space. This process is completed using the pre-trained word embedding model Transformer;

[0053] Step 5, the mask decoder decodes the joint code output by the hint encoder to obtain the embedded representation of the image of the compressed and abstract representation of the remote sensing image and the hint code of the hint point and its hint information. The mask decoder generates a segmentation mask representing the location and boundary of the farmland plot in the remote sensing image based on the embedded representation and hint information provided by the hint encoder. The mask decoder inputs the features processed by the encoder into the upsampling layer (Upsampling Layers), where upsampling technology including transposed convolution is used to achieve this, and then the result is input into the multi-layer convolutional layer (Convolutional Layers) to better restore the details of the image. In order to more effectively restore the details of the image, the uncompressed features are directly introduced from the encoder using jump connections in the multi-layer convolutional layers to help the model use more contextual information in the reconstruction process. And after each convolutional layer, the activation layer (Activation Layers) is used to introduce nonlinearity, which is achieved using the activation function ReLU (Rectified Linear Unit);

[0054] Step 6, obtain each field image in the single-date image, convert them all into vector graphics, and then perform an overlay operation to overlay all the field vector graphics extracted from each individual image. Then, based on the position information contained in the field, remove the field generated by the geometric registration error in the generated vector graphics to ensure that the smallest field in the time series is extracted.

[0055] It should be noted that the parameter results of the convolutional network in the model of the present invention are obtained from the open source parameter set, and the traditional network training is to obtain the parameter results of the network, so the convolutional network in the present invention does not need to be trained separately. Figure 1 The parameter results of the image encoder in are obtained through the corresponding open source parameter set available on the Internet.

[0056] In a preferred embodiment of the method for extracting farmland distribution information based on remote sensing images, in step 3, the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, calculates the gradient value to represent the edge position, and uses the Sobel filter to calculate the gradient map so that the prompt point is far away from the PF edge. img =sobel(x), where x represents the input remote sensing image, Grad img Represents the extracted gradient map.

[0057] In a preferred embodiment of the method for extracting farmland distribution information based on remote sensing images, the category homogeneity index HI is used to compare the category of each pixel with that of adjacent pixels to calculate the category homogeneity of each pixel to obtain a homogeneity map. Among them, I k It is the category value of each adjacent pixel in a moving window composed of m pixels. When the category value of the kth pixel is the same as that of the central pixel i, the category value is set to 1; otherwise, it is set to 0. The category homogeneity index HI value ranges from 0 to 1. The larger the category value, the more uniform the category value. By combining the category homogeneity index HI and the gradient, a candidate cue point is generated at a position where the HI value is large and the gradient value is small.

[0058] In a preferred embodiment of the method for extracting farmland distribution information based on remote sensing images, the combination of the category homogeneity index HI and the gradient is realized through a weighted index - the planting index PI to obtain a planting index map.

[0059] PI i =ε·(1-G rad i )+(1-ε)·HI i ,

[0060] Among them, the ε variable represents the gradient weight, (1-ε) represents the HI weight, and the calculated value of each pixel is sorted from small to large. The pixels with values ​​greater than the threshold are regarded as candidate pixels of the prompt point.

[0061] In a preferred implementation of the method for extracting farmland distribution information based on remote sensing images, the point positioning optimization module optimizes the candidate prompt points, creates an expansion window around each candidate prompt point, selects all prompt points within the expansion window, and extracts the cultivated farmland PF of all selected prompt points through the model. The model also outputs the predicted probability of each PF, which can be used as a prediction score. The extraction results are evaluated based on the prediction score, and the prompt point with the highest prediction score in the expansion window is selected, and the position of the prompt point is retained, and other prompt points are removed.

[0062] In a preferred implementation of the method for extracting farmland distribution information based on remote sensing images, all the PFs of planted farmlands are arranged in descending order according to the prediction scores, and the PF of planted farmlands with the highest score is selected. The non-maximum suppression (NMS) is used to calculate the union intersection (IoU) between the PF with the highest score and each of the remaining PFs in the candidate list to quantify the overlap between the two fields.

[0063] Among them, f1 and f2 represent the two cultivated farmland PFs to be analyzed, area is the overlapping area between the two PFs calculated using the intersection and union operators, and if the joint intersection IoU value of the fields is greater than the threshold, the overlapping fields are deleted.

[0064] In a preferred implementation of the method for extracting farmland distribution information based on remote sensing images, the prompt includes a point, a box or text.

[0065] In a preferred embodiment of the method for extracting farmland distribution information based on remote sensing images, in step 5,

[0066] Seg img =V mask (U img (x), U pro (p point )),

[0067] Among them, U img represents the image encoder, U pro Corresponding prompt encoder, V mask refers to the mask decoder, x represents the remote sensing image to be processed, P point Is the prompt information entered by the user, Seg img The corresponding predicted mask.

[0068] In a preferred implementation of the method for extracting farmland distribution information based on remote sensing images, the prompt point generation module generates a gradient map, a cluster map and a homogeneity map based on the remote sensing image, and then obtains a planting index map based on the gradient map, the cluster map and the homogeneity map, and obtains candidate field prompt points based on the planting index map.

[0069] In a preferred implementation of the method for extracting farmland distribution information based on remote sensing images, in step 6, for remote sensing images from different time points, an edge overlapping module is used to process multi-temporal data to integrate farmland information reflecting seasonal changes and dynamic characteristics of farmland at different time points.

[0070] In one embodiment, a method for extracting farmland distribution information based on remote sensing images includes, first, using publicly available high-resolution remote sensing data remote sensing images to obtain input data, including but not limited to domestic high-resolution satellites, EU Sentinel-2 satellite images, etc. Then, using the input remote sensing images, standardization processing is performed, including image cropping, denoising, contrast enhancement, etc., to improve image quality in preparation for subsequent model analysis; thereafter, the field extraction model designed by the present invention is used to identify potential farmland areas. According to the features extracted by the deep learning module, prompt points are automatically generated, and these prompt points are used as key areas of the model to guide the precise identification of farmland boundaries.

[0071] Preferably, the method further comprises the steps of: indicating the approximate position of the edge by calculating the gradient value, and using a Sobel filter to calculate the gradient map to ensure that the prompt point is far away from the PF edge.

[0072] Grad img =sobel(x)

[0073] Where x represents the input image of the study area, and Grad img Represents the extracted gradient map.

[0074] Preferably, the method further comprises the step of considering that pixels inside the PF should be more homogeneous in spectral and textural characteristics than pixels near the edge of the PF.Using a class homogeneity index (HI), the class homogeneity of each pixel is calculated by comparing the class of each pixel with neighboring pixels.

[0075] Among them, I k is the category value of each adjacent pixel in a moving window of m pixels. When the category value of the kth pixel is the same as that of the center pixel (i), the value is set to 1; otherwise, it is set to 0. The HI value ranges from 0 to 1, and a larger value indicates a more uniform category value.

[0076] Preferably, the method further comprises the step of generating prompt points at locations where the HI value is large and the gradient value is small by combining the HI and the gradient, ensuring that each prompt point is within the PF range and away from the PF edge. The combination of HI and gradient can be achieved by a weighted index - the planting index (PI). The positions of the prompt points are adjusted to ensure that they can effectively represent the key features of the farmland, especially at the edge of the farmland and in areas with complex internal structures.

[0077] Preferably, the method further comprises the steps of extracting PF from all selected cue points by creating an expansion window around each candidate cue point, evaluating the extraction results according to the prediction scores, selecting the cue point with the highest prediction score within the expansion window for the region, retaining the position of the cue point, and removing other cue points.

[0078] Preferably, the method further comprises the steps of: arranging all predicted PFs in descending order of prediction scores, selecting the PF with the highest score, and adding it to the final list, and calculating the "intersection over union" (IoU) between the PF with the highest score and each remaining PF in the candidate list using non-maximum suppression (NMS). Where f1 and f2 represent the two PFs to be analyzed, and area is the overlapping area between the two PFs calculated using the intersection and union operators. The above-mentioned cue points are constructed and optimized cue points are used to enable more accurate identification and extraction of farmland areas.

[0079] The output of the deep learning model is combined with image processing technology to refine the extraction of farmland boundaries, especially for planted farmlands (PFs) with soft boundaries. Preferably, the method further includes the following steps: evaluating the extraction results, manually adjusting or regenerating prompt points as needed to further improve the extraction accuracy.

[0080] Preferably, the method further comprises the following steps: for remote sensing images from different time points, using an edge overlap module to process multi-temporal data, integrating farmland information at different time points to reflect seasonal changes and dynamic characteristics of the farmland.

[0081] In one embodiment, a method for extracting farmland distribution information based on remote sensing images includes:

[0082] S1. Data input and model initialization

[0083] Data input: Select high-resolution remote sensing images that contain the agricultural plots to be analyzed.

[0084] Model initialization: This algorithm is implemented through Python code and can be directly called in any computer through the host command program.

[0085] S2. Data processing

[0086] S2-1. Image Data Coding

[0087] Data input: remote sensing image data

[0088] Data output: The model processes the input image through the 'image encoder' and generates an embedded representation of the image through this process. This embedding is a compressed and abstract representation of the input image, providing basic information for subsequent mask generation.

[0089] S2-2. Prompt information coding

[0090] Data input: embedded representation of the image and 'point' hint information.

[0091] Data output: The model encodes the input 'point' prompt information through the 'prompt encoder', which is translated into 'point' prompt codes in a format that the model can understand, guiding it to specifically identify the scope of farmland plots in the image. Then, this module implements the joint encoding of the embedded representation of the image output by the 'image encoder' and the 'point' prompt code.

[0092] S2-3. Mask decoding:

[0093] Data input: 'Joint encoding' of the prompt encoder output.

[0094] Data output: The 'joint code' output by the hint encoder is decoded by the 'mask decoder', and the embedded representation of the decoded image provides compressed and abstract information about the entire image, while the hint code provides localization information about specific areas or objects. The main function of the mask decoder in this process is to generate segmentation masks based on the image embedding information provided by the encoder and the hint code information. These masks accurately represent the location and boundaries of farmland plots in remote sensing images.

[0095] S3. Cue point automatic generation module

[0096] S3-1. Cue point generation module, such as Figure 3 shown.

[0097] Data input: remote sensing image data

[0098] Data output: automatically generated prompt 'point' distribution data

[0099] Image processing: Since manually providing hint 'points' in the entire remote sensing image limits the feasibility of the extraction task, it is necessary to automatically generate hint 'points', and the location of the hint 'points' also plays a vital role in the accuracy of segmentation. Since the location of the hint points needs to be far away from the edge, the algorithm can calculate the gradient value to represent the approximate location of the edge. The model calculates the gradient map by using the Sobel filter, which is an effective edge detection technique for identifying edge parts in the image. Then, the hint points are placed at locations with smaller gradient values, ensuring that the hint points are far away from the edge of the field.

[0100] Grad img =sobel(x) (1)

[0101] Parameter description: x represents the input image of the study area, Grad img Represents the extracted gradient map.

[0102] In addition, the location of the cue point needs to be close to the center of the field, and according to another characteristic of farmland plots, the pixels inside the field are more uniform than those near the edge of the field. The category homogeneity of each pixel is calculated by comparing the category of each pixel with that of its neighboring pixels using the category homogeneity index (HI) (Formula 2) according to the research of Zhu et al. (2016) to ensure that the location of the obtained cue point is close to the center of the field. The category information is obtained by an unsupervised classification algorithm, such as ISODATA.

[0103]

[0104] Parameter Description: I k is the category value of each adjacent pixel in a moving window consisting of m pixels. When the category value of the kth pixel is the same as the center pixel (i), the value is set to 1; otherwise, it is set to 0. The HI value ranges from 0 to 1, and the larger the value, the closer the point is to the center of the field.

[0105] By combining HI and gradient, cue points can be generated at locations where HI values ​​are large and gradient values ​​are small, ensuring that each cue point is in the center of the field and away from the edge of the field. The combination of HI and gradient can be achieved through a weighted index - the planting index (PI), as defined in Formula 3. Finally, STAMP generates cue points based on the size of the PI value.

[0106] PI i =ε·(1-G rad i )+(1-ε)·HI i (3)

[0107] Parameter Description: The ε variable represents the gradient weight.

[0108] S3-2. Point positioning optimization module, such as Figure 4 shown.

[0109] Data input: automatically generated prompt 'point' distribution data

[0110] Data output: Optimized prompt 'point' distribution data

[0111] Image processing process: There are a large number of cue points generated by the previous step S3-1, some of which may be redundant and incorrectly located. The main task of this optimization module is to remove redundant points and retain cue points with better locations as optimized cue points. The details of this process are as follows:

[0112] First, a window is created around each candidate cue point. All cue points within the window are selected. Then, the model extracts fields from all cue points in the window and evaluates the extraction results based on the prediction scores. Finally, the cue point with the highest prediction score within the expanded window is selected for the area, and the position of the cue point is retained, while the other cue points are removed. There are two benefits of using an expanded window instead of a traditional window here: first, it expands the search range and makes the cue points more diverse; second, it ensures that each selected cue point maintains a certain spatial distance from the central cue point, which helps reduce redundancy. The kernel size of the expanded window is set to n, and the dilution rate is d, which determines the number of spaces inserted between cue points in the expanded window. The recommended dilution rate is 2 and the expanded window size is 5.

[0113] In addition, since the cue points generated by the above process do not limit the number of cue points in the same plot, a plot may have multiple cue points. Therefore, it is necessary to remove inaccurate cue points to ensure that each plot remains spatially independent and does not intersect with other plots. In the model, this problem is solved by the application of non-maximum suppression (NMS). Based on the input of the selected cue points, all plots predicted by the model are arranged in a candidate list in descending order of prediction scores. The plot with the highest score is selected from the candidate list. It will be added to the final list and deleted from the candidate list. Then, NMS calculates the "intersection over union" (IoU) between the plot with the highest score and each of the remaining plots in the candidate list (Formula 4). The IoU metric quantifies the degree of overlap between two plots, that is, an IoU close to 0 indicates little or no overlap. If the IoU value of a plot in the candidate list is greater than a threshold (the default value is 0.01), the overlapping plot will be deleted from the candidate list.

[0114]

[0115] Parameter description: f1 and f2 represent the two fields to be analyzed, and area is the overlapping area between the two fields calculated using the intersection and union operators.

[0116] S4. Extended application module: edge overlap module

[0117] Extension motivation: In actual field extraction applications, images from different dates may show different boundaries due to different planting and harvesting times of crops and changes in soil moisture. Therefore, the boundaries extracted from images of a single date may not accurately represent the actual field distribution in the area. The edge overlap module is designed to solve this problem.

[0118] Data input: Field distribution map obtained on each date in the same area

[0119] Data output: the smallest field distribution map obtained in this area during this time period

[0120] Image processing: Each segmentation mask corresponds to a field image at the corresponding position; the module obtains each field image in a single date image through the model, converts all of them into vector maps, and then performs an overlay operation to overlay all field vector maps extracted from each individual image. Then, the tiny fields generated by geometric registration errors in the generated vector map are removed to ensure that the smallest field in the time series is extracted. This ensures that not only the smallest PF in the growing season is extracted, but also that the extracted PF boundary is topologically closed.

[0121] In one embodiment, increasing the number of cue points can reduce the ambiguity of the cue, thereby segmenting more objects. The location of the cue points also plays a crucial role in the accuracy of the segmentation. If the cue points are placed accurately at or near the center of the target object, the model can correctly segment the target object. On the contrary, if the cue points are placed at the edge of the target object, the segmentation results produced by the model may be less accurate, such as segmenting a part of the target object or segmenting an object that is larger than the actual target object. The number and location of cue points have a significant impact on the performance, highlighting the importance of automatically generating an appropriate number and correct location of cue points. The cue point generation module is used for the PF extraction task of a single image, with the aim of automatically generating an appropriate number of cue points. The cue point optimization module is specifically used to reduce redundant points and errors caused by inaccurate cue point locations. The edge overlap module aims to extract PF from multi-temporal images by overlapping the edges of the PF extracted from each independent image.

[0122] In one embodiment, the spatial distribution of the extracted field plots is compared with the distribution of the actual samples by using the extraction model provided by the above method of the present invention. Figure 2As shown, it shows that the yield prediction model established by the present invention has a relatively reliable accuracy. As shown in Table 1 below, when the actual spatial distribution of the fields in the region is known, the extraction results of the fields predicted by the model are very close to the actual fields. Among them, the F1 score (F1-Score) of the model extraction result is 0.960, and the average cross entropy (mIoU) is 0.948.

[0123] Table 1 shows the yield data of known plots at the plot scale and the yield table predicted by the model.

[0124] Plots F1 score QUR Funan County 0.960 0.948

[0125] In order to test the performance under various complex geomorphic conditions, four typical agricultural areas (Songyuan City, Funan County, Ruian City and Chengdu City) were selected (see Table 2 and Figure 5 ). Songyuan City is located in the temperate monsoon climate zone with flat terrain, the singlest crop type (summer corn) and the largest field among the four study areas. In contrast, Chengdu City is located in the subtropical humid monsoon climate zone with complex hilly terrain and diverse crop types, and is the most dispersed farmland area among the four study areas. The characteristics of Funan City and Ruian City are between the above two study areas. We collected GF-2PMS images acquired between April 2018 and September 2022 (Table 2), covering an area of ​​approximately 30 km × 30 km in each study area.

[0126] Table 2 Study locations and data used

[0127]

[0128] We chose the state-of-the-art IAFRes model as the baseline method, which is a multi-task semantic segmentation architecture based on deep learning. First, we manually annotated the full coverage distribution data of the different study areas using GF-2 data as the ground truth data to verify the performance of IAFRes and STAMP models. At the same time, some image patches (512×512 pixels) in the surrounding areas of these study areas were manually annotated using GF-2 data for training the IAFRes model. The training data and validation data are spatially independent. STAMP was not trained and was directly used on the Gaofen-2 data of the study area. The RGB images synthesized from the near-infrared, green and red bands were used in the experiment. In the study of the model effect, the predictions of different methods were visually compared, and the quantitative evaluation was obtained by comparing the distribution of manually checked fields. The F1-score, mean cross entropy mIoU and average accuracy OA were calculated and compared with each other.

[0129] Figure 5The prediction results of the two field extraction methods in four different areas are shown. STAMP of the present invention shows better consistency with the reference image than IAFRes, and the performance of STAMP in the white box is also much better than IAFRes, and each plot is well segmented. This shows that STAMP outperforms the current best extraction model IAFRes. In addition, from Songyuan to Chengdu, as the complexity of the terrain increases, IAFRes produces more errors, and STAMP maintains a similar trend, but still maintains higher accuracy than IAFRes. The reason for the decline in the performance of IAFRes and STAMP may be the inherent limitations of complex landform processing, that is, as the complexity of the landform increases, the accurate extraction of deep features is usually more difficult. More importantly, compared with IAFRes, STAMP can segment more fields, and the boundaries of the extracted fields are more closed in topology. This is because STAMP pays more attention to the homogeneity of the fields and can distinguish various edges, while IAFRes pays more attention to hard edges.

[0130] The quantitative verification based on accuracy indicators is consistent with the intuitive comparison results, see Table 3. In Songyuan City and Funan County, where the terrain is not too complex, STAMP performs comparable to IAFRes. As the complexity of the region increases, the fields become more fragmented, and the accuracy of the fields extracted by STAMP is much better than that of IAFRes, but the performance of both methods decreases. It is worth noting that STAMP does not require any training samples, but achieves higher accuracy than IAFRes, with an average F1 score of 0.854, an overall accuracy (OA) of 0.886, and a mean intersection over union (mIoU) of 0.837, while the average F1 score of the IAFRes model is only 0.793, the overall accuracy (OA) is 0.818, and the mean intersection over union (mIoU) is 0.772. The accuracy indicators of the STAMP model have been improved by nearly 6%. All these results show that STAMP is highly suitable for field extraction tasks in complex areas of different landscapes.

[0131] Table 3 Quantitative evaluation of field extraction results generated by IAFRes and STAMP in four study areas

[0132]

[0133] Note: The difference (Diff) is calculated as the difference between the accuracy indicators of STAMP and IAFRes. The average is calculated as the average of each indicator between different regions. The bold value indicates the most accurate result.

[0134] also, Figure 6The results of the field extracted by the edge overlap module are shown, which combines the time segmentation results obtained by the STAMP model. This is achieved by the edge overlap module of STAMP. Obviously, STAMP retains the spatial details of the field at different times and different spatial locations. This is Figure 6 The areas within the boxes in (gi) are particularly clear, showing the contrast between the spatial extent of the initially extracted fields at different time points and the spatial extent of the fields extracted by the edge overlap module. The final fields extracted by the edge overlap module further subdivide the field distribution over time and determine the minimum field distribution that is spatially separated by crop type or farming method.

[0135] In addition, in one embodiment, the present invention also discloses a device for extracting farmland distribution information based on remote sensing images, which includes:

[0136] The first unit is used to obtain a remote sensing image containing a farmland plot to be measured;

[0137] A second unit is used to process the remote sensing image, wherein the remote sensing image is subjected to image cropping, denoising, and contrast enhancement, and the remote sensing image is processed through an image encoder to generate an embedded representation of an image that compresses and abstractly represents the remote sensing image;

[0138] The third unit is used for the prompt point generation module to generate candidate prompt points and prompt information based on the remote sensing image, and the point positioning optimization module optimizes the candidate prompt points and prompt information to obtain optimized prompt points and prompt information.

[0139] A fourth unit is used for encoding the cue information of the cue point through a cue encoder, and then performing joint encoding of the embedded representation and the cue point and its cue information;

[0140] The fifth unit is used to decode the joint code output by the hint encoder through the mask decoder to obtain the embedded representation of the image compressed and abstractly represented by the remote sensing image and the hint code of the hint point and its hint information, and then the mask decoder generates a segmentation mask representing the position and boundary of the farmland plot in the remote sensing image according to the embedded representation and hint information provided by the hint encoder;

[0141] The sixth unit is used to obtain all the field images in the single-date remote sensing image, convert them all into vector graphics, and then perform an overlay operation to overlay all the vector graphics extracted from each remote sensing image, and then remove the fields generated by geometric registration errors in the generated vector graphics to ensure that the smallest field in the time series is extracted.

[0142] In a preferred embodiment of the farmland distribution information extraction device based on remote sensing images, in the third unit, the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, calculates the gradient value to represent the edge position, and uses the Sobel filter to calculate the gradient map so that the prompt point is far away from the edge of the planted farmland PF. img =sobel(x), where x represents the input remote sensing image, Grad img Represents the extracted gradient map.

[0143] In a preferred embodiment of the farmland distribution information extraction device based on remote sensing images, the category homogeneity index HI is used to compare the category of each pixel with that of adjacent pixels to calculate the category homogeneity of each pixel to obtain a homogeneity map. Among them, I k It is the category value of each adjacent pixel in a moving window composed of m pixels. When the category value of the kth pixel is the same as that of the central pixel i, the category value is set to 1; otherwise, it is set to 0. The category homogeneity index HI value ranges from 0 to 1. The larger the category value, the more uniform the category value. By combining the category homogeneity index HI and the gradient, a candidate cue point is generated at a position where the HI value is large and the gradient value is small.

[0144] In a preferred embodiment of the farmland distribution information extraction device based on remote sensing images, the combination of the category homogeneity index HI and the gradient is realized by the planting index PI to obtain a planting index map.

[0145] PI i =ε·(1-G rad i )+(1-ε)·HI i ,

[0146] Among them, the ε variable represents the gradient weight, (1-ε) represents the HI weight, and the calculated value of each pixel is sorted from small to large. The pixels with values ​​greater than the threshold are regarded as candidate pixels of the prompt point.

[0147] In a preferred embodiment of the farmland distribution information extraction device based on remote sensing images, the point positioning optimization module optimizes the candidate prompt points, creates an expansion window around each candidate prompt point, selects all prompt points within the expansion window, extracts the cultivated farmland PF from all selected prompt points, and evaluates the extraction results based on the prediction score, selects the prompt point with the highest prediction score within the expansion window, retains the position of the prompt point and uses it as the optimal prompt point, and then removes other prompt points.

[0148] Those skilled in the art should understand that although the present invention is described in the form of multiple embodiments, not every embodiment includes only one independent technical solution. Such description in the specification is only for the sake of clarity. Those skilled in the art should understand the specification as a whole and regard the technical solutions involved in each embodiment as being able to be combined into different embodiments to understand the scope of protection of the present invention.

[0149] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.

Claims

1. A method for extracting farmland distribution information based on remote sensing images, characterized in that: It includes the following steps: Step 1, obtaining a remote sensing image containing the farmland to be measured; Step 2, processing the remote sensing image, wherein the remote sensing image is subjected to image cropping, denoising, and contrast enhancement, and the remote sensing image is processed by an image encoder to generate an embedded representation of an image that compresses and abstractly represents the remote sensing image; Step 3: the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, and the point positioning optimization module optimizes the candidate prompt points and their prompt information to obtain optimized prompt points and their prompt information. Step 4, the cue encoder encodes the cue information of the cue point, and then performs joint encoding of the embedded representation and the cue encoding of the cue point and its cue information; Step 5, the mask decoder first decodes the joint code output by the prompt encoder to obtain the embedded representation of the image compressed and abstractly represented by the remote sensing image and the prompt code of the prompt point and its prompt information, and then the mask decoder generates a segmentation mask representing the position and boundary of the farmland plot in the remote sensing image according to the embedded representation and the prompt information provided by the prompt encoder; Step 6, obtain all the field images in the single-date remote sensing image, convert them all into vector graphics, and then perform a superposition operation to superimpose all the vector graphics extracted from each remote sensing image, and then remove the field plots generated by geometric registration errors in the generated vector graphics to ensure that the smallest field plot in the time series is extracted, where In step 3, the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, calculates the gradient value to represent the edge position, and uses the Sobel filter to calculate the gradient map so that the prompt point is far away from the edge of the planted farmland PF. img =sobel(x), where x represents the input remote sensing image, Grad img represents the extracted gradient map, The category homogeneity index HI is used to compare the category of each pixel with that of the neighboring pixels to calculate the category homogeneity of each pixel to obtain a homogeneity map. Among them, I k It is the category value of each adjacent pixel in a moving window composed of m pixels. When the category value of the kth pixel is the same as that of the central pixel i, the category value is set to 1; otherwise, it is set to 0. The category homogeneity index HI value ranges from 0 to 1. The larger the category value, the more uniform the category value. By combining the category homogeneity index HI and the gradient, a candidate cue point is generated at a position where the HI value is large and the gradient value is small.

2. The method for extracting farmland distribution information based on remote sensing images according to claim 1, characterized in that: The combination of the class homogeneity index HI and the gradient is realized through the planting index PI to obtain the planting index map, PI i =ε·(1-Grad i )+(1-e)·HI i , Among them, the ε variable represents the gradient weight, (1-ε) represents the HI weight, and the calculated value of each pixel is sorted from small to large. The pixels with values ​​greater than the threshold are regarded as candidate pixels of the prompt point.

3. A method for extracting farmland distribution information based on remote sensing images according to claim 1 or 2, characterized in that: The point positioning optimization module optimizes the candidate prompt points, creates an expansion window around each candidate prompt point, selects all prompt points within the expansion window, extracts the cultivated field PF from all selected prompt points, and evaluates the extraction results based on the prediction scores, selects the prompt point with the highest prediction score within the expansion window, retains the position of the prompt point and uses it as the optimal prompt point, and then removes other prompt points.

4. A device for extracting farmland distribution information based on remote sensing images, characterized in that: It includes: The first unit is used to obtain a remote sensing image containing a farmland plot to be measured; A second unit is used to process the remote sensing image, wherein the remote sensing image is subjected to image cropping, denoising, and contrast enhancement, and the remote sensing image is processed through an image encoder to generate an embedded representation of an image that compresses and abstractly represents the remote sensing image; The third unit is used for the prompt point generation module to generate candidate prompt points and prompt information based on the remote sensing image, and the point positioning optimization module optimizes the candidate prompt points and prompt information to obtain optimized prompt points and prompt information. A fourth unit is used for encoding the cue information of the cue point through a cue encoder, and then performing joint encoding of the embedded representation and the cue point and its cue information; The fifth unit is used to decode the joint code output by the hint encoder through the mask decoder to obtain the embedded representation of the image compressed and abstractly represented by the remote sensing image and the hint code of the hint point and its hint information, and then the mask decoder generates a segmentation mask representing the position and boundary of the farmland plot in the remote sensing image according to the embedded representation and hint information provided by the hint encoder; The sixth unit is used to obtain all the field images in the single-date remote sensing image, convert them all into vector graphics, and then perform a superposition operation to superimpose all the vector graphics extracted from each remote sensing image, and then remove the field plots generated by geometric registration errors in the generated vector graphics to ensure that the smallest field plot in the time series is extracted. In the third unit, the prompt point generation module generates candidate prompt points and their prompt information based on the remote sensing image, calculates the gradient value to represent the edge position, and uses the Sobel filter to calculate the gradient map so that the prompt point is far away from the edge of the planted farmland PF. img =sobel(x), where x represents the input remote sensing image, Grad img represents the extracted gradient map, The category homogeneity index HI is used to compare the category of each pixel with that of the neighboring pixels to calculate the category homogeneity of each pixel to obtain a homogeneity map. Among them, I k It is the category value of each adjacent pixel in a moving window composed of m pixels. When the category value of the kth pixel is the same as that of the central pixel i, the category value is set to 1; otherwise, it is set to 0. The category homogeneity index HI value ranges from 0 to 1. The larger the category value, the more uniform the category value. By combining the category homogeneity index HI and the gradient, a candidate cue point is generated at a position where the HI value is large and the gradient value is small.

5. The device for extracting farmland distribution information based on remote sensing images according to claim 4, characterized in that: The combination of the class homogeneity index HI and the gradient is realized through the planting index PI to obtain the planting index map, PI i =ε·(1-Grad i )+(1-e)·HI i , Among them, the ε variable represents the gradient weight, (1-ε) represents the HI weight, and the calculated value of each pixel is sorted from small to large. The pixels with values ​​greater than the threshold are regarded as candidate pixels of the prompt point.

6. The device for extracting farmland distribution information based on remote sensing images according to claim 4 or 5, characterized in that: The point positioning optimization module optimizes the candidate prompt points, creates an expansion window around each candidate prompt point, selects all prompt points within the expansion window, extracts the cultivated field PF from all selected prompt points, and evaluates the extraction results based on the prediction scores, selects the prompt point with the highest prediction score within the expansion window, retains the position of the prompt point and uses it as the optimal prompt point, and then removes other prompt points.