Agricultural field shape local updating method and device based on semantic edge and segmentation

Abstract

A method and apparatus for local updating of agricultural plot morphology based on semantic edges and segmentation. The method includes: preparing the previous period's plot boundary vectors for a specified application area and the subsequent remote sensing image of the target area to be locally updated; selecting and designing a multi-task learning network model capable of simultaneously generating semantic segmentation and semantic edges; training the previous period's plot boundary vectors and the subsequent period's remote sensing image using the multi-task learning model to obtain the model weights; predicting the high-resolution remote sensing image data of the target application area to obtain the line and area results of agricultural farmland in the application area; removing small patches from the area prediction results; comparing the semantic segmentation results of the two periods; and selecting the updated plot elements based on a threshold. The updated plot elements are classified and processed according to the following cases: 1. No common edges: In the case where no common edges exist, the edge vector results generated by the subsequent remote sensing image through the multi-task learning model are directly used. 2. Existence of Common Edges: The area of ​​agricultural plots may change (increase or decrease) between two sets of remote sensing images. This invention compares the two images using an edge fusion algorithm, retaining the same edges in both images while updating the edges that have changed. This invention ultimately achieves local updates of plots in remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and remote sensing image information extraction, and involves semantic segmentation, semantic edge extraction, and a series of post-processing methods. Specifically, this invention is a method and apparatus for locally updating the morphology of agricultural plots in high-resolution remote sensing images using a combination of semantic edge extraction and semantic segmentation. Background Technology

[0002] Agricultural land is the material foundation upon which humans depend for survival. With the rapid development of remote sensing technology, its spatial information has found many important applications, such as crop condition monitoring and risk analysis in agricultural development planning, playing an indispensable role in improving agricultural productivity. Due to frequent human intervention, the morphology and boundaries of land parcels undergo local changes, especially in scenarios such as land subdivision and recultivation. This has led to a greater demand for methods for local updating of agricultural land parcels. For the extraction of updates from remotely sensed features, existing methods are mostly based on change detection, directly extracting the agricultural land parcel areas that have changed without retaining the boundary information in the original vectors. Furthermore, current models do not outperform manual annotation in edge extraction for complex agricultural land parcels and are affected by complex remote sensing backgrounds and noise. Therefore, this invention proposes to update the vector information of locally updated agricultural land parcels while retaining the vector information of agricultural land parcels that have not been updated.

[0003] Currently, mainstream algorithms for updating land parcel changes fall into two main categories: end-to-end direct detection methods and methods based on feature extraction and subsequent analysis. Methods based on feature extraction and subsequent analysis first extract key features from the data using feature extraction techniques, and then use subsequent analysis methods (such as classification, clustering, and statistical detection) to update changes. Examples include trend analysis methods, such as trend detection in time series data, or methods for detecting the statistical characteristics of changes. End-to-end change detection algorithms, on the other hand, are generally implemented using neural networks, directly detecting changes from the input data. They typically require no preprocessing or feature extraction; the model directly outputs the changed area or result from the raw input data. For example, HANet utilized a lightweight self-attention mechanism to design a discriminative conjoined network integrating multi-scale features for change detection. With the development of neural networks, the latest network models are also being applied to change updates in remote sensing, such as the transformer and graph convolutional neural networks (GCNs). These have achieved a high level of accuracy in extracting and classifying agricultural land parcels, and their image understanding capabilities are gradually improving. This makes further research into end-to-end local update methods based on neural networks worthwhile. Summary of the Invention

[0004] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a method and apparatus for local updating of high-resolution remote sensing land parcels based on semantic edges.

[0005] The purpose of this invention is to solve the problem of local updating of agricultural plots in remote sensing images. Considering the different advantages of semantic segmentation and semantic edge detection, this invention specifically uses a deep learning edge detection and segmentation joint task model to learn the features of agricultural plots in remote sensing images. Then, it predicts the agricultural plot elements in the area where changes in agricultural plots need to be detected, outputs an image containing all agricultural plot information in the next period, and performs preliminary post-processing on the segmentation results. Based on the segmentation results, it finds the plots that have been updated according to a threshold and performs local updates on the edges of the changed agricultural plots.

[0006] The present invention provides a method for local updating of agricultural plot morphology based on semantic edges and segmentation, comprising the following steps:

[0007] Step 1: Prepare the dataset. Select the image data of the target application area where change detection is ultimately needed, and select samples that contain a large number of densely agricultural plots.

[0008] Step 1.1: Acquire high-resolution remote sensing imagery. Select high-resolution remote sensing imagery with high spatial resolution that can clearly show the details of some fragmented agricultural plots, and whose corresponding image change area does not exceed 10%. Only samples that meet these requirements can be used for training with the vector labels from the previous period.

[0009] Step 1.2: Create remote sensing image samples. Crop the high-resolution remote sensing image to 1000*1000 pixels. Also, crop the vectors accordingly for use in subsequent training steps.

[0010] Step 1.3: Divide the sample set. Divide the cropped dense remote sensing plot samples into training and test sets according to the proportions.

[0011] Step 2: Select a multi-task learning model for semantic segmentation and semantic edge detection based on the characteristics of agricultural plots in high-resolution remote sensing images, and modify the network to make it more suitable for the task of extracting plots in high-resolution remote sensing images as required by this invention.

[0012] Step 2.1: Choosing a multi-task model instead of a semantic segmentation or semantic edge model is to ensure that the extraction results of semantic segmentation and semantic edge are consistent, and to reduce the impact of model performance on the experiment.

[0013] In semantic edge extraction, in order to address the imbalance between edge features and non-edge pixel distribution in images, an edge ratio parameter β can be introduced to mitigate the impact of this imbalance on network training.

[0014] L=-βlogPr(yi =1)-(1-β)∑logPr(y j =0)#(1)

[0015] Step 2.2: After determining the model, adjust the network depth and branches according to the requirements of the agricultural land extraction task. After the network architecture is determined, adjust the hyperparameters to improve the model's performance in the agricultural land extraction task.

[0016] Step 3: Use the improved semantic segmentation and semantic edge extraction network model designed in Step 2, which is suitable for agricultural plots, to train the high-resolution remote sensing image data prepared in Step 1 to obtain a multi-task learning network model. Then, use the test data classified in Step 1 to evaluate the model. Based on the evaluation results, decide whether to repeat Step 2 to fine-tune the network model structure and parameters to obtain the final optimal extraction model.

[0017] Step 4: Using the network model trained in Step 3, predict the high-resolution remote sensing image of the target application area prepared in Step 1 to obtain the edge results and segmentation results of densely populated agricultural plots in the application area. Then, perform patch removal on the segmentation results. Next, set a threshold value for the semantic segmentation overlap, and select agricultural plots that need local updates based on the threshold.

[0018] Step 4.1: Input the high-resolution remote sensing image prepared in Step 1 into the network model for prediction, and obtain the semantic segmentation result and semantic edge result at the same time.

[0019] Step 4.2: Based on the prediction results from Step 4.1, including the semantic segmentation results and the edge intensity maps generated by the semantic edges, a threshold is set for the segmentation results according to the number of points constituting the surface features. Then, patch removal is performed based on this threshold to reduce the impact of noise.

[0020] Step 4.3: This invention uses the Vector Difference method to calculate the geometric difference between two vector maps to identify the updated region. This method is simple to calculate and can achieve small errors even for complex remote sensing plots such as agricultural plots. The specific method is as follows:

[0021] 1. Calculate vector difference: The updated region is obtained by calculating the difference set between the two agricultural plots.

[0022] ChangeArea=(Seg1-Seg2)∪(Seg2-Seg1)

[0023] Seg1 and Seg2 are the two phases of agricultural land parcels, respectively.

[0024] 2. Set a threshold: Set a threshold by the ratio of the area of ​​the changed region to the total area. If the proportion of the updated area to the total area exceeds a certain threshold (e.g., 10%), the region is considered to have been updated.

[0025] Step 5: Based on the agricultural plots identified in Step 4 that have been updated, for agricultural plots that have not been updated, the boundaries of the plots marked manually in the previous stage are directly used; for agricultural plots that have been updated, the edge results of the later agricultural plots are used, and the local boundaries are adjusted using a boundary fusion algorithm.

[0026] Step 5.1: Differentiate between two cases of local updates: 1. No common edges: This case involves an increase or decrease in the area of ​​independent plots. In this case, the edge vector results generated by the multi-task learning model from the later remote sensing images are directly used. 2. Common edges exist: The area of ​​agricultural plots may change (increase or decrease) in the two remote sensing images. This invention compares the two images based on the edge fusion algorithm, retaining the same edges in both images, while updating the edges that have changed.

[0027] Step 5.2: Boundary Fusion Algorithm. Traverse each line segment on the parcel vector feature and store these points using an N x 2 (x, y) tensor. By setting a dynamic threshold around the corresponding point in the edge intensity map based on the surrounding intensity values, the common point where the intensity value and coordinates best match is selected, thus identifying the line segments in the feature that have not been updated. For updated portions, the start and end points are reused from the previously updated vector portion, while the updated portion uses the vectorized results of the line segments in the later edge intensity map. Morphological operations such as dilation and erosion are then used to further smooth the edges, reducing breaks and discontinuities, improving edge continuity, and finally obtaining the edge vector map of the new remote sensing image.

[0028] A second aspect of the present invention relates to an apparatus for local updating of agricultural plot morphology based on semantic edges and segmentation, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the method for local updating of agricultural plot morphology based on semantic edges and segmentation of the present invention.

[0029] A third aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the method for local updating of agricultural plot morphology based on semantic edges and segmentation of the present invention.

[0030] This invention proposes a method for detecting agricultural plot edges based on semantic segmentation and semantic edges. This method can quickly locate changing plots through semantic segmentation and finally achieve accurate local updates of agricultural plots through semantic edge detection.

[0031] Compared to previous update extraction methods, this invention offers the advantage of high accuracy. In high-resolution remote sensing imagery, agricultural plots differ significantly from other plots. These plots typically have dense boundaries, are closely adjacent, have high boundary overlap, and often cluster together. Local update methods based on semantic segmentation, because they extract segmentation results, cannot accurately determine the local updates of agricultural plots within the vectorized post-processed image. Therefore, this invention ultimately achieves local updates through edge detection. Another advantage of this invention is efficiency. Previous change detection methods show that the proportion of changed agricultural plots in the entire image is generally much smaller than that of unchanged areas. Directly extracting from two high-resolution remote sensing images wastes computational resources. Therefore, this invention first processes the two remote sensing images using a semantic segmentation algorithm to extract agricultural plot elements. Then, by comparing the extracted elements, it identifies the updated areas and further performs semantic edge detection on these areas to accurately extract the boundaries of the changed regions.

[0032] This invention, based on deep learning-based semantic segmentation and semantic edge methods, reuses existing data and network models to further improve the accuracy and efficiency of local updates of agricultural plots.

[0033] Due to the adoption of the above-mentioned technical methods, the present invention has the following advantages and beneficial effects:

[0034] 1. By combining semantic segmentation and semantic edge extraction tasks, this invention can accurately extract locally updated land parcels in complex high-resolution remote sensing environments, solving problems particularly acutely when there are complex boundaries between agricultural land parcels and other land features, complex boundaries within agricultural land parcels, and high overlap rates between multiple land parcels. Furthermore, it reduces errors caused by segmentation results, and by applying semantic edge detection to the segmentation results, it can accurately delineate the boundaries of changed areas.

[0035] 2. This invention first performs preliminary semantic segmentation and semantic edge extraction on agricultural plots, compares the segmentation results, calculates the differences in vector maps, and sets a threshold to filter update areas, updating edges only for the plots to be updated. Compared to traditional methods that perform edge detection on the entire image, this reduces the waste of computational resources and mitigates problems such as edge blurring and false extraction caused by current model extraction methods, maximizing the advantage of high accuracy in manual annotation.

[0036] 3. By using morphological operations, the edges of the unupdated and updated regions are merged, ensuring precise boundaries for the updated regions and smooth boundaries for the unupdated regions. This method improves the handling of boundary details while avoiding the discontinuities or inconsistencies that may occur when processing regions individually. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention;

[0038] Figure 2 This is an example of the reduction in plot elements when agricultural plots do not have common edges.

[0039] Figure 3 This is an example of an increase in plot features when agricultural plots do not have common edges.

[0040] Figure 4 This is an example of changes in plot features when agricultural plots have shared edges. Detailed Implementation

[0041] To better understand the specific content of the present invention, the present invention will be described in more detail below with reference to the accompanying drawings and embodiments.

[0042] (1) The main tasks of the data preparation stage include preparing high-resolution remote sensing image data with a resolution of 0.5m and selecting data images of the application area for training.

[0043] (1.1) Select the final application area of ​​the method and download the high-resolution remote sensing image data of the application area.

[0044] (1.2) Based on the selected high-resolution remote sensing data images, select areas with relatively dense agricultural plots, and the image change area should not exceed 10%. The training data for the later period uses the image of the next period and the label data of the previous period.

[0045] (1.3) Divide the training set and the test set. Divide the labeled remote sensing images and divide the samples according to the proportion.

[0046] (2) Based on the characteristics of agricultural plots in high-resolution remote sensing images, a multi-task learning model is selected. In this invention, the MFFE model is selected and the network is modified to suit the task of extracting plots in high-resolution remote sensing images required by this invention.

[0047] (2.1) Select a multi-task learning model for semantic segmentation and semantic edge detection to generate segmentation and edge results simultaneously. In semantic edge extraction, to address the imbalance between edge elements and non-edge pixels in the image, an edge ratio parameter β can be introduced to mitigate the impact of this imbalance on network training.

[0048] L=-βlogPr(y i =1)-(1-β)∑logPr(y j =0)#(1)

[0049] (2.2) After determining the model, adjust the network depth and branches according to the requirements of the agricultural land extraction task. After the network architecture is determined, adjust the hyperparameters to improve the model's performance in the agricultural land extraction task.

[0050] (3) The improved multi-task learning model for semantic segmentation and semantic edge extraction of agricultural plots, designed in step 2, is used to train the high-resolution remote sensing image training data prepared in step 1 to obtain the network weights of the multi-task learning model for agricultural plots. The number of iterations of the model can be further set according to the number of fitting iterations during testing. The batch_size is set according to the size of the dataset, as well as the network performance and computer performance. It should not be too small or too large.

[0051] (4) Use the weights of the network model trained in step 3 to predict the high-resolution remote sensing image data prepared in step 1, obtain the segmentation surface results and edge line results of agricultural plots, perform patch removal operation on the segmentation results, and filter out agricultural plots that need to be locally updated according to the threshold.

[0052] (4.1) Transform the high-resolution remote sensing image from step 1 into a data format that can be input into the multi-task learning model.

[0053] (4.2) Input the large map data of the application area into the multi-task learning model to obtain the surface prediction results and the line prediction results respectively. The surface results generated by the semantic segmentation branch are grayscale and cannot be directly converted into vectors. In addition, the surface results contain many small patches, which will also cause many small fragments in the subsequent vectorization results, affecting the accuracy.

[0054] The formula for the small patch removal algorithm is as follows: block(i) is the i-th block region of the surface result, PIXEL_VALUE represents the gray value in the block region, and THRESHOLD represents the threshold for the area of ​​the small patch to be removed. When the area is greater than or equal to the threshold, the pixel value of the block region remains unchanged at 255. When the area of ​​the block region is less than the threshold, the pixel value of the block region becomes 0, the same as the background value, and the small patch is removed.

[0055] (4.3) The Vector Difference method is used on the vector map to calculate the geometric difference between two vector maps to identify the updated area. This method is simple to calculate and can achieve small errors for relatively complex remote sensing plots such as agricultural plots. The specific method is as follows:

[0056] 1. Calculate vector differences: The updated region is obtained by calculating the difference set of the vector regions of agricultural plots in two periods.

[0057] ChangeArea=(Seg1-Seg2)∪(Seg2-Seg1)

[0058] Seg1 and Seg2 are the vector map areas of the two phases of agricultural land parcels, respectively.

[0059] 2. Set a threshold: Set a threshold by the ratio of the area of ​​the changed region to the total area. If the proportion of the updated area to the total area exceeds a certain threshold (e.g., 10%), the region is considered to have been updated.

[0060] (5) Based on step 4, the updated agricultural plots are determined. For agricultural plots that have not been updated, the semantic edges detected and vectorized in the early stage are directly used. For agricultural plots that have been updated, the edge results of the later agricultural plots are used, and the edge fusion algorithm is used to fuse the edges of the two stages.

[0061] (5.1) Distinguishing between two cases of local updates: 1. No common edges: In the case of no common edges, the result of independent plot segmentation is an increase or decrease. In this case, the edge vector results generated by the multi-task learning model of the later remote sensing are directly used. 2. Common edges exist: In the two remote sensing images, the area of ​​agricultural plots may change (increase or decrease). This invention compares the two images based on the edge fusion algorithm, retains the same edges in the two images, and updates the edges that have changed.

[0062] (5.2) Boundary Fusion Algorithm. Each line segment on the parcel vector feature is traversed, and these points are stored using an N x 2 (x, y) tensor. By setting a dynamic threshold around the corresponding point in the edge intensity map based on the surrounding intensity values, the common point where the intensity value and coordinates best match is selected, thus identifying the line segments in the feature that have not been updated. For updated portions, the start and end points are reused from the previously updated vector portion, while the updated portion uses the vectorized results of the line segments in the later edge intensity map. Morphological operations such as dilation and erosion are then used to further smooth the edges, reducing breaks and discontinuities, improving edge continuity, and finally obtaining the edge vector map of the new remote sensing image.

[0063] Practical experience has proven that the agricultural land parcel morphology local update method based on semantic edges and semantic segmentation, as proposed in this invention, can effectively solve the accuracy and efficiency problems in existing remote sensing feature extraction methods. By combining a multi-task learning model of deep learning, semantic segmentation is used to quickly locate the changed areas of agricultural land parcels, and semantic edges are used to further accurately extract the boundaries of the changed areas. This invention can provide more accurate agricultural land parcel change detection results in high-resolution remote sensing imagery. In summary, this invention provides an effective solution for high-precision and high-efficiency remote sensing feature extraction, with broad application prospects, especially in fields such as agricultural monitoring and land use change detection.

Claims

1. A method for local updating of agricultural plot morphology based on semantic edges and segmentation, comprising the following steps: Step 1: Prepare the dataset; Select the target application area image data for which change detection is needed, and choose areas containing a large amount of densely cultivated agricultural land; Step 2: Select a multi-task learning model based on the characteristics of agricultural farmland in high-resolution remote sensing images. MFFE is selected, and the network is modified to suit the task of extracting farmland from high-resolution remote sensing images. Step 3: Use the improved semantic segmentation and semantic edge extraction network model designed in Step 2 to train the high-resolution remote sensing image data prepared in Step 1 to obtain the multi-task learning network model. Then, use the test data classified in Step 1 to evaluate the model. Based on the evaluation results, decide whether to repeat Step 2 to fine-tune the network model structure and parameters to obtain the final optimal extraction model. Step 4: Use the network model trained in Step 3 to predict the high-resolution remote sensing image of the target application area prepared in Step 1, and obtain the edge results and segmentation results of dense agricultural plots in the application area; and perform patch removal operation on the segmentation results; then set a certain threshold for the semantic segmentation overlap, and select agricultural plots that need to be locally updated according to the threshold. Step 5: Based on the agricultural plots identified in Step 4 that have been updated, for agricultural plots that have not been updated, the boundaries of the plots marked manually in the previous stage are directly used; for agricultural plots that have been updated, the edge results of the later agricultural plots are used, and the local boundaries are adjusted using a boundary fusion algorithm.

2. The method for local updating of agricultural plot morphology based on semantic edges and segmentation according to claim 1, characterized in that: Step 1 includes: Step 1.1: Acquire high-resolution remote sensing images; Select high-resolution remote sensing images with high spatial resolution that can clearly show the details of some fragmented agricultural plots, and whose corresponding image change area does not exceed 10%. Only samples that meet the requirements can be used for training with the vector labels from the previous period. Step 1.2: Create remote sensing image samples; crop the high-resolution remote sensing image to 1000*1000; and crop the vector accordingly for training in subsequent steps; Step 1.3: Divide the sample set; divide the cropped dense remote sensing plot samples into training set and test set according to the proportion.

3. The method for local updating of agricultural plot morphology based on semantic edges and segmentation according to claim 1, characterized in that, Step 2 includes: Step 2.1: Choosing a multi-task model instead of a semantic segmentation or semantic edge model is to ensure that the extraction results of semantic segmentation and semantic edge are consistent, and to reduce the impact of model performance on the experiment; In semantic edge extraction, in order to address the imbalance between edge elements and non-edge pixels in an image, an edge ratio parameter β can be introduced to mitigate the impact of this imbalance on network training. L=-βlogPr(y i =1)-(1-β)∑logPr(y j =0)#(1) Step 2.2: After determining the model, adjust the depth and branches of the network according to the requirements of the agricultural land extraction task; after the network architecture is determined, adjust the hyperparameters to improve the model's performance in the agricultural land extraction task.

4. The method for local updating of agricultural plot morphology based on semantic edges and segmentation according to claim 1, characterized in that, Step 3 includes: The improved semantic segmentation and semantic edge extraction network model for agricultural plots, designed in step 2, is used to train the high-resolution remote sensing image data prepared in step 1 to obtain a multi-task learning network model. Then, the model is evaluated using the test data classified in step 1. Based on the evaluation results, it is decided whether to repeat step 2 to fine-tune the network model structure and parameters to obtain the final optimal extraction model.

5. The method for local updating of agricultural plot morphology based on semantic edges and segmentation according to claim 1, characterized in that, Step 4 includes: Step 4.1: Input the high-resolution remote sensing image prepared in Step 1 into the network model for prediction, and obtain the semantic segmentation result and semantic edge result at the same time; Step 4.2: For the prediction results in Step 4.1, including the semantic segmentation results and the edge intensity map generated by the semantic edges; for the segmentation results, set a certain threshold according to the number of points of the constituent surface elements, and perform patch removal processing according to the threshold to reduce the impact of noise; Step 4.3: Use the VectorDifference method to calculate the geometric differences between the two vector maps to update the region identification; specifically including:

1. Calculate vector differences: Obtain the updated region by calculating the difference between the agricultural plots in the two periods; ChangeArea=(Seg1-Seg2)∪(Seg2-Seg1) Seg1 and Seg2 are the dividing areas of the two phases of agricultural land parcels, respectively.

2. Set a threshold: Set a threshold by the ratio of the area of ​​the changed region to the total area; if the ratio of the updated area to the total area exceeds a certain threshold, the region is considered to have been updated.

6. The method for local updating of agricultural plot morphology based on semantic edges and segmentation according to claim 1, characterized in that, Step 5 includes: Step 5.1: Distinguish between two cases of local updates:

1. No common edges: In the case of no common edges, the result of the segmentation of independent plots is either increased or decreased; in this case, the edge vector result generated by the multi-task learning model of the later remote sensing is directly used; 2. Common edges exist: In the two remote sensing images, the area of ​​agricultural plots may change. According to the edge fusion algorithm, the two images are compared, the same edges in the two images are retained, and the changed edges are updated. Step 5.2: Boundary fusion algorithm; Traverse each line segment on the parcel vector feature and store these points using an N x 2 (x, y) tensor; By setting a dynamic threshold around the corresponding point in the edge intensity map based on the surrounding intensity values, select the common point where the intensity value and coordinates best match, thereby determining the line segments in the feature that have not been updated; For the updated parts, the start and end points are retained from the previously updated vector parts, and the updated parts use the results of the line segment vectorization in the later edge intensity map; Then, morphological operations such as dilation and erosion are used to further smooth the edges, reduce breakage and discontinuity, improve the continuity of the edges, and finally obtain the edge vector map of the new remote sensing image.

7. A device for local updating the morphology of agricultural plots based on semantic edges and segmentation, characterized in that, The system includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the method for local updating of agricultural plot morphology based on semantic edges and segmentation as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the method for local updating of agricultural plot morphology based on semantic edges and segmentation as described in any one of claims 1-6.

Images