Remote sensing image field extraction method based on transfer learning
By applying a transfer learning method in remote sensing image processing, combining UNet and DeepLabV3+ networks, and using the FADA framework for model migration, the boundary integrity and domain adaptability problems of field extraction in remote sensing images are solved, and a more accurate and general field information extraction effect is achieved.
Patent Information
- Application Number
- CN202510016182.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2025-04-04
AI Technical Summary
When the prior art automatically extracts field information from remote sensing images, it faces the problems of image noise affecting boundary integrity, feature extraction depends on expert knowledge, and poor migration of CNN models between different fields.
A transfer learning-based method is adopted, combined with UNet and DeepLabV3+ networks, field feature extraction is performed on remote sensing images, and the model is migrated to the target domain through the fine-grained adversarial learning framework FADA, optimized the field boundaries and performed post-processing to complete field extraction.
Model conversion between different regions and field boundary optimization is realized, which improves the accuracy and segmentation effect of field spatial distribution information, reduces the dependence on expert knowledge, and improves the universality and robustness of the algorithm.
Smart Images
Figure CN119762974A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of computer vision and agricultural technology, and in particular to a field extraction method of remote sensing images based on transfer learning. Background Art
[0002] Traditional methods for obtaining the spatial distribution of IAF mainly rely on field surveys, which are inefficient in applications over large areas. With the development of remote sensing technology, especially the method of using satellite images to generate detailed geometric information over large areas, it has become an effective means to study the spatial distribution of fields. The widespread application of remote sensing technology, especially high-resolution satellite images, is also promoting the development of a variety of remote sensing-based IAF analysis methods. Among them, visual interpretation, as a traditional method, is effective in small-scale applications, but is less practical at the national or regional level. Therefore, researchers have turned to methods for automatically extracting IAF from remote sensing images. These methods include the application of image segmentation technology, among which automatic IAF extraction is of particular concern. Depending on the feature extractor used, these methods can be divided into graph operator-based methods and deep learning-based methods.
[0003] Although graph operator-based methods are effective in some cases, image noise seriously affects the application of these methods, resulting in incomplete or fragmented boundaries. In addition, the features and parameters related to model performance often rely on the experience of specific users or study areas, which limits the generality and robustness of the algorithms
[217] . In this regard, deep learning-based methods provide a more attractive alternative because they can automatically extract features, thereby reducing the reliance on expert knowledge. These methods mainly use convolutional neural networks (CNNs) to extract features at different spatial scales to achieve semantic segmentation of IAFs. However, even deep learning methods face the challenge of improving the accuracy of IAF boundaries and ranges because there is a trade-off between the extraction of rich contextual information and accurate boundary localization.
[0004] Another challenge is the transferability of CNN models between different research domains, also known as the "domain adaptation problem". The key to this problem lies in the similarity of data distribution between the model training area (source domain) and the untrained area (target domain). The difference between regions will reduce the accuracy of the model in identifying objects in non-training areas. Summary of the invention
[0005] Purpose of the invention: The purpose of the present invention is to provide a field extraction method for remote sensing images based on transfer learning, using UNet and DeepNetV3+ networks as field detection to obtain remote sensing image information of the spatial distribution of fields within a larger spatial and temporal range.
[0006] Technical solution: The field extraction method of remote sensing images based on transfer learning described in the present invention comprises the following steps: (1) Use open source websites to obtain remote sensing images of the source domain and target domain and perform preprocessing; (2) Use DeepLabV3+ to extract field features from images in the source domain; (3) Based on the results obtained in step (2), the Unet model is used to extract field features from the source domain image again; (4) The source domain model trained in steps (2) and (3) is transferred to the target domain through the fine-grained adversarial learning framework FADA; (5) Optimize field boundaries and post-process fields to complete field extraction in the target domain.
[0007] Furthermore, in step (1), the remote sensing images of the source domain and the target domain are uniformly cropped into 256×256 images.
[0008] Furthermore, in step (2), the range of the field in the source domain is obtained through DeeplabV3+. Furthermore, in step (3), the boundary distribution of the field is predicted by the Unet model.
[0009] Furthermore, in step (4), the network trained in the source domain is trained together with the discriminator to form an adversarial network, which includes a feature extractor, a label predictor, and a domain discriminator. The feature extractor is used to extract deep features. The label predictor is used to determine whether the feature is easy to classify. The domain discriminator is used to determine whether the feature is distinguishable in different domains.
[0010] Furthermore, in step (5), the field boundary is optimized as follows: the field road boundary is morphologically processed: first, the point farthest from the boundary is extracted as the skeleton of the field boundary through distance calculation; second, the skeleton is expanded and corroded to connect the broken parts, and the burrs are removed through median filtering; the skeleton is extracted again, and the skeleton is expanded to merge the extracted skeleton with the previous field boundary.
[0011] Furthermore, in step (5), the post-processing of the farmland is as follows: isolated pixel clusters are removed by using the area and length screening threshold method; for the cutting streaks that still exist after cutting, the expansion and corrosion operations are performed separately on each farmland to fill the missing areas; and the final result of the farmland segmentation is obtained.
[0012] The field extraction system of remote sensing images based on transfer learning of the present invention comprises: Acquisition module: used to obtain remote sensing images of source and target domains using open source websites and perform preprocessing; DeepLabV3+ module: used to extract field features from images in the source domain using DeepLabV3+; Unet module: used to extract field features from source domain images again using the Unet model based on the results obtained from the DeepLabV3+ module; FADA module: used to migrate the source domain model trained by DeepLabV3+ module and Unet module to the target domain through the fine-grained adversarial learning framework FADA; Optimization module: used for field boundary optimization and field post-processing to complete field extraction in the target domain.
[0013] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the methods for extracting field plots from remote sensing images based on transfer learning is implemented.
[0014] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the methods for extracting field plots from remote sensing images based on transfer learning.
[0015] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the UNet and DeepNetV3+ networks widely used in image semantic segmentation are used as field detection problems. Combined with simple post-processing, the automatic generation of closed geometric figures is achieved. Moreover, through the fine-grained adversarial learning framework (FADA), model conversion is achieved between different regions without adding more data; independent field units can be accurately segmented from the only high-resolution remote sensing impact, serving more precision agriculture remote sensing applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a flow chart of the present invention; Figure 2 It is the fine-grained adversarial domain adaptation framework FADA of the present invention; Figure 3 It is the post-processing of connecting the field side roads of the present invention; Figure 4 It is the field segmentation result of the present invention; Figure 5 It is the field post-processing of the present invention; Figure 6 It is a display of the farmland segmentation results in the study area of the present invention; Figure 7 It is a display of the original image and segmentation results of farmland in part of the study area of the present invention. DETAILED DESCRIPTION
[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0018] like Figure 1 As shown, an embodiment of the present invention provides a field extraction method of a remote sensing image based on transfer learning, comprising the following steps: (1) Use open source websites to obtain remote sensing images of the source domain and target domain and perform preprocessing. The preprocessing is as follows: The remote sensing images of the source domain and target domain are uniformly cropped into 256×256 images.
[0019] (2) Use DeepLabV3+ to extract field features from images in the source domain; the details are as follows: Use DeeplabV3+ to obtain the range of fields in the source domain.
[0020] (3) Based on the result obtained in step (2), the Unet model is used to extract field features from the source domain image again; specifically, the Unet model is used to predict the boundary distribution of the field; in order to further improve the reasoning accuracy and provide seamless segmentation results, overlapping reasoning windows are used in the reasoning process to enhance the application of pixels to surrounding information to determine the classification results of different windows of the same object. The step size of each window is set to 128 pixels to obtain multiple reasonings for each pixel, and the average of all predicted values is used.
[0021] (4) If Figure 2 As shown, the source domain model trained in steps (2) and (3) is transferred to the target domain through the fine-grained adversarial learning framework FADA; the network trained in the source domain is trained together with the discriminator to form an adversarial network; including: feature extractor, label predictor and domain discriminator; the feature extractor is used to extract deep features; the label predictor is used to determine whether the feature is easy to classify; and the domain discriminator is used to determine whether the feature is distinguishable in different domains. The specific process is as follows: During the training process, the network trained in the source domain is trained together with the discriminator to form an adversarial network. The fine-grained domain adaptation problem is solved by alternately optimizing the feature extractor F, the classifier C and the discriminator D, including the following steps: (41) First, fix F and C and train D: fix the parameters of the trained feature extractor F and classifier C, combine the soft classification results output by the classifier in the target domain and the source domain, and only train the discriminator to achieve feature alignment between different domains and categories.
[0022] (42) Fix D and train F and C: The parameters of the discriminator D are fixed, and the labeled source domain data is brought into the model. The feature extractor F and the classifier C are trained to ensure the accuracy of the model in data classification and achieve the separability of features while arranging the features. At the same time, the data of the target domain is brought into the feature extractor to extract features and input into the discriminator D. F and C are trained with the identification loss of the target domain to ensure that the obtained features are invariant in the classes of different domains.
[0023] (43) Inference strategy in the target domain: During the inference process, the discriminator D can be removed, and only the weights of the feature extractor and the classifier need to be the same.
[0024] (5) Optimize the field boundary and post-process the field to complete the field extraction in the target domain. Figure 3 As shown in the figure, the optimization of the field boundary is as follows: the field road boundary is morphologically processed: first, the point farthest from the boundary is extracted through distance calculation as the skeleton of the field boundary; secondly, the skeleton is expanded and corroded to connect the broken parts, and the burrs are removed through median filtering; the skeleton is extracted again, and the skeleton is expanded to merge the extracted skeleton with the previous field boundary.
[0025] like Figure 4 As shown in the figure, the post-processing of the fields is as follows: isolated pixel clusters are removed by using the area and length screening threshold method; for the cutting marks that still exist after cutting, the expansion and corrosion operations are performed separately on each plot to fill the missing areas; and the final result of the farmland segmentation is obtained.
[0026] The model is used to predict fields in the source domain (Anhui) and the target domain (Zhejiang). The original image and segmentation results of Anhui Province are shown in the attached figure. Figure 5 As shown in the figure. On the validation set, the overall classification accuracy of the network for farmland in this area is 92%. The F1 score is 0.91. The overall segmentation results of farmland are shown in the figure. Figure 5 and Figure 6 As shown in FIG. 1 , the algorithm of the present invention basically identifies the farmland area and successfully segments it. Figure 7 As shown in the figure, from the comparison of labeled samples, it can be seen that the segmentation result of the plot does not directly separate the small plots very well. Moreover, the prediction result of the plot boundary can make up for the adhesion problem in the semantic segmentation of the plot to a certain extent, making the extraction of the plot instance more advantageous.
[0027] After pre-training, domain adversarial transfer training, and negative sample model fine-tuning training, the image and segmentation results of the final model inference in the target area are shown in the figure: Figure 7It can be seen that the segmentation model trained by the method of the present invention has a very accurate boundary between different plots, showing a strong segmentation ability; the segmentation coverage of the plot segmentation module is also very accurate, and by combining the extracted boundary and plot results, it also optimally achieves accurate semantic segmentation results, extracts different instance individuals, and largely overcomes the problem of individual adhesion. The details of edge processing are also close to perfection.
[0028] The entire network contains about 93M trainable parameters. On a GTX 3080 16G graphics card, it takes half an hour to complete the entire training process. The training process includes 675 256×256 images and a total of 150 epochs. It takes about 4 minutes to infer the entire study area (an area of about 27000×27000). The time consumption of post-processing in the same size processing area can also be controlled to be completed in about 45 seconds through parallel computing and algorithm optimization.
[0029] The embodiment of the present invention further provides a field extraction system of remote sensing images based on transfer learning, comprising: Acquisition module: used to obtain remote sensing images of source and target domains using open source websites and perform preprocessing; DeepLabV3+ module: used to extract field features from images in the source domain using DeepLabV3+; Unet module: used to extract field features from source domain images again using the Unet model based on the results obtained from the DeepLabV3+ module; FADA module: used to migrate the source domain model trained by DeepLabV3+ module and Unet module to the target domain through the fine-grained adversarial learning framework FADA; Optimization module: used for field boundary optimization and field post-processing to complete field extraction in the target domain.
[0030] An embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the method for extracting field plots from remote sensing images based on transfer learning as described in any one of the above is implemented.
[0031] The present invention discloses a storage medium in an embodiment of the present invention, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting field blocks from remote sensing images based on transfer learning is implemented.
Claims
1. A field extraction method for remote sensing images based on transfer learning, characterized in that: The following steps are involved: (1) Use open source websites to obtain remote sensing images of the source domain and target domain and perform preprocessing; (2) Use DeepLabV3+ to extract field features from images in the source domain; (3) Based on the results obtained in step (2), the Unet model is used to extract field features from the source domain image again; (4) The source domain model trained in steps (2) and (3) is transferred to the target domain through the fine-grained adversarial learning framework FADA; (5) Optimize field boundaries and post-process fields to complete field extraction in the target domain.
2. The method for extracting field blocks from remote sensing images based on transfer learning according to claim 1, characterized in that: In step (1), the remote sensing images of the source domain and the target domain are uniformly cropped into 256×256 images.
3. The field extraction method of remote sensing images based on transfer learning according to claim 1 is characterized in that: In step (2), the range of the field in the source domain is obtained through DeeplabV3+.
4. The method for extracting field blocks from remote sensing images based on transfer learning according to claim 1, characterized in that: In step (3), the boundary distribution of the field is predicted by the Unet model.
5. The method for extracting field blocks from remote sensing images based on transfer learning according to claim 1, characterized in that: In step (4), the network trained in the source domain is trained together with the discriminator to form an adversarial network, which includes a feature extractor, a label predictor, and a domain discriminator. The feature extractor is used to extract deep features. The label predictor is used to determine whether the feature is easy to classify. The domain discriminator is used to determine whether the feature is distinguishable in different domains.
6. The method for extracting field blocks from remote sensing images based on transfer learning according to claim 1, characterized in that: In step (5), the field boundary is optimized as follows: the field road boundary is morphologically processed: first, the point farthest from the boundary is extracted as the skeleton of the field boundary through distance calculation; second, the skeleton is expanded and corroded to connect the broken parts, and the burrs are removed through median filtering; the skeleton is extracted again, and the skeleton is expanded to merge the extracted skeleton with the previous field boundary.
7. The method for extracting field blocks from remote sensing images based on transfer learning according to claim 1, characterized in that: In step (5), the post-processing of the field is as follows: isolated pixel clusters are removed by using the area and length screening threshold method; for the cutting streaks that still exist after cutting, the expansion and corrosion operations are performed separately on each field to fill the missing areas; The final result of farmland segmentation is obtained.
8. A field extraction system for remote sensing images based on transfer learning, characterized in that: include: Acquisition module: used to obtain remote sensing images of source and target domains using open source websites and perform preprocessing; DeepLabV3+ module: used to extract field features from images in the source domain using DeepLabV3+; Unet module: used to extract field features from source domain images again using the Unet model based on the results obtained from the DeepLabV3+ module; FADA module: used to migrate the source domain model trained by DeepLabV3+ module and Unet module to the target domain through the fine-grained adversarial learning framework FADA; Optimization module: used for field boundary optimization and field post-processing to complete field extraction in the target domain.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, a method for extracting field blocks from remote sensing images based on transfer learning according to any one of claims 1 to 8 is implemented.
10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a method for extracting field blocks from remote sensing images based on transfer learning is implemented according to any one of claims 1 to 8.