A field plot extraction method, device and equipment based on multi-task deep learning

Abstract

This invention discloses a method, apparatus, and device for field extraction based on multi-task deep learning. The method includes: acquiring remote sensing time-series images of the phenological stages of agricultural crops within a region to be identified; inputting the remote sensing time-series images of the phenological stages of agricultural crops within the region to be identified into a trained farmland field extraction framework for prediction, thereby obtaining farmland field identification results; wherein, the farmland field extraction framework is trained based on a remote sensing time-series image training set, which is a dataset within the study area after data preprocessing. This invention solves the problems of poor accuracy and low efficiency in field identification.

Description

Technical Field

[0001] This invention relates to the field of agricultural remote sensing application technology, and in particular to a method, apparatus and equipment for field extraction based on multi-task deep learning. Background Technology

[0002] The spatial distribution and location information of farmland plots are closely related to grain yield prediction and related agricultural development, and are an important guarantee for food security. Therefore, accurate identification of farmland plots is of great significance for safeguarding people's livelihoods and national development. Earth observation technologies, represented by remote sensing satellites, have greatly reduced the reliance of practitioners on on-site inspections of farmland plots and can quickly identify relevant information within plots at a large scale. Therefore, using remote sensing data to extract farmland plots at a large scale, to a certain extent, meets the needs of practitioners and institutions for obtaining spatial information and distribution of farmland plots.

[0003] Currently, methods for identifying farmland plots using remote sensing imagery generally fall into two categories: pixel-based methods and object-oriented methods. Pixel-based methods classify different land features based on the differences within image pixels, enabling rapid extraction of farmland information at large scales. Object-oriented methods, on the other hand, merge neighboring pixels with similar structures into a single "object," extracting spectral and texture information from a spatial perspective and combining this spectral information with plot identification. This significantly reduces the "salt-and-pepper noise" generated in pixel-based methods, improving the accuracy of farmland plot identification.

[0004] However, existing research on farmland plot identification using remote sensing technology still suffers from low accuracy and weak generalization ability. Summary of the Invention

[0005] This invention provides a method, apparatus, and device for field extraction based on multi-task deep learning, which solves the problems of poor accuracy and low efficiency in field identification.

[0006] A field extraction method based on multi-task deep learning includes:

[0007] Acquire remote sensing time-series images of the phenological periods of agricultural crops within the area to be identified;

[0008] The remote sensing time-series images of agricultural crop phenological periods in the area to be identified are input into the trained farmland plot extraction framework for prediction, and the farmland plot identification results are obtained.

[0009] The farmland plot extraction framework is trained based on a remote sensing time-series image training set, which is a dataset within the study area after data preprocessing.

[0010] In one embodiment of the present invention, training a farmland plot extraction framework based on a remote sensing time-series image training set specifically includes: acquiring remote sensing time-series images of the study area; preprocessing the remote sensing time-series images to obtain sample data; acquiring farmland plot vector data; determining a plot semantic dataset and a plot edge dataset based on the farmland plot vector data and the sample data; constructing a plot spatiotemporal spectral feature extraction branch structure; training the plot spatiotemporal spectral feature extraction branch structure based on the plot semantic dataset; constructing a plot edge feature extraction branch structure; training the plot edge feature extraction branch structure based on the plot edge dataset; and constructing a plot feature fusion module, inputting the data output from the plot spatiotemporal spectral feature extraction branch structure and the plot edge feature extraction branch structure into the plot feature fusion module for plot recognition training to obtain a trained farmland plot extraction framework.

[0011] In one embodiment of the present invention, the step of obtaining farmland plot vector data and determining plot semantic dataset and plot edge dataset based on the farmland plot vector data and the sample data specifically includes: preprocessing the farmland plot vector data to obtain plot semantic data labels; extracting the edge features of the plot semantic data labels as plot edge detection labels; and pairing sample data, plot semantic data labels, and plot edge detection labels from the same geographical location to obtain plot semantic dataset and plot edge dataset.

[0012] In one embodiment of the present invention, the construction of the field spatiotemporal spectral feature extraction branch structure, and the training of the field spatiotemporal spectral feature extraction branch structure based on the field semantic dataset, specifically includes: performing a convolution operation on the field semantic dataset based on a three-dimensional convolution operator to obtain a first field feature; compressing the spatial dimension of the first field feature through a global average pooling operation to obtain a second field feature; encoding the second field feature based on a two-dimensional convolution operator to obtain field time-spectral information; processing the field time-spectral information through a one-dimensional convolution operator to obtain field spatial-spectral correlation features; then decoding the field spatial-spectral correlation features based on a two-dimensional convolution operator; and multiplying the value obtained after processing the decoding result with the first field feature in the spatial dimension by the value obtained after processing with the Sigmoid activation function to obtain the time-spectral information. The first field feature is compressed using average pooling and max pooling operators to obtain the third field feature. A three-dimensional convolution operation is performed on the third field feature to obtain the field's temporal-spatial features. The value obtained after processing the field's temporal-spatial features with the Sigmoid activation function is multiplied by the first field feature in the spectral dimension to obtain the features in the temporal-spatial dimension. The features in the temporal-spectral dimension are added to the features in the temporal-spatial dimension, and features are extracted from the added result based on the three-dimensional convolution operation to obtain the field features in the spatial-spectral-temporal dimension, thus completing the construction of the space-spectral-temporal (SST) module. The branch structure for extracting the field's spatiotemporal spectral features is determined based on the SST module.

[0013] In one embodiment of the present invention, the construction of the field edge feature extraction branch structure, and the training of the field edge feature extraction branch structure based on the field edge dataset, specifically includes: obtaining shallow field features from the field spatiotemporal spectral feature extraction branch structure; performing position encoding on the temporal information of remote sensing time-series images, and combining a Light-temporal-attention-encoder (LTAE) module to compress the temporal dimension of the shallow field features, thereby enhancing the field spatial-spectral information at important time nodes; and using a first difference convolution operator, a second difference convolution operator, and a third difference convolution operator... The sub-processes the spatial-spectral information of the fields to obtain different enhanced field feature edge information; based on the superposition strategy and two-dimensional convolution operator, the different enhanced field feature edge information are fused to obtain field edge features; the field edge features are subjected to two-dimensional convolution operation according to the field feature edge information to obtain a field edge feature probability map with contextual semantic relationship; the ReLU activation function is applied to the field edge feature probability map to generate a field edge distribution map, thereby completing the construction of the boundary extraction (BE) module; the field edge feature extraction branch structure is determined according to the BE module.

[0014] In one embodiment of the present invention, the step of inputting the data output from the field spatiotemporal spectral feature extraction branch structure and the field edge feature extraction branch structure into the field feature fusion module for field recognition training to obtain a trained farmland field extraction framework specifically includes: encoding the field features output by the SST module through the LTA module; supplementing the spatial-spectral information of the field features based on a two-dimensional deconvolution operator; subtracting the field features with the supplemented spatial-spectral information from the field edge features to obtain the field semantic features; performing multi-scale fusion of the field semantic features through dilated spatial convolution pooling pyramid operations; adding the field edge features and the multi-scale fused field semantic features; and performing a two-dimensional convolution operation on the added result to obtain a field semantic segmentation map, thereby completing the construction of the field feature fusion module and further completing the training of the farmland field extraction framework.

[0015] In one embodiment of the present invention, the remote sensing time-series image is preprocessed to obtain sample data, specifically including: the remote sensing time-series image is a Sentinel-2 image; the Sentinel-2 image is corrected, the correction including radiometric correction, orthorectification, atmospheric correction and projection transformation; the corrected Sentinel-2 image is cropped, discarding part of the image data; the cropped data is superimposed with images at the same location according to the time dimension as sample data, and the acquisition time of all images in the sample data is recorded in chronological order.

[0016] In one embodiment of the present invention, the step of determining the field spatiotemporal spectral feature extraction branch structure according to the SST module specifically includes: using the SST module in series 7 times through a series strategy to complete the construction of the field spatiotemporal spectral feature extraction branch structure; wherein, the first 4 SST modules include two output branches and the output results are shallow field features, the output results of the last 3 SST modules are deep field features, and the first 2 SST modules use 32 three-dimensional average pooling layers with a kernel size of 3×3×3.

[0017] A field extraction device based on multi-task deep learning includes:

[0018] The data acquisition module is used to acquire remote sensing time-series images of the phenological periods of agricultural crops within the area to be identified;

[0019] The farmland plot identification module is used to input remote sensing time-series images of agricultural crop phenological periods in the area to be identified into a trained farmland plot extraction framework for prediction, and obtain farmland plot identification results.

[0020] The farmland plot extraction framework is trained based on a remote sensing time-series image training set, which is a dataset within the study area after data preprocessing.

[0021] A field extraction device based on multi-task deep learning includes:

[0022] At least one processor; and,

[0023] The memory is connected to the at least one processor via a bus; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor to implement the method as described in any of the above embodiments.

[0025] This invention provides a method, apparatus, and device for field extraction based on multi-task deep learning, which has at least the following beneficial effects: This invention deeply mines the temporal-spatial-spectral information in remote sensing time-series image data to construct a remote sensing image time-series dataset for farmland field extraction, and fully utilizes the advantages of deep neural networks to simultaneously learn the semantic and edge information of farmland fields to construct geometric topological relationships, thereby achieving high-precision extraction of farmland fields; at the same time, the entire process does not require manual intervention and is automatically processed, which helps to achieve automatic classification of crops. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0027] Figure 1 A schematic diagram of a farmland plot extraction framework based on multi-task deep learning provided in an embodiment of the present invention;

[0028] Figure 2 A schematic diagram illustrating the training process of a farmland plot extraction framework based on multi-task deep learning, provided in an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram illustrating the steps of a field extraction method based on multi-task deep learning, provided in an embodiment of the present invention.

[0030] Figure 4 This is an example of the identification effect of farmland plots within the study area provided by an embodiment of the present invention;

[0031] Figure 5 A schematic diagram of a field extraction device based on multi-task deep learning provided in an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of a field extraction device based on multi-task deep learning, provided as an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described clearly and completely below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0034] It should be noted that those skilled in the art will understand, explicitly and implicitly, that the embodiments described in this invention can be combined with other embodiments without conflict. Unless otherwise defined, the technical or scientific terms used in this invention should be understood in their ordinary sense by those skilled in the art. The terms "a," "an," "an," "the," etc., used in this invention do not indicate quantity limitation and can represent singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; the terms "first," "second," "third," etc., used in this invention are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0035] Current research on farmland plot identification using remote sensing technology suffers from low accuracy and weak generalization ability. This is mainly due to the following factors:

[0036] (1) Insufficient research on the temporal and spatial dimensions of farmland. Seasonality is the most prominent characteristic of crops in farmland. The obvious difference between farmland and other land features is that the crops planted in farmland have obvious and unique spectral characteristics that change over time. Therefore, remote sensing time-series images are of great significance for the extraction of farmland plots;

[0037] (2) There is a lack of research on the impact of field edges on field extraction. Unlike other land use categories such as crop identification, the obvious characteristic of farmland is that there are obvious boundaries between fields composed of field ridges. This geometric and topological information of farmland is an essential requirement for refined management such as land ownership confirmation and cadastral management, and it is also an important reason affecting the accuracy of farmland field extraction.

[0038] (3) Most field identification methods rely on shallow features. Current field identification methods mostly rely on manual feature extraction. These shallow spatial spectral features not only increase the risk of error propagation and make it difficult to describe the inherent attributes of farmland, but also increase computational consumption, which to some extent affects the accuracy and efficiency of farmland identification.

[0039] In view of this, there is an urgent need to provide a solution that can accurately extract farmland plots, so as to provide a scientific basis for decision-making by agricultural-related departments.

[0040] This invention proposes a method, apparatus, and device for farmland plot extraction based on multi-task deep learning. Addressing the shortcomings of existing farmland plot extraction methods, this invention aims to provide a method and system for farmland plot extraction based on remote sensing spatiotemporal spectral features using multi-task deep learning. This method fully mines the temporal-spatial-spectral (spatial-spectral) information in remote sensing image time-series data, establishes a connection with farmland plot attributes, and then mines farmland plot edge features, using these as geometric and topological information constraints for farmland plot identification. Furthermore, through end-to-end neural network learning, the risk of human error is avoided, and deep features are mined, enabling accurate and rapid extraction of the spatial distribution of farmland plots, which is beneficial for agricultural management and planning. A detailed explanation follows.

[0041] Before using a trained farmland plot extraction framework to predict data from acquired remote sensing time-series images, a farmland plot extraction framework model needs to be constructed. The acquired and preprocessed remote sensing time-series images are used as the training set to train the model, thereby improving the accuracy and efficiency of farmland plot extraction. The farmland plot extraction framework model and its training process are as follows: Figure 1 As shown.

[0042] In one embodiment of the present invention, training a farmland plot extraction framework based on a remote sensing time-series image training set specifically includes: acquiring remote sensing time-series images of the study area; preprocessing the remote sensing time-series images to obtain sample data; acquiring farmland plot vector data; determining a plot semantic dataset and a plot edge dataset based on the farmland plot vector data and sample data; constructing a plot spatiotemporal spectral feature extraction branch structure; training the plot spatiotemporal spectral feature extraction branch structure based on the plot semantic dataset; constructing a plot edge feature extraction branch structure; training the plot edge feature extraction branch structure based on the plot edge dataset; and constructing a plot feature fusion module, inputting the data output from the plot spatiotemporal spectral feature extraction branch structure and the plot edge feature extraction branch structure into the plot feature fusion module for plot recognition training, thereby obtaining a trained farmland plot extraction framework.

[0043] Specifically, such as Figure 2 As shown, the training method for constructing a farmland plot extraction framework provided by the present invention may include the following steps:

[0044] S1, Collect remote sensing time-series images of the study area and preprocess the remote sensing time-series images;

[0045] S2, based on farmland plot vector data, combined with preprocessed image data, constructs the plot semantic dataset and plot edge dataset required for the plot extraction framework;

[0046] S3. Construct a branch structure for extracting spatiotemporal spectral features of fields, and train the branch structure using a field semantic dataset to enable adaptive extraction of field features from remote sensing time-series images from three dimensions: time, space, and spectrum.

[0047] S4. Construct a branch structure for extracting field edges. Using the field edge dataset, learn the edge information of the fields from the shallow field features extracted in S2 to obtain the field edge features.

[0048] S5, construct a field feature fusion module, use field edge features to morphologically constrain the semantic information of the field, and realize the extraction of farmland fields.

[0049] In one embodiment of the present invention, the remote sensing time-series image is preprocessed to obtain sample data, specifically including: the remote sensing time-series image is a Sentinel-2 image; the Sentinel-2 image is corrected, including radiometric correction, orthorectification, atmospheric correction and projection transformation; the corrected Sentinel-2 image is cropped, and some image data is discarded; the cropped data is superimposed with images at the same location according to the time dimension as sample data, and the acquisition time of all images in the sample data is recorded in chronological order.

[0050] Specifically, step S1 further includes:

[0051] S11. Acquire remote sensing time-series image data of the study area. The time-series range of the images is referenced to the crop phenology period in the study area. The acquired images are then preprocessed, including radiometric correction, orthorectification, atmospheric correction and projection transformation.

[0052] S12, crop the preprocessed image to 128×128 pixels and discard image data with a missing value rate of more than 35% and a cloud coverage rate of more than 20% after cropping;

[0053] S13, overlay images of the same location according to the time dimension to use them as sample data, and record the acquisition time of all images in the sample data in chronological order.

[0054] In one embodiment of the present invention, farmland plot vector data is obtained, and a plot semantic dataset and a plot edge dataset are determined based on the farmland plot vector data and sample data. Specifically, this includes: preprocessing the farmland plot vector data to obtain plot semantic data labels; extracting edge features from the plot semantic data labels as plot edge detection labels; and pairing sample data, plot semantic data labels, and plot edge detection labels from the same geographical location to obtain the plot semantic dataset and the plot edge dataset.

[0055] Specifically, step S2 further includes:

[0056] S21, preprocessing the vector data of farmland plots interpreted or collected by experts, including converting the vector data to raster (with spatial resolution consistent with remote sensing images) and cropping the plot data (128×128 pixels) as semantic data labels for the plots;

[0057] S22, extract the edges of the semantic data labels of the fields as field edge detection labels;

[0058] S23, pair sample data, field semantic data labels, and field edge detection labels with the same geographical location to construct field semantic dataset and field edge dataset respectively.

[0059] Step S23 further includes: In order to ensure the accuracy of model training, the constructed dataset is divided into a training set, a validation set, and a test set, and randomly divided in a ratio of 3:1:1.

[0060] In one embodiment of the present invention, a field spatiotemporal spectral feature extraction branch structure is constructed, and the branch structure is trained based on a field semantic dataset. Specifically, this includes: performing a convolution operation on the field semantic dataset using a three-dimensional convolution operator to obtain first field features; compressing the spatial dimension of the first field features using global average pooling to obtain second field features; encoding the second field features using a two-dimensional convolution operator to obtain field time-spectral information; processing the field time-spectral information using a one-dimensional convolution operator to obtain field spatial-spectral correlation features; then decoding the field spatial-spectral correlation features using a two-dimensional convolution operator; and finally, processing the decoding result using a Sigmoid activation function to obtain a value that spatially correlates with the first field features. The first field feature is obtained by multiplying the first field feature by a dot product. The second field feature is obtained by compressing the first field feature by average pooling and max pooling. The third field feature is then subjected to a 3D convolution operation to obtain the field's temporal-spatial features. The spectral value of the field's temporal-spatial features, after being processed by the Sigmoid activation function, is multiplied by the first field feature in the spectral dimension to obtain the features in the temporal-spatial dimension. The features in the temporal-spectral dimension are then added together, and the addition result is used for feature extraction based on the 3D convolution operation to obtain the field features in the spatial-spectral-temporal dimension, thus completing the construction of the spatial-spectral-temporal (SST) module. The branch structure for extracting the field's spatiotemporal spectral features is determined based on the SST module.

[0061] Specifically, step S3 further includes:

[0062] S31, using 32 three-dimensional convolution operators with 3×3×3 kernels to perform convolution operations on the input training set to obtain field features;

[0063] S32 utilizes global average pooling to compress the spatial dimension of field features, encodes the temporal-spectral information of the compressed field features using 8 two-dimensional convolution operators with 1×1 kernels, constructs the field spatial-spectral correlation features using 8 one-dimensional convolution operators with 1×1 kernels, and decodes the field spatial-spectral correlation features using 32 two-dimensional convolution operators with 1×1 kernels. After the decoding result is passed through the Sigmoid activation function, it is multiplied by the field features obtained from the input S31 in the spatial dimension to achieve adaptive extraction of features in the temporal-spectral dimension of farmland fields.

[0064] S33 uses average pooling and max pooling operators to compress the temporal dimension of the field features obtained in S31, respectively. It uses 32 three-dimensional convolution operations with 3×3×3 kernels to extract the spatiotemporal features of the field. After inputting the spatiotemporal features into the Sigmoid function, it multiplies the input field features obtained in S31 in the spectral dimension to achieve adaptive extraction of the spatiotemporal features of farmland.

[0065] S34 adds the outputs of S32 and S33, and uses 32 3×3×3 convolution kernels to perform a three-dimensional convolution operation on the added result in the spatiotemporal-spectral dimensions to achieve adaptive extraction of field spatiotemporal-spectral features, thereby constructing the SST module.

[0066] In one embodiment of the present invention, the branch structure for extracting spatiotemporal spectral features of fields is determined according to the SST module. Specifically, the SST module is used in series 7 times through a series strategy to complete the construction of the branch structure for extracting spatiotemporal spectral features of fields. The first 4 SST modules include two output branches, and the output results are shallow field features. The output results of the last 3 SST modules are deep field features. The first 2 SST modules use 32 three-dimensional average pooling layers with a kernel size of 3×3×3.

[0067] Specifically, in order to ensure the learning of field features at different levels, step S34 further includes: the SST module is used 7 times through a concatenation strategy, where the output of the first 4 times is shallow field features and the output of the last 3 times is deep field features. At the same time, in order to reduce the interference of image noise, the first 2 times use 32 three-dimensional average pooling layers with a kernel size of 3×3×3.

[0068] In one embodiment of the present invention, a field edge feature extraction branch structure is constructed, and the field edge feature extraction branch structure is trained based on the field edge dataset. Specifically, this includes: obtaining shallow field features from the field spatiotemporal spectral feature extraction branch structure; performing positional encoding on the temporal information of remote sensing time-series images, and combining the Lightweight Temporal Attention Encoding (LTAE) module to compress the temporal dimension of the shallow field features, thereby enhancing the field spatial-spectral information at important time nodes; processing the field spatial-spectral information through a first differential convolution operator, a second differential convolution operator, and a third differential convolution operator to obtain different enhanced field feature edge information; fusing the different enhanced field feature edge information based on a stacking strategy and a two-dimensional convolution operator to obtain field edge features; performing a two-dimensional convolution operation on the field edge features based on the field feature edge information to obtain a field edge feature probability map with contextual semantic relationships; applying the ReLU activation function to the field edge feature probability map to generate a field edge distribution map, thereby completing the construction of the BE module; and determining the field edge feature extraction branch structure based on the BE module.

[0069] Specifically, step S4 further includes:

[0070] S41, performs location encoding on the temporal information of remote sensing time series images, and combines the LTAE module to compress the temporal dimension of shallow field features, thereby enhancing the spatial-spectral information of fields at important time nodes;

[0071] S42 utilizes the central difference convolution operator, the angular difference convolution operator, and the ring difference convolution operator to extract different field feature edge information from the enhanced field spatial-spectral information. Based on the superposition strategy and a two-dimensional convolution operator with 32 convolution kernels and 1×1, the extracted different field feature edge information is fused to obtain field edge features, thereby improving the model's ability to represent field edges.

[0072] S43, based on the field feature edge information extracted from the first four layers of the field spatiotemporal spectrum feature extraction branch structure in S42, the field edge feature probability map with contextual semantic relationship is extracted from the field edge features using two 2D convolution operators with 1×1 kernel and two 2D convolution operators with 3×3 kernel. The ReLU function is used to generate the field edge distribution map on the field edge feature probability map, thereby constructing the BE module;

[0073] Step S43 further includes: the loss function l used for edge detection adopts the Tanimoto loss function. Considering the convergence speed, the Tanimoto loss function is complemented, as shown in the following formula:

[0074]

[0075]

[0076] in, Represents the Tanimoto loss function. This represents the true value corresponding to a pixel. This represents the posterior probability of the predicted result for a pixel.

[0077] In one embodiment of the present invention, the data output from the field spatiotemporal spectral feature extraction branch structure and the field edge feature extraction branch structure are input into the field feature fusion module for field recognition training to obtain a trained farmland field extraction framework. Specifically, this includes: encoding the field features output by the SST module through the LTA module; supplementing the spatial-spectral information of the field features based on a two-dimensional deconvolution operator; subtracting the field features with the supplemented spatial-spectral information from the field edge features to obtain the field semantic features; performing multi-scale fusion of the field semantic features through dilated spatial convolution pooling pyramid operations; adding the field edge features and the multi-scale fused field semantic features; and performing a two-dimensional convolution operation on the added result to obtain a field semantic segmentation map. This completes the construction of the field feature fusion module and further completes the training of the farmland field extraction framework.

[0078] Specifically, step S5 further includes:

[0079] S51 uses the LTEA module to encode the field spatiotemporal-spectral features output by S34, and combines two two-dimensional deconvolution operators with 3×3 convolution kernels to supplement the spatiotemporal information of the field features;

[0080] S52, subtract the field spatiotemporal-spectral features after supplementing the spatial-spectral information from the extracted field edge features to obtain the field semantic features, and use the dilated spatial convolution pooling pyramid operation to perform multi-scale fusion of the field semantic features to enhance the multi-scale expressive ability of the field semantic features.

[0081] S53 adds the field edge features and the multi-scale fused field semantic features to constrain the geometric edges of the fields. The result of combining the added features with two 3×3 two-dimensional convolution operators is the field semantic segmentation map.

[0082] Step S53 further includes: learning field semantic information using the Tanimoto loss function, therefore the formula for the overall model loss function L is as follows:

[0083]

[0084] in, Represents semantic samples of fields. This indicates the semantic prediction results for the field. Indicates samples at the edge of the field. This indicates the prediction results for field edges. Represents the semantic loss function. Represents the edge loss function. Represents the weight coefficients of the semantic loss function.

[0085] Figure 3A schematic diagram illustrating the steps of a field extraction method based on multi-task deep learning provided in this embodiment of the invention may include the following steps:

[0086] S310: Acquire remote sensing time-series images of the phenological periods of agricultural crops within the area to be identified.

[0087] S320: Input the remote sensing time-series image of the phenological period of agricultural crops in the area to be identified into the trained farmland plot extraction framework for prediction, and obtain the farmland plot identification result.

[0088] The farmland plot extraction framework was trained based on a remote sensing time-series image training set, which is a dataset within the study area that has undergone data preprocessing.

[0089] Specifically, the remote sensing time-series image to be identified is used as input, and a pre-trained farmland plot extraction framework based on multi-task deep learning is used for prediction to obtain the farmland plot identification result.

[0090] For example, the results of farmland plot identification in certain areas of the Netherlands in 2019, such as... Figure 4 The map shown is a distribution map of fields identified as a local area in western Netherlands.

[0091] The above describes a field extraction method based on multi-task deep learning provided by an embodiment of the present invention. Based on the same inventive concept, the present invention also provides a corresponding field extraction device based on multi-task deep learning, such as... Figure 5 As shown, it includes:

[0092] The data acquisition module 502 is used to acquire remote sensing time-series images of the phenological periods of agricultural crops in the area to be identified; the farmland plot identification module 504 is used to input the remote sensing time-series images of the phenological periods of agricultural crops in the area to be identified into a trained farmland plot extraction framework for prediction, and obtain farmland plot identification results; wherein, the farmland plot extraction framework is trained based on a remote sensing time-series image training set, and the remote sensing time-series image training set is a dataset within the study area after data preprocessing.

[0093] This invention also provides a corresponding field extraction device based on multi-task deep learning, such as... Figure 6 As shown, it includes:

[0094] The system includes at least one processor 602, a communication interface 604, a memory 606, and a communication bus 608; wherein the processor 602, the communication interface 604, and the memory 606 communicate with each other through the communication bus 608; the processor 602 can call the logical instructions stored in the memory 606 to cause at least one processor 602 to execute the various method steps of the above embodiments.

[0095] Based on the same idea, some embodiments of the present invention also provide media corresponding to the above methods.

[0096] Some embodiments of the present invention provide a storage medium storing computer-executable instructions, which are executed by a processor to implement the various method steps of the above embodiments.

[0097] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0098] The devices, media, and methods provided in the embodiments of the present invention are one-to-one correspondences. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0099] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process method or product that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process method or product. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process method or product that includes that element.

[0100] The above are merely embodiments of the present invention and are not intended to limit the invention. Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A field extraction method based on multi-task deep learning, characterized in that, include: Acquire remote sensing time-series images of the phenological periods of agricultural crops within the area to be identified; The remote sensing time-series images of agricultural crop phenological periods in the area to be identified are input into the trained farmland plot extraction framework for prediction, and the farmland plot identification results are obtained. The farmland plot extraction framework is trained based on a remote sensing time-series image training set, which is a dataset within the study area after data preprocessing. The training of the farmland plot extraction framework based on the remote sensing time-series image training set specifically includes: acquiring remote sensing time-series images of the study area; preprocessing the remote sensing time-series images to obtain sample data; acquiring vector data of farmland plots corresponding to the study area; constructing a plot semantic dataset and a plot edge dataset based on the plot vector data and the sample data; constructing a plot spatiotemporal spectral feature extraction branch structure; training the plot spatiotemporal spectral feature extraction branch structure based on the plot semantic dataset to obtain plot features in the spatial-spectral-temporal dimensions; constructing a plot edge feature extraction branch structure; training the plot edge feature extraction branch structure based on the plot edge dataset to obtain plot edge features; and constructing a plot feature fusion module, inputting the plot features in the spatial-spectral-temporal dimensions and the plot edge features into the plot feature fusion module for plot recognition training to obtain the trained farmland plot extraction framework.

2. The method according to claim 1, characterized in that, The process of acquiring farmland plot vector data and determining plot semantic datasets and plot edge datasets based on the farmland plot vector data and the sample data specifically includes: The vector data of the farmland plots are preprocessed to obtain semantic data labels for the plots; Extract the edge features of the semantic data labels of the fields as field edge detection labels; By pairing sample data, field semantic data labels, and field edge detection labels from the same geographical location, we obtain field semantic datasets and field edge datasets.

3. The method according to claim 1, characterized in that, The construction of the field spatiotemporal spectral feature extraction branch structure involves training the branch structure based on the field semantic dataset to obtain field features in the spatial-spectral-temporal dimensions, specifically including: The field semantic dataset is convolutionally processed using a 3D convolution operator to obtain the first field feature. The spatial dimension of the first field feature is compressed by global average pooling to obtain the second field feature; the second field feature is encoded by a two-dimensional convolution operator to obtain the field temporal-spectral information; the field temporal-spectral information is processed by a one-dimensional convolution operator to obtain the field spatial-spectral correlation feature; then the field spatial-spectral correlation feature is decoded by a two-dimensional convolution operator; the value obtained after processing the decoding result by the Sigmoid activation function is multiplied by the first field feature in the spatial dimension to obtain the feature in the temporal-spectral dimension. The time dimension of the first field feature is compressed by the average pooling operator and the max pooling operator respectively to obtain the third field feature; the third field feature is subjected to a three-dimensional convolution operation to obtain the field time-space feature; the value obtained by processing the field time-space feature with the Sigmoid activation function is multiplied by the first field feature in the spectral dimension to obtain the feature in the time-space dimension. The features in the time-spectral dimension are added to the features in the time-space dimension, and the added result is extracted based on a three-dimensional convolution operation to obtain the field features in the space-spectral-time dimension, thereby completing the construction of the space-spectral-time SST module. The branch structure for extracting spatiotemporal spectral features of the field is determined based on the SST module.

4. The method according to claim 1, characterized in that, The construction of the field edge feature extraction branch structure, and the training of the field edge feature extraction branch structure based on the field edge dataset to obtain field edge features, specifically includes: The shallow field features are obtained by extracting the branch structure from the spatiotemporal spectral features of the field. The temporal information of remote sensing time-series images is positionally encoded, and the Lightweight Temporal Attention Encoding (LTAE) module is combined to compress the temporal dimension of the shallow field features, thereby enhancing the field spatial-spectral information at important time nodes. The field spatial-spectral information is processed by the first, second, and third differential convolution operators respectively to obtain different enhanced field feature edge information. Based on the overlay strategy and two-dimensional convolution operator, the edge information of different enhanced field features is fused to obtain the field edge features; A two-dimensional convolution operation is performed on the field edge features based on the field feature edge information to obtain a field edge feature probability map with contextual semantic relationship; The ReLU activation function is applied to the field edge feature probability map to generate a field edge distribution map, thereby completing the construction of the BE module; The BE module determines the branch structure for extracting field edge features.

5. The method according to claim 1, characterized in that, The step of inputting the field features in the spatial-spectral-temporal dimensions and the field edge features into the field feature fusion module for field recognition training to obtain a trained farmland field extraction framework specifically includes: The field features output by the SST module are encoded by the LTEA module, and the spatial-spectral information of the field features is supplemented by a two-dimensional deconvolution operator; Subtracting the field features from the field edge features after supplementing with spatial-spectral information yields the field semantic features. Multi-scale fusion of the semantic features of the fields is performed by dilated spatial convolution pooling pyramid operation; The field edge features and the multi-scale fused field semantic features are added together, and a two-dimensional convolution operation is performed on the added result to obtain a field semantic segmentation map. This completes the construction of the field feature fusion module and further completes the training of the farmland field extraction framework.

6. The method according to claim 1, characterized in that, The remote sensing time-series images are preprocessed to obtain sample data, specifically including: The remote sensing time-series image is a Sentinel-2 image; The Sentinel-2 image is corrected, including radiometric correction, orthorectification, atmospheric correction, and projection transformation. The corrected Sentinel-2 images were cropped, and some image data was discarded; The cropped data is used to overlay images of the same location according to the time dimension to form sample data, and the acquisition time of all images in the sample data is recorded in chronological order.

7. The method according to claim 3, characterized in that, The step of determining the spatiotemporal spectral feature extraction branch structure of the field based on the SST module specifically includes: The SST module was used seven times in a series to complete the construction of the branch structure for extracting spatiotemporal spectral features of the field. The first four SST modules include two output branches, and the output results are shallow field features. The output results of the last three SST modules are deep field features. The first two SST modules use 32 three-dimensional average pooling layers with a kernel size of 3×3×3.

8. A field extraction device based on multi-task deep learning, characterized in that, include: The data acquisition module is used to acquire remote sensing time-series images of the phenological periods of agricultural crops in the area to be identified; The farmland plot identification module is used to input remote sensing time-series images of agricultural crop phenological periods in the area to be identified into a trained farmland plot extraction framework for prediction, and obtain farmland plot identification results. The farmland plot extraction framework is trained based on a remote sensing time-series image training set, which is a dataset within the study area after data preprocessing. The device is also used to: train a farmland plot extraction framework based on a remote sensing time-series image training set, specifically including: acquiring remote sensing time-series images of the study area, preprocessing the remote sensing time-series images to obtain sample data; acquiring farmland plot vector data corresponding to the study area, and constructing a plot semantic dataset and a plot edge dataset based on the farmland plot vector data and the sample data; constructing a plot spatiotemporal spectral feature extraction branch structure, and training the plot spatiotemporal spectral feature extraction branch structure based on the plot semantic dataset to obtain plot features in the spatial-spectral-temporal dimensions; constructing a plot edge feature extraction branch structure, and training the plot edge feature extraction branch structure based on the plot edge dataset to obtain plot edge features; and constructing a plot feature fusion module, inputting the plot features in the spatial-spectral-temporal dimensions and the plot edge features into the plot feature fusion module for plot recognition training to obtain a trained farmland plot extraction framework.

9. A field extraction device based on multi-task deep learning, characterized in that, include: At least one processor; as well as, The memory is communicatively connected to the at least one processor via a bus; wherein, The memory stores instructions executable by the at least one processor, which are executed to implement the method as described in any one of claims 1-7.

Images