A potato planting area extraction method integrating prior spectral information and multi-temporal deep learning network

Abstract

The present invention proposes a method for extracting potato planting areas by integrating prior spectral information and multi-temporal deep learning networks, including: obtaining remote sensing image data of potato growth period; obtaining potato prior spectral information based on the remote sensing image data; constructing a data set based on the potato prior spectral information and spectral image information; training a multi-temporal deep learning network using the data set to obtain a potato planting area prediction model; inputting the remote sensing image data of the potato planting area to be predicted into the potato planting area prediction model to obtain the remote sensing extraction result of the potato planting area. The present invention adopts a novel multi-temporal deep learning network, which comprehensively improves the diagnostic ability of the classifier by integrating multi-source information such as prior information, space and time, and overcomes the disadvantage that single-source feature information is poor in identifying homogeneous crops.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural remote sensing, and in particular relates to a method for extracting potato planting areas by integrating prior spectral information and a multi-temporal deep learning network. Background Art

[0002] Potato is the fourth largest food crop after wheat, rice and corn. Accurate and timely monitoring of potato spatiotemporal distribution is essential for large-scale yield estimation and monitoring dynamic crop growth. Traditionally, crop spatial distribution information is usually obtained and updated by field surveys. However, this process is prone to errors, time-consuming and expensive. Sentinel-2 remote sensing images have become an effective tool for large-scale crop mapping due to their wide spatial coverage, high spatial resolution and revisit frequency, low cost and simple acquisition. However, during the critical growth period of crops, they are often affected by cloudy, rainy and foggy weather, resulting in the inability to obtain sufficient and highly available optical image data. Synthetic Aperture Radar (SAR) has the advantage of being independent of weather, so optical and SAR data are combined to improve the accuracy of potato mapping.

[0003] In practical applications, it is found that the challenge of potato planting area extraction is that the planting structure is relatively fragmented and the crops are seriously mixed. Most crop extraction in planting areas uses feature optimization strategies to screen out sensitive feature combinations, and then inputs them into traditional machine learning models for prediction. This type of method usually makes it difficult to ensure the spatiotemporal generalization of the model, resulting in unsatisfactory classification accuracy in complex planting areas. Compared with traditional machine learning, data-driven deep learning networks are more conducive to accurate crop extraction in large-scale areas. However, due to the heterogeneity of spectra and differences in crop management, deep learning models that rely on a single phase find it difficult to accurately identify crop types in mixed planting areas.

[0004] Typically, potatoes have unique growth patterns, calendars, and trends compared to other crops. The multi-temporal structure of the time series model matches the multi-temporal satellite observations of the crop growth process, and can capture the changes and dynamic characteristics of the crop growth process. However, deep learning models may often learn unimportant features, causing the model to focus on undesirable features. This feature collapse problem limits the further improvement of the model's generalization ability. Prior information of crops, such as unique phenological characteristics, spectral index, etc., is an important indicator reflecting the growth status and characteristics of crops. In particular, it is very useful for solving the problem of "same object, different spectrum" or "different objects, same spectrum". Therefore, how to embed prior information into a multi-temporal deep learning network to achieve high-precision extraction of potatoes has become a technical problem that needs to be solved in the field of agricultural remote sensing. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of low precision, poor transplantability, and lack of practical potato planting extraction methods in existing crop extraction methods, and to provide a potato planting area extraction method that integrates prior information and multi-temporal deep learning networks to solve the above problems. The method adopts a novel multi-temporal deep learning network, which integrates multi-source information such as prior information, space and time, comprehensively improves the diagnostic ability of the classifier, and overcomes the disadvantage that single-source feature information is poor in identifying homogeneous crops.

[0006] To achieve the above object, the present invention provides a potato planting area extraction method integrating prior spectral information and multi-temporal deep learning network, comprising:

[0007] Obtain remote sensing image data of potato growth period;

[0008] Based on the remote sensing image data, obtaining cloud-free synthetic image data of potato phenological period;

[0009] Based on the cloud-free synthetic image data, obtaining potato prior spectral information;

[0010] Constructing a data set based on the potato prior spectral information and cloud-free synthetic image data;

[0011] Using the data set to train a multi-temporal deep learning network to obtain a potato planting area prediction model;

[0012] The remote sensing image data of the potato planting area to be predicted is input into the potato planting area prediction model to obtain the remote sensing extraction results of the potato planting area.

[0013] Optionally, cloud-free synthetic image data of potato phenological period is obtained including:

[0014] Based on the remote sensing image data, obtaining potato phenological period;

[0015] In each potato phenological period, the median of the valid observation values ​​of all remote sensing image data within the time window is calculated to obtain the cloud-free synthetic image data of the potato phenological period.

[0016] Optionally, remote sensing image data of potato growth period is obtained including:

[0017] Search all preset available images of potatoes throughout the growth period on the GEE platform;

[0018] All preset available images are declouded using the preset bands, and all available pixels with cloud cover less than the preset value are retained to generate images without cloud interference in the study area.

[0019] The nearest neighbor resampling algorithm was used to resample the resolution of each band in the cloud-free images of the study area to obtain the initial remote sensing image data of the entire potato growth period;

[0020] The ESRI land cover data were applied to the initial remote sensing image data for masking, generating a stable cultivated land layer, excluding non-crop pixels, and obtaining the final remote sensing image data of the potato growing period.

[0021] Optionally, based on the remote sensing image data, obtaining the potato phenological period includes:

[0022] Obtaining the spectral index NDVI and EVI time series derived from the remote sensing image data;

[0023] For the missing observations in the NDVI and EVI time series, we search for the nearest preset high-quality observations in the forward and backward time windows respectively, and use the linear interpolation method to fill the current missing values ​​according to the relationship between the two high-quality observations found;

[0024] For the interpolated NDVI and EVI time series, the SG filter algorithm was used for smoothing and filtering to obtain the complete potato growth period time series;

[0025] The potato phenological period is extracted based on the potato growth period time series; wherein the potato phenological period includes: potato budding period, flowering period, tuber enlargement period, starch accumulation period and maturity period.

[0026] Optionally, based on the cloud-free synthetic image data, obtaining potato prior spectral information includes:

[0027] Using the cloud-free synthetic image of the potato phenological period, spectral band feature U, polarization feature V, spectral index feature VI and phenological feature W are extracted to construct multi-temporal potato prior spectral information;

[0028] The potato prior spectral information is:

[0029] P ro =[U,V,VI,W]

[0030] Among them, P ro represents the potato prior spectral information vector;

[0031] The spectral band characteristic U is:

[0032]

[0033] Among them, U represents the spectral band characteristics, Indicates t 1 Reflectance of band 1 during the phenological period, Indicates t2 Reflectance of band 1 during the phenological period, Indicates t 5 Reflectance of band 1 during the phenological period, Indicates t 1 Reflectance of band 2 during the phenological period, Indicates t 2 Reflectance of band 2 during the phenological period, Indicates t 5 Reflectance of band 2 during the phenological period, Indicates t 1 The reflectance of band c during the phenological period, Indicates t 2 The reflectance of band c during the phenological period, Indicates t 5 The reflectance of band c in the phenological period, c represents the number of spectral bands;

[0034] The polarization characteristic V is:

[0035]

[0036] Where V represents the polarization characteristic, Indicates t 1 Polarization band 1 of phenological period, Indicates t 5 Polarization band 1 of phenological period, Indicates t 1 Polarization band 2 of phenological period, Indicates t 5 Polarization band 2 of phenological period, Indicates t 1 Polarization band m of phenological period, Indicates t 5 Polarization band m of the phenological period, where m represents the number of polarization bands;

[0037] The spectral index characteristic VI is:

[0038]

[0039] Among them, VI represents the spectral index characteristic, Indicates t 1 Category 1 vegetation index for phenological period, Indicates t 5 Category 1 vegetation index for phenological period, Indicates t 1 Category 2 vegetation index of phenological period, Indicates t 5 Category 2 vegetation index of phenological period, Indicates t 1 Category k vegetation index of phenological period, Indicates t 5 Category k vegetation index of phenological period, k represents the number of vegetation index categories;

[0040] The phenological characteristics W are:

[0041] W = [PPI min ,PPI max ,PPI mean ,PI]

[0042] Among them, W represents phenological characteristics, PPI min Indicates the minimum value of PPI in the phenological period, PPI max Indicates the maximum value of PPI in the phenological period, PPI mean It represents the average value of PPI in phenological period, and PI represents phenological index.

[0043] Optionally, the multi-temporal deep learning network is a dual-stream deep neural network structure, the upper branch is a 1D CNN network module, which is used to process potato prior spectral information and automatically extract convolution features, and the lower branch is a CNN2D-LSTM network module, which is used to process cloud-free synthetic image data and obtain spatiotemporal information features;

[0044] The multi-temporal deep learning network uses N×N image blocks as object units for prediction, and uses a connection layer to connect the outputs of the two branches together, and outputs the predicted labels through Softmax.

[0045] Optionally, the 1D CNN network module includes 4 Conv1D convolutional layers, 3 pooling layers and 1 flatten layer;

[0046] The input of the 1D CNN network module is: the average value of the potato prior spectral information of the N×N image blocks calculated on the cloud-free synthetic image data;

[0047] The average value of the potato prior spectral information is subjected to 4 convolution blocks and 3 poolings, and finally a feature of a preset size is output in a 1D CNN network, and the feature of the preset size is reshaped using a flattening layer.

[0048] Optionally, the CNN2D-LSTM network module includes a CNN2D subnetwork and a LSTM subnetwork;

[0049] The input of the CNN2D subnetwork is: N×N image blocks extracted from the cloud-free synthetic image data, and the output is a preset dimensional temporal feature matrix, and the preset dimensional temporal feature matrix is ​​input into the LSTM subnetwork for spatiotemporal feature extraction.

[0050] The present invention has the following beneficial effects:

[0051] The present invention first obtains remote sensing image data of potato growth period; secondly, based on the remote sensing image data, obtains potato prior spectral information; then, based on the potato prior spectral information and spectral image information, constructs a data set; then, uses the data set to train a multi-temporal deep learning network to obtain a potato planting area prediction model; finally, the remote sensing image data of the potato planting area to be predicted is input into the potato planting area prediction model to obtain the remote sensing extraction result of the potato planting area. The present invention adopts a novel multi-temporal deep learning network, which comprehensively improves the diagnostic ability of the classifier by integrating multi-source information such as prior information, space and time, and overcomes the disadvantage that single-source feature information is poor in identifying homogeneous crops. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 A schematic diagram of a process for extracting potato planting areas according to an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of the network structure of a multi-temporal deep learning network extracted from a potato planting area according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the potato planting area extraction results using the multi-temporal deep learning network model and the pixel-based method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] This embodiment provides a potato planting area extraction method integrating prior spectral information and a multi-temporal deep learning network, including:

[0059] Obtain remote sensing image data of potato growth period;

[0060] Based on the remote sensing image data, obtaining cloud-free synthetic image data of potato phenological period;

[0061] Based on the cloud-free synthetic image data, obtaining potato prior spectral information;

[0062] Constructing a data set based on the potato prior spectral information and cloud-free synthetic image data;

[0063] Using the data set to train a multi-temporal deep learning network to obtain a potato planting area prediction model;

[0064] The remote sensing image data of the potato planting area to be predicted is input into the potato planting area prediction model to obtain the remote sensing extraction results of the potato planting area.

[0065] Furthermore, cloud-free synthetic image data of potato phenology period are obtained including:

[0066] Obtain potato phenological period based on remote sensing image data;

[0067] In each potato phenological period, the median of the valid observation values ​​of all remote sensing image data within the time window is calculated to obtain the cloud-free synthetic image data of the potato phenological period.

[0068] Furthermore, remote sensing image data of potato growth period are obtained including:

[0069] Search all preset available images of potatoes throughout the growth period on the GEE platform;

[0070] All preset available images are declouded using the preset bands, and all available pixels with cloud cover less than the preset value are retained to generate images without cloud interference in the study area.

[0071] The nearest neighbor resampling algorithm was used to resample the resolution of each band in the cloud-free images of the study area to obtain the initial remote sensing image data of the entire potato growth period;

[0072] The ESRI land cover data were applied to the initial remote sensing image data for masking, generating a stable cultivated land layer, excluding non-crop pixels, and obtaining the final remote sensing image data of the potato growing period.

[0073] Furthermore, based on remote sensing image data, the potato phenological period is obtained including:

[0074] Obtain the spectral index NDVI and EVI time series derived from remote sensing image data;

[0075] For the missing observations in the NDVI and EVI time series, we search for the nearest preset high-quality observations in the forward and backward time windows respectively, and use the linear interpolation method to fill the current missing values ​​according to the relationship between the two high-quality observations found;

[0076] For the interpolated NDVI and EVI time series, the SG filter algorithm was used for smoothing and filtering to obtain the complete potato growth period time series;

[0077] The potato phenological period is extracted based on the potato growth period time series; the potato phenological period includes: potato budding period, flowering period, tuber enlargement period, starch accumulation period and maturity period.

[0078] Furthermore, based on the cloud-free synthetic image data, the potato prior spectral information is obtained, including:

[0079] Using cloud-free synthetic images of potato phenological period, spectral band features U, polarization features V, spectral index features VI and phenological features W were extracted to construct multi-temporal prior spectral information of potato.

[0080] Furthermore, the multi-temporal deep learning network is a two-stream deep neural network structure, the upper branch is a 1D CNN network module, which is used to process potato prior spectral information, and the lower branch is a CNN2D-LSTM network module, which is used to process cloud-free synthetic image data and obtain spatiotemporal information;

[0081] The multi-temporal deep learning network uses N×N image patches as object units for prediction, connects the outputs of the two branches together using a connection layer, and outputs the predicted label through Softmax.

[0082] Furthermore, the 1D CNN network module includes 4 Conv1D convolutional layers, 3 pooling layers and 1 flatten layer;

[0083] The input of the 1D CNN network module is: the average value of potato prior spectral information of N×N image patches calculated on cloud-free synthetic image data;

[0084] The average value of potato prior spectral information is processed through 4 convolution blocks and 3 pooling blocks, and finally outputs features of preset size in the 1D CNN network, and the features of preset size are reshaped using the flatten layer.

[0085] Furthermore, the CNN2D-LSTM network module includes a CNN2D subnetwork and a LSTM subnetwork;

[0086] The input of the CNN2D subnetwork is: N×N image blocks extracted from the cloud-free synthetic image data, and the output is a preset dimensional temporal feature matrix, which is then input into the LSTM subnetwork for spatiotemporal feature extraction.

[0087] like Figure 1 As shown, the potato planting area extraction method integrating prior spectral information and multi-temporal deep learning network described in this embodiment specifically includes the following steps:

[0088] Step S1, acquisition and processing of remote sensing satellites: According to the growth period of potatoes in the study area, Sentinel-1, Sentinel-2 and ESRI land cover data of the study area are acquired. The images within the time range are declouded to obtain Sentinel-2 images with missing and no cloud interference. The non-agricultural elements in the study area are eliminated using ESRI land cover data to obtain the distribution of farmland in the study area. Sentinel-1 is synthetic aperture radar (SAR) data, and Sentinel-2 is multispectral data.

[0089] Note: During the critical phenological period of crops, crops are often affected by cloudy, rainy, foggy weather, etc., which makes it impossible to obtain sufficient and highly available optical image data, greatly limiting its application in actual research. This invention combines optical and SAR data to improve the extraction accuracy of planting areas with severe cloud pollution.

[0090] (1) All available images of the potato growing season were retrieved on the GEE platform, and all images were declouded using the QA60 band, retaining all available pixels with cloud cover less than 30%, to generate cloud-free images of the study area. The nearest neighbor resampling algorithm was used to resample the resolution of each band to 10 m. At the same time, Sentinel-1 images of the potato growing season were obtained.

[0091] (2) ESRI land cover data were applied to Sentinel-2 images for masking to generate a stable cultivated land layer and exclude non-crop pixels.

[0092] Step S2, determination of potato phenological phase: five potato phenological phases, namely, budding (P1), flowering (P2), tuber enlargement (P3), starch accumulation (P4), and maturity (P5), were determined using the spectral index NDVI and EVI time series derived from Sentinel-2.

[0093] (1) For the missing observations in the NDVI and EVI time series, search for the nearest high-quality observations in the forward and backward time windows respectively. Some quality control criteria, such as data reliability and cloud coverage, can be considered to select high-quality observations. Use the linear interpolation method to fill the current missing values ​​based on the relationship between the two high-quality observations found. The calculation formula is as follows:

[0094]

[0095] Where: NDVI t and EVI t is the interpolated value at time t; and Interpolation time window forward t1 The observed value at time; and The interpolation time window is backward t 0 The observed value at time t; t is the time at which the missing value is located.

[0096] (2) For the interpolated time series, the SG filtering algorithm is used for smoothing and filtering to construct a complete potato growth period time series. The calculation formula is as follows:

[0097]

[0098] Where: Filtered time series; Y j is the original time series; c i is the filter coefficient; N is the sliding window width.

[0099] (3) Extraction of potato phenological stages based on NDVI and EVI time profiles. Bud stage: As the potato grows, NDVI and EVI increase monotonically. Flowering stage: The stems and leaves grow rapidly, and NDVI and EVI rise rapidly. Tuber enlargement stage: The stage when NDVI and EVI reach their peak during the potato growing season. Starch accumulation stage: NDVI and EVI decrease slowly. Maturity stage: NDVI and EVI decrease sharply. Based on these spectral characteristics, five potato phenological stages were identified: bud stage (P1), flowering stage (P2), tuber enlargement (P3), starch accumulation (P4), and maturity stage (P5).

[0100] (4) Verify and calibrate the determined phenological period based on the NDVI or EVI threshold.

[0101] Step S3, synthesis of remote sensing images: In each potato phenological period, the median of the effective observation values ​​of all remote sensing images in these time windows is calculated to obtain cloud-free synthetic images of phenological periods P1-P5 respectively.

[0102] Note: The special phenological characteristics of potatoes are effective indicators for distinguishing them from other mixed crops. All remote sensing images are synthesized according to the phenological period. The constructed image time series not only reflects the unique growth calendar characteristics of potatoes, but also reduces the uncertainty of the number of effective pixels caused by the rainy season and satellite revisit period.

[0103] (1) Due to the influence of cloud and rain, it is usually difficult to collect complete Sentinel-2 images to construct time series images of the entire growth period. Considering the availability of data and the complexity of calculation, the median method was used to calculate the reconstruction of each phenological period, and 5 cloud-free synthetic images were generated at the budding, flowering, tuber enlargement, starch accumulation and maturity stages.

[0104] (2) The two SAR polarization characteristics (VV polarization and VH polarization) of Sentinel-1 images are calculated using the remote sensing image synthesis method to obtain the synthesized VV polarization and VH of the crops in the study area during the growth period.

[0105] Step S4, establishment of potato prior spectral information: using cloud-free synthetic images of five phenological periods to extract spectral band features U, polarization features V, spectral index features VI and phenological features W, and constructing a multi-temporal potato prior spectral information vector P ro =[U,V,VI,W].

[0106] (1) Extract the spectral feature vectors composed of the Sentinel-2 image waves Red, Green, Blue, Red Edge 1, Red Edge 2, Red Edge 3, NIR, VRE, SWIR 1, and SWIR 2 at five potato growth stages

[0107] (2) Extraction of polarization characteristics of VV, VH, and VV / VH components of Sentinel-1 image waves at five potato growth stages Wherein, m represents the mth polarization band.

[0108] (3) Construct the potato productivity index PPI, the calculation formula is as follows:

[0109]

[0110] Where: blue , ρ green , ρ red , ρ RE1 , ρ 8A and ρ WV They represent B2, B3, B4, B5, B8A and B9 of the Sentinel-2 images respectively.

[0111] (5) Using the PPI index to reconstruct the unique phenological characteristic vector of potato W = [PPI min ,PPI max ,PPI mean ,PI], the specific calculation is as follows:

[0112] PPI min =min(PPI t1 ,PPI t2 ,…,PPI t5 )

[0113] PPI max =max(PPI t1 ,PPI t2 ,…,PPI t5)

[0114] PPI mean =mean(PPI t1 ,PPI t2 ,…,PPI t5 )

[0115]

[0116] Where: PPI t represents the PPI of the tth phenological period, and n is the number of phenological periods.

[0117] (6) Calculate NDVI, EVI, GNDVI, MTCI, NDSI, PPI, NDVIre1, NDVIre2, and REP vegetation indices based on Sentinel-2 images and construct a spectral index vector Where k represents the kth spectral index.

[0118] (7) Combining the above spectral band features U, polarization features V, spectral index features VI and phenological features W, a multi-temporal potato prior spectral information vector P is constructed. ro =[U,V,VI,W].

[0119] Step S5, construction of multi-temporal deep learning network (MTDLN-PS): The multi-temporal deep learning network is designed as a dual-stream deep neural network architecture. The upper branch is designed as a 1D CNN network, and the lower branch is designed as a CNN2DLSTM network. The two sub-networks are connected in parallel. Figure 2 shown.

[0120] (1) The multi-temporal deep learning network is constructed as a two-stream deep neural network structure to simultaneously process prior spectral information and spectral image information. The design of each branch in the two-branch network tends to focus on the advantages of its own data. The upper branch is designed as a 1D CNN network module to process prior spectral information. The lower branch is designed as a CNN2D-LSTM network module to process spatiotemporal information.

[0121] (2) In order to eliminate salt and pepper noise and ensure the continuity of the extracted potato area, the multi-temporal deep learning network uses N×N image blocks as object units for prediction. The size of the N×N image block can be set to 2×2, 3×3, and 4×4, etc., which is determined according to the actual area of ​​the study area.

[0122] Note: Potatoes are usually planted in rows. When the plants are small, they cannot completely cover the ground, so the potato vegetation signal in the remote sensing image is mixed with soil background information. If the existing pixel-based crop extraction method is used, mixed pixels are easily identified as non-potato targets, resulting in a large number of noise points in the extraction area. Aiming at the special planting pattern of potato crops, the present invention designs a multi-temporal deep learning network with N×N image blocks as input.

[0123] (3) Calculate the prior spectral information P of the N×N image block on the synthetic images P1, P2, P3, P4, and P5 in sequence ro =The average value of [U, V, VI, W] is used as the time series input of the upper branch 1D CNN network module. The feature dimension of the time series data established in the five phenological periods is 114×1, which represents the feature dimension and the number of channels respectively.

[0124] (4) The 1D CNN network module includes 4 Conv1D convolutional layers, 3 pooling layers and 1 Flatten layer. The 4 Conv1D convolutional layers are all 1D convolutional layers, and the number of channels is 32, 64, 128, and 32 respectively. The size of the convolution kernel is 3×1, and each convolutional layer is followed by a batch normalization layer and a relu activation function layer. The 3 pooling layers are all 2×1 maximum pooling layers.

[0125] (5) After 4 convolution blocks and 3 pooling blocks, the 1D CNN network finally outputs features of size 10×1×32, and the features are reshaped to 320×1 using the Flatten layer.

[0126] (6) The CNN2D-LSTM network module is composed of two sub-networks, CNN2D and LSTM, which are used to process spectral image information. The spectrum involved in the N×N image block is the 9 bands of Red, Green, Blue, Red Edge1, RedEdge2, Red Edge 3, NIR, SWIR1 and SWIR2 of Sentinel-2 image and the 3 bands of VV, VH and VV / VH of Sentinel-1. The dimension of the spectral image input to the CNN2D network is N×N×12.

[0127] (7) Extract N×N image blocks from the synthetic images of P1, P2, P3, P4, and P5 and input them into the CNN2D network, outputting an n×t5-dimensional temporal feature matrix And input into the LSTM network to extract spatiotemporal features.

[0128] (8) The CNN2D-LSTM described is composed of 3 2D convolutional layers, 2 LSTM units and 1 dense layer.

[0129] (9) Use the connection layer to connect the outputs of the two branches together and output the predicted label through Softmax.

[0130] Step S6, extraction of potato planting areas: input the remote sensing images of the potato planting areas in the study area into the trained multi-temporal deep learning network to obtain the remote sensing extraction results of the potato planting areas.

[0131] A dataset was established by cropping N×N image patches from the labeled potato-growing areas and training a multi-temporal deep learning network.

[0132] The remote sensing images of the study area were divided into N×N image blocks and sequentially input into the trained multi-temporal deep learning network for prediction to obtain a complete potato distribution map in the study area.

[0133] The effect of this embodiment is further described below in conjunction with experiments:

[0134] As shown in Table 1, in order to verify the effectiveness of the method proposed in this embodiment, three deep learning models, CNN1D, LSTM and CNN-LSTM, and the machine learning model RF were selected for comparison with MTDLN-PS. Among them, the input features of the CNN1D, LSTM and RF models are the prior spectral information of the five phenological periods, and the CNN-LSTM input is an N×N image block. The accuracy evaluation indicators used include overall accuracy (OA), F1-Score and Kappa coefficient. The comparison results show that the method MTDLN-PS proposed in this embodiment is the optimal model for potato planting area extraction, and its OA, F1-Score and Kappa are 91.65%, 92.67% and 0.83 respectively. The results show that the fusion of prior spectral information can effectively improve the extraction accuracy of potatoes in complex planting areas.

[0135] Table 1 Comparison results of different models

[0136]

[0137] Figure 3 The potato planting area extraction results of the pixel-based method (CNN1D) and MTDLN-PS are shown. Figure 3 The first column is the original remote sensing image, the second column is the extraction result of the pixel-based method, and the third column is the extraction result using the MTDLN-PS network. In general, most potato-growing areas can be successfully extracted and produce satisfactory visual effects. The potato-growing areas extracted by the pixel-based method produce more salt and pepper noise and misclassification results. In contrast, since this embodiment uses the image block-based MTDLN-PS network, the extracted potato distribution is more refined, and the field integrity and continuity are also better.

[0138] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A potato planting area extraction method integrating prior spectral information and multi-temporal deep learning network, characterized in that: include: Obtain remote sensing image data of potato growth period; Based on the remote sensing image data, obtaining cloud-free synthetic image data of potato phenological period; Based on the cloud-free synthetic image data, obtaining potato prior spectral information; Constructing a data set based on the potato prior spectral information and cloud-free synthetic image data; Using the data set to train a multi-temporal deep learning network to obtain a potato planting area prediction model; Inputting the remote sensing image data of the potato planting area to be predicted into the potato planting area prediction model to obtain the remote sensing extraction results of the potato planting area; Cloud-free synthetic image data for potato phenology include: Based on the remote sensing image data, obtaining potato phenological period; In each potato phenological period, the median of the effective observation values ​​of all remote sensing image data in the time window is calculated to obtain cloud-free synthetic image data of the potato phenological period; Based on the remote sensing image data, the potato phenological period is obtained including: Obtaining the spectral index NDVI and EVI time series derived from the remote sensing image data; For the missing observations in the NDVI and EVI time series, we search for the nearest preset high-quality observations in the forward and backward time windows respectively, and use the linear interpolation method to fill the current missing values ​​according to the relationship between the two high-quality observations found; For the interpolated NDVI and EVI time series, the SG filter algorithm was used for smoothing and filtering to obtain the complete potato growth period time series; Extracting potato phenological periods based on potato growth period time series; wherein the potato phenological periods include: potato budding period, flowering period, tuber enlargement period, starch accumulation period and maturity period; Based on the cloud-free synthetic image data, obtaining potato prior spectral information includes: Using the cloud-free synthetic image of the potato phenological period, spectral band feature U, polarization feature V, spectral index feature VI and phenological feature W are extracted to construct multi-temporal potato prior spectral information; The potato prior spectral information is: P ro =[U,V,VI,W] Among them, P ro represents the potato prior spectral information vector; The spectral band characteristic U is: Among them, U represents the spectral band characteristics, represents the reflectance of band 1 in phenological period t1, represents the reflectance of band 1 in the t2 phenological period, represents the reflectance of band 1 in the t5 phenological period, represents the reflectance of band 2 in phenological period t1, represents the reflectance of band 2 in phenological period t2, represents the reflectance of band 2 in the t5 phenological period, represents the reflectance of band c in the phenological period t1, represents the reflectance of band c in the t2 phenological period, It represents the reflectance of band c in phenological period t5, and C represents the number of spectral bands; The polarization characteristic V is: Where V represents the polarization characteristic, represents the polarization characteristics of band 1 in phenological period t1, represents the polarization characteristics of band 1 in the t5 phenological period, represents the polarization characteristics of band 2 in the phenological period t1, represents the polarization characteristics of band 2 in the t5 phenological period, represents the polarization characteristics of band m in the phenological period t1, It represents the polarization characteristics of band m in the phenological period t5, where m represents the number of polarization bands; The spectral index characteristic VI is: Among them, VI represents the spectral index characteristic, represents the category 1 vegetation index of phenological period t1, represents the category 1 vegetation index of the phenological period t5, represents the category 2 vegetation index of phenological period t1, represents the category 2 vegetation index of the t5 phenological period, represents the vegetation index of category k in phenological period t1, represents the vegetation index of category k in phenological period t5, where k represents the number of vegetation index categories; The phenological characteristics W are: W=[PPI min ,PPI max ,PPI mean ,PI] Among them, W represents phenological characteristics, PPI min Indicates the minimum value of PPI in the phenological period, PPI max Indicates the maximum value of PPI in the phenological period, PPI mean represents the average value of PPI in the phenological period, and PI represents the phenological index; PPI is the potato productivity index, and the calculation formula is as follows: Among them, ρ blue , ρ green , ρ red , ρ RE1 , ρ 8A and ρ WV They represent B2, B3, B4, B5, B8A and B9 of Sentinel-2 images respectively; Among them, PPI t represents the PPI of the tth phenological period, and n is the number of phenological periods; The multi-temporal deep learning network is a dual-stream deep neural network structure, the upper branch is a 1D CNN network module, which is used to process potato prior spectral information and automatically extract convolution features, and the lower branch is a CNN2D-LSTM network module, which is used to process cloud-free synthetic image data and obtain spatiotemporal information features; The multi-temporal deep learning network uses N×N image blocks as object units for prediction, and uses a connection layer to connect the outputs of the two branches together, and outputs the predicted labels through Softmax.

2. The potato planting area extraction method integrating prior spectral information and multi-temporal deep learning network according to claim 1 is characterized in that: Remote sensing image data for potato growth period include: Search all preset available images of potatoes throughout the growth period on the GEE platform; All preset available images are declouded using the preset bands, and all available pixels with cloud cover less than the preset value are retained to generate images without cloud interference in the study area. The nearest neighbor resampling algorithm was used to resample the resolution of each band in the cloud-free images of the study area to obtain the initial remote sensing image data of the entire potato growth period; The ESRI land cover data were applied to the initial remote sensing image data for masking, generating a stable cultivated land layer, excluding non-crop pixels, and obtaining the final remote sensing image data of the potato growing period.

3. The potato planting area extraction method integrating prior spectral information and multi-temporal deep learning network according to claim 1 is characterized in that: The 1D CNN network module includes 4 Conv1D convolutional layers, 3 pooling layers and 1 flatten layer; The input of the 1D CNN network module is: the average value of the potato prior spectral information of the N×N image blocks calculated on the cloud-free synthetic image data; The average value of the potato prior spectral information is subjected to 4 convolution blocks and 3 pooling blocks, and finally outputs features of a preset size in the 1DCNN network, and the features of the preset size are reshaped using the flatten layer.

4. The method for extracting potato planting areas by integrating prior spectral information and multi-temporal deep learning network according to claim 1, characterized in that: The CNN2D-LSTM network module includes a CNN2D subnetwork and a LSTM subnetwork; The input of the CNN2D subnetwork is: N×N image blocks extracted from the cloud-free synthetic image data, and the output is a preset dimensional temporal feature matrix, and the preset dimensional temporal feature matrix is ​​input into the LSTM subnetwork for spatiotemporal feature extraction.

Images