Deep learning crop extraction method, device and equipment integrating key growth periods

Abstract

The present invention provides a deep learning crop extraction method, device and equipment integrating key growth periods, including: obtaining initial earth observation data corresponding to the study area, and screening two-period target earth observation data from the initial earth observation data according to the crop growth distribution characteristics; using the crop extraction model obtained through training, predicting crop distribution based on the two-period target earth observation data to generate crop distribution prediction data corresponding to the study area; optimizing the crop patches contained in the crop distribution prediction data to generate crop distribution results corresponding to the study area. The present invention can not only realize crop extraction using earth observation data of different resolutions, but also make full use of the growth period of crops to realize large-scale and accurate crop distribution prediction.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a deep learning crop extraction method, device and equipment integrating key growth periods. Background Art

[0002] Rapid, accurate and large-scale acquisition of the planting status and area of ​​grain crops plays an important role in crop growth monitoring, yield assessment, disaster warning, and food security.

[0003] At present, there are two main extraction algorithms for grain crops, namely time series analysis and single-period classification. The time series analysis method is based on the phenological characteristics of crops and uses medium- and low-resolution long-time series earth observation data to establish a classification model to achieve the purpose of rapid extraction. However, this method is difficult to obtain long-time series data in full, and the processing of time data is complex, which seriously restricts its engineering application; the single-period classification method is mainly based on single-view high-resolution earth observation remote sensing data, and uses a variety of classification methods such as pixel-oriented, object-oriented, and deep learning to achieve a more precise extraction of crops. However, this method is difficult to monitor on a large scale, and rarely uses crop-specific phenological information, so the extraction accuracy is limited.

[0004] At the same time, deep learning, especially generative adversarial deep learning network (GAN), has the advantages of fast running speed and strong applicability, and is widely used in various fields. However, the GAN network is difficult to apply to ground observation data of different resolutions, especially it is difficult to obtain crop growth period information.

[0005] To address the above issues, there is an urgent need for a crop deep learning extraction algorithm that can be applied to earth observation data of different resolutions, high, medium and low, can utilize the key growth period of crops, and can be implemented on a large scale through engineering. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a deep learning crop extraction method, device and equipment integrating key growth periods, which can not only realize crop extraction by utilizing earth observation data of different resolutions, but also make full use of the growth period of crops to realize large-scale and accurate crop distribution prediction.

[0007] In a first aspect, an embodiment of the present invention provides a deep learning crop extraction method integrating key growth periods, comprising:

[0008] Acquire initial earth observation data corresponding to the study area, and select two periods of target earth observation data from the initial earth observation data according to crop growth distribution characteristics; wherein the crop growth distribution characteristics are used to characterize the key growth period of the crop;

[0009] The crop extraction model obtained through training is used to predict crop distribution based on the two phases of the target earth observation data to generate crop distribution prediction data corresponding to the study area; wherein the crop extraction model adopts a dual-temporal generative adversarial network framework, and the crop extraction model includes a spatiotemporal analysis layer, and the spatiotemporal analysis layer is used to reduce the impact of the resolution of the target earth observation data on the crop distribution prediction data;

[0010] The crop patches contained in the crop distribution prediction data are optimized to generate crop distribution results corresponding to the study area.

[0011] In one embodiment, the step of selecting two periods of target earth observation data from the initial earth observation data according to the crop growth distribution characteristics comprises:

[0012] Determine the key growth period of crops according to the growth distribution characteristics of crops; wherein the key growth period of crops includes the period from sowing to germination, the period from jointing to heading, and the period from maturity to harvest;

[0013] Screening out one period of initial ground observation data corresponding to the sowing to germination period or the maturity to harvest period, and screening out another period of initial ground observation data corresponding to the jointing to heading period;

[0014] The two phases of the screened initial earth observation data are preprocessed to obtain two phases of apparent reflectance data, the two phases of the apparent reflectance data are visually enhanced using a standard deviation stretching algorithm to obtain two phases of visual enhancement data, and the two phases of the visual enhancement data are mosaicked to obtain two phases of target earth observation data.

[0015] In one embodiment, the dual-temporal generative adversarial network framework includes an improved generator and a discriminator, the crop extraction model is the improved generator in the dual-temporal generative adversarial network framework, the improved generator includes multiple stages, and the two phases of target ground observation data are respectively first target ground observation data and second target ground observation data;

[0016] The step of performing crop distribution prediction based on the two phases of target earth observation data using the crop extraction model obtained through training to generate crop distribution prediction data corresponding to the study area includes:

[0017] The improved generator is used to extract the feature vectors corresponding to the first target ground observation data and the feature vectors corresponding to the second target ground observation data respectively, and the data features of different growth periods are extracted according to the feature vectors to generate crop distribution prediction data corresponding to the study area based on the data features.

[0018] In one embodiment, the improved generator includes a multi-stage feature extraction unit, a spatiotemporal analysis layer, an upsampling layer, a multi-layer perceptron, and an activation function layer;

[0019] The step of respectively extracting the feature vector corresponding to the first target earth observation data and the feature vector corresponding to the second target earth observation data by the improved generator, extracting data features of different growth periods according to the feature vectors, and generating crop distribution prediction data corresponding to the study area based on the data features includes:

[0020] Performing feature extraction on the first target earth observation data and the second target earth observation data through the multi-level feature extraction unit to obtain a feature vector output by each level of the feature extraction unit;

[0021] Through the spatiotemporal analysis layer, the feature vectors output by the feature extraction units at each level are subjected to 2D convolution operations, activation function operations, and batch normalization operations to generate data features of different growth periods;

[0022] The crop distribution prediction data corresponding to the study area is generated based on the data features through the multi-layer perceptron and the activation function layer.

[0023] In one embodiment, the multi-level feature extraction units are divided into multi-level first feature extraction units and multi-level second feature extraction units, and each level of the feature extraction unit includes a downsampling layer and a Transformer layer;

[0024] The step of performing feature extraction on the first target earth observation data and the second target earth observation data by the multi-level feature extraction unit to obtain a feature vector output by each level of the feature extraction unit includes:

[0025] For the first feature extraction unit, downsampling the first target earth observation data or the feature vector output by the previous feature extraction unit is performed through the downsampling layer in the first feature extraction unit, and feature extraction is performed on the first target earth observation data after the downsampling process through the Transformer layer in the first feature extraction unit to obtain the feature vector output by the first feature extraction unit;

[0026] And, for the second feature extraction unit, the second target ground observation data or the feature vector output by the previous feature extraction unit is downsampled through the downsampling layer in the second feature extraction unit, and feature extraction is performed on the second target ground observation data after the downsampling through the Transformer layer in the second feature extraction unit to obtain the feature vector output by the second feature extraction unit.

[0027] In one embodiment, before the step of using the crop extraction model obtained through training to perform crop distribution prediction based on the target earth observation data of two periods to generate crop distribution prediction data corresponding to the study area, the method further includes:

[0028] Filter out the two phases of the target earth observation data corresponding to the designated sub-region in the study area, and obtain the crop patches corresponding to the two phases of the filtered target earth observation data;

[0029] Slicing the two phases of the selected target earth observation data and the corresponding crop patches to obtain a slice data set;

[0030] The improved generator and the discriminator are trained respectively using the slice data set, and the trained improved generator is used as a crop extraction model.

[0031] In one embodiment, the step of optimizing the crop patches included in the crop distribution prediction data to generate the crop distribution results corresponding to the study area includes:

[0032] Converting the crop patches included in the crop distribution prediction data into vector patches;

[0033] For each of the vector graphics spots, weighted processing is performed on the current pixel point and the next pixel point in the vector graphics spot with different weights to generate a smooth pixel point between the current pixel point and the next pixel point, and the sawtooth information of the vector graphics spot is removed by using the smooth pixel point to obtain an optimized vector graphics spot;

[0034] Based on each optimized vector map patch, the crop distribution result corresponding to the study area is obtained.

[0035] In a second aspect, an embodiment of the present invention further provides a deep learning crop extraction device integrating key growth periods, comprising:

[0036] An acquisition module is used to acquire initial earth observation data corresponding to the study area, and filter out two periods of target earth observation data from the initial earth observation data according to crop growth distribution characteristics; wherein the crop growth distribution characteristics are used to characterize the key growth period of the crop;

[0037] A prediction module, which is used to perform crop distribution prediction based on the two phases of the target earth observation data using the crop extraction model obtained through training, so as to generate crop distribution prediction data corresponding to the study area; wherein the crop extraction model adopts a dual-temporal generative adversarial network framework, and the crop extraction model includes a spatiotemporal analysis layer, and the spatiotemporal analysis layer is used to reduce the impact of the resolution of the target earth observation data on the crop distribution prediction data;

[0038] The optimization module is used to optimize the crop patches contained in the crop distribution prediction data to generate crop distribution results corresponding to the study area.

[0039] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0040] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0041] The embodiment of the present invention provides a deep learning crop extraction method, device and equipment integrating key growth periods. First, the initial ground observation data corresponding to the study area is obtained, and two phases of target ground observation data are screened out from the initial ground observation data according to the crop growth distribution characteristics. The crop growth distribution characteristics are used to characterize the key growth period of the crop; then, the crop extraction model obtained by training is used to predict the crop distribution based on the two phases of target ground observation data to generate the crop distribution prediction data corresponding to the study area. The crop extraction model adopts a dual-phase generative adversarial network framework. The crop extraction model includes a spatiotemporal analysis layer, which is used to reduce the impact of the resolution of the target ground observation data on the crop distribution prediction data; finally, the crop patches contained in the crop distribution prediction data are optimized to generate the crop distribution results corresponding to the study area. The above method can not only extract crops from ground observation data of different resolutions by constructing a dual-phase generative adversarial network framework integrating key growth periods, but also make full use of the growth period of crops to monitor the crop distribution status over a large area and accurately. It has the advantages of fast speed, high accuracy and easy engineering application, and can provide basic support for food security supervision.

[0042] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic flow chart of a deep learning crop extraction method integrating key growth periods provided in an embodiment of the present invention;

[0046] Figure 2 A flowchart of analyzing the key growth period of crops provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of a key growth period of a crop provided by an embodiment of the present invention;

[0048] Figure 4 A flow chart of earth observation data processing provided by an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the structure of a dual-phase generative adversarial network framework provided by an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of the structure of an improved generator provided by an embodiment of the present invention;

[0051] Figure 7 A schematic diagram of the structure of a transformer layer provided in an embodiment of the present invention;

[0052] Figure 8 A schematic diagram of the structure of a discriminator provided by an embodiment of the present invention;

[0053] Fig. 9 A crop extraction model training flow chart provided by an embodiment of the present invention;

[0054] Fig.10A schematic diagram of the overall process of a deep learning crop extraction method integrating key growth periods provided in an embodiment of the present invention;

[0055] Fig.11 A schematic diagram of a crop distribution prediction result provided by an embodiment of the present invention;

[0056] Fig.12 A schematic diagram of crop distribution results provided by an embodiment of the present invention;

[0057] Fig.13 A schematic diagram of the structure of a deep learning crop extraction device integrating key growth periods provided in an embodiment of the present invention;

[0058] Fig.14 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] At present, there is an urgent need for a crop deep learning extraction algorithm that can be applied to earth observation data of different resolutions, high, medium and low and can utilize the key growth period of crops and can be implemented on a large scale. Based on this, the present invention provides a deep learning crop extraction method, device and equipment integrating key growth periods, which can not only realize crop extraction by utilizing earth observation data of different resolutions, but also make full use of the growth period of crops to realize large-scale and accurate crop distribution prediction.

[0061] To facilitate understanding of this embodiment, a deep learning crop extraction method integrating key growth periods disclosed in an embodiment of the present invention is first described in detail. Figure 1 A schematic flow chart of a deep learning crop extraction method integrating key growth periods is shown, and the method mainly includes the following steps S102 to S106:

[0062] Step S102, obtaining initial earth observation data corresponding to the study area, and selecting two phases of target earth observation data from the initial earth observation data according to crop growth distribution characteristics.

[0063] Among them, the initial earth observation data can be earth observation remote sensing data crop growth distribution characteristics used to characterize the key growth period of crops. The key growth period of crops can include sowing to germination period, jointing to heading period, maturity to harvest period, and the target earth observation data can be understood as earth observation remote sensing data that covers at least two key growth periods after data processing. In one embodiment, the crop growth distribution characteristics are analyzed and detected to extract the key growth period of the crop, so as to preferably obtain two periods of initial earth observation data covering the key period of the crop based on the key growth period of the crop and the period of earth observation remote sensing data acquisition. Furthermore, the two periods of initial earth observation data obtained can also be processed to generate earth observation data products suitable for deep learning, and the earth observation data products are the target earth observation data.

[0064] Step S104, using the crop extraction model obtained through training, crop distribution prediction is performed based on the two-phase target earth observation data to generate crop distribution prediction data corresponding to the study area.

[0065] Among them, the crop extraction model adopts a dual-phase generative adversarial network framework, and the improved generator in the dual-phase generative adversarial network framework can be used as a crop extraction model. The crop extraction model includes a spatiotemporal analysis layer, which is used to reduce the impact of the resolution of the target ground observation data on the crop distribution prediction data. The crop distribution prediction data includes crop patches, and the crop patches are used to characterize the distribution of crops within the corresponding range. In one embodiment, first, the basic structure of DCGAN is referenced, and its network structure is modified to adapt to target ground observation data with different resolutions and different spectral channels, and the time characteristics of the key growth period of crops can be learned, so as to construct a spatiotemporal deep learning network framework integrating the key growth period, which is a dual-phase generative adversarial network framework; then, based on the manually interpreted crop patches and target ground observation data, a slice data set for deep learning training is produced, and the optimal crop extraction model is generated; finally, the crop extraction model is used to predict the two-phase target ground observation data of the entire region to generate crop distribution prediction data.

[0066] Step S106, optimizing the crop patches contained in the crop distribution prediction data to generate crop distribution results corresponding to the study area.

[0067] In one implementation, the Chaikin smoothing algorithm may be used to optimize the crop patches to generate crop distribution results.

[0068] The deep learning crop extraction method integrating key growth periods provided in the embodiment of the present invention, by constructing a dual-phase generative adversarial network framework integrating key growth periods, can not only extract crops using ground observation data of different resolutions, but also make full use of the growth period of crops to accurately monitor the distribution of crops over a large area. It has the advantages of fast speed, high precision, and ease of engineering application, and can provide basic support for food security supervision.

[0069] For ease of understanding, an embodiment of the present invention provides a specific implementation of a deep learning crop extraction method integrating key growth periods.

[0070] For the aforementioned step S102, the embodiment of the present invention provides an implementation method for selecting two periods of target earth observation data from the initial earth observation data according to the crop growth distribution characteristics, see the following steps a to b:

[0071] Step a, determining the key growth period of crops according to the growth distribution characteristics of crops, and selecting one period of initial ground observation data corresponding to the sowing to germination period or the maturity to harvest period, and selecting another period of initial ground observation data corresponding to the jointing to heading period.

[0072] In one example, see Figure 2 A flowchart for analyzing the critical growth period of crops is shown, comprising: performing a critical growth period analysis based on the entire growth period of the crop to extract multiple critical growth periods (including the sowing to germination period Q1, the jointing to heading period Q2, and the maturity to harvest period Q3); and then screening the initial earth observation data for images of the critical growth periods in combination with manually interpreted crop patches to screen out the initial earth observation data M1 and the initial earth observation data M2.

[0073] In practice, crops go through the stages of germination, flowering, fruiting, aging and death in their growth cycle. In crop planting areas, this is reflected in the local agricultural calendar, which includes different crop growth periods such as tillage, sowing, germination, growth, maturity and harvesting. The growth status of crops in different growth periods can be reflected based on earth observation remote sensing data, which is the basic theoretical basis for extracting crops from earth observation data.

[0074] According to the distribution characteristics of the entire growth cycle of crops and the difficulty of obtaining earth observation remote sensing data, three key growth periods of crops are extracted first, namely, sowing to germination period Q1, jointing to heading period Q2, and maturity to harvest period Q3. Figure 3 The diagram shows a schematic diagram of the key growth period of a crop, wherein the crop coverage is low from sowing to germination period Q1; the crop coverage is highest from jointing to heading period Q2, and the growth is vigorous; from maturity to harvest period Q3, the crop coverage decreases again as the crop withers and dies.

[0075] According to the determined time from sowing to germination period Q1, jointing to heading period Q2, and maturity to harvest period Q3, combined with the manually interpreted crop patches, the initial ground observation data in the key growth period for model training is screened. In order to reduce the difficulty of data acquisition and improve the engineering level, according to the actual situation of the data, one of the sowing to germination period Q1 and maturity to harvest period Q3 is selected. The initial ground observation data screened from sowing to germination period Q1 or maturity to harvest period Q3 is recorded as M1, and the initial ground observation data within the range of jointing to heading period Q2 is recorded as M2. The initial ground observation data M1 and the initial ground observation data M2 can be homologous data or heterologous data, which further improves the applicability of the algorithm. The initial ground observation data M1 and the initial ground observation data M2 have the same number of bands, and the number of bands is recorded as N.

[0076] Step b: preprocess the two selected initial earth observation data to obtain two periods of apparent reflectance data, use the standard deviation stretching algorithm to perform visual enhancement processing on the two periods of apparent reflectance data to obtain two periods of visual enhancement data, and mosaic the two periods of visual enhancement data to obtain two periods of target earth observation data.

[0077] In one example, see Figure 4 A flow chart of earth observation data processing is shown: geometric correction processing, apparent reflectance calculation, resampling processing, visual enhancement processing, and mosaic processing are performed on initial earth observation data M1 and initial earth observation data M2 to obtain target earth observation data M3 and target earth observation data M4.

[0078] In the specific implementation, since the target ground observation data corresponding to the two key growth periods are prioritized, there is no need to perform complex data processing such as time series analysis, time reconstruction, and filtering and denoising on the time series data, and it also has the key time information of the crop growth, so the algorithm operation speed and accuracy can be improved. At the same time, since the crop extraction model is derived from computer vision, it is necessary to perform visual enhancement processing on the target ground observation data to further improve the quality of deep learning extraction. The specific process is as follows:

[0079] The geometry, apparent reflectance calculation and resampling processing of the two initial earth observation data were performed to generate apparent reflectance data products. Based on the apparent reflectance data, the standard deviation stretching processing algorithm was used to visually enhance the apparent reflectance data. The visual enhancement formula is as follows:

[0080]

[0081] ρ max =μ+n*σ;

[0082] ρ min =μ-n*σ;

[0083] Among them, v is the pixel value of visual enhancement, ρ is the pixel reflectance value of a certain band of apparent reflectance data, μ and σ are the mean and standard deviation of the band, n is the multiple of the standard deviation, ρ is the pixel reflectance value of a certain band of apparent reflectance data, μ is the mean and standard deviation of the band, n is the multiple of the standard deviation, and ρ is the pixel reflectance value of a certain band of apparent reflectance data. max is the maximum value, ρ min The two-phase apparent reflectance data generate visual enhancement data, and the visual enhancement data are mosaicked and recorded as target earth observation data M3 and target earth observation data M4. The target earth observation data M3 and target earth observation data M4 have the same spatial resolution and the same band. The target earth observation data M3 and target earth observation data M4 are in the threshold range of 0-255.

[0084] Before explaining the aforementioned step S104, a specific implementation of a dual-temporal generative adversarial network framework is first provided. Using the basic structure of DCGAN (Deep Convolution Generative Adversarial Networks), its network structure is modified so that it can adapt to earth observation data of different resolutions and different spectral channels, and can learn the time characteristics of the key growth period of crops, so as to improve the adaptability and performance of crop extraction based on earth observation data. The modified and optimized crop extraction GAN network for the key growth period is denoted as a dual-temporal generative adversarial network framework (bitemporal-GAN, BGAN). BGAN makes the following modifications and optimizations in four aspects relative to DCGAN, namely:

[0085] 1) Remove the noise input of the generator and replace it with earth observation data of another time phase;

[0086] 2) Changing the generator from a CNN-based network structure to a transformer-based network structure to reduce the impact of the resolution of earth observation data on feature extraction;

[0087] 3) A new spatiotemporal analysis layer is added to learn the characteristics of crops at different growth stages and further reduce the impact of different resolutions;

[0088] 4) Optimize the number of bands input to the generator so that multispectral data can be input to accommodate earth observation data from different spectral channels.

[0089] See also Figure 5A structural schematic diagram of a dual-temporal generative adversarial network framework is shown, which includes an improved generator G and a discriminator D, T1 and T2 are slice data generated by target ground observation data M3 and target ground observation data M4, fake is a pseudo label output by the improved generator (that is, crop distribution prediction data), real is a real label generated by the crop interpretation map, and the generator includes a transformer-based encoder and decoder.

[0090] In one example, the improved generator includes a multi-level feature extraction unit, a spatiotemporal analysis layer, an upsampling layer, a multi-layer perceptron and an activation function layer (tanh). The multi-level feature extraction unit is divided into a multi-level first feature extraction unit and a multi-level second feature extraction unit. The first feature extraction layer is used to extract the feature vector of the slice data T1, and the second feature extraction layer is used to extract the feature vector of the slice data T2. Each level of the feature extraction unit includes a downsampling layer and a Transformer layer.

[0091] In a specific embodiment, see Figure 6 A schematic diagram of the structure of an improved generator is shown in FIG. Figure 6 The generator is shown to consist of a downsampling layer, a transformer layer, a spatial-temporal analysis layer, an upsampling layer, a multi-layer perception layer (MLP) and tanh. The input data of the generator is the slice data of the two-phase key generator, the size of which is data_size*data_size*N, and the output is a pseudo label of data_size*data_size (that is, crop distribution prediction data).

[0092] In one example, the downsampling layer is composed of a 3*3 convolutional layer Conv2D, with a convolution kernel of 7 and a stride of 4. The input of this layer is of size data_size / (2i+1)*data_size / (2i+1)*N1, and the output is data_size / (2i+2)*data_size / (2i+2)*(2*N1), where i is the i-th layer number.

[0093] In one example, see Figure 7 The figure shows a schematic diagram of the structure of a transformer layer. The transformer layer consists of a sequence distribution layer (sequence), a self-attention mechanism layer (self-attention), an MPL and a convolution layer (convolution). The input and output data sizes of this layer remain unchanged.

[0094] In one example, the spatiotemporal analysis layer mainly extracts data features of different growth periods to characterize data differences in different periods. The structure is as follows:

[0095] F=BN(ReLU(Con2D(Cat(T1,T2))));

[0096] It is composed of Convolutional2D, ReLU and Batch Normalization (BN). The length and width of the input and output data of this layer remain unchanged, and the number of bands is halved.

[0097] In one example, the upsampling layer is composed of a 3*3 convolutional layer DeConv2D, with a convolution kernel of 7 and a stride of 4. The input of this layer is of size data_size / (2i+2)*data_size / (2i+2)*N2, and the output is data_size / (2i+1)*data_size / (2i+1)*(N2 / 2), where i is the i-th layer number.

[0098] In one example, the MLP consists of three parts: a downsampling layer, a connection layer, and an upsampling layer. The upsampling layer uses the above network structure, and the connection layer is as follows:

[0099] F=Linear(Cat(F1,F2)).

[0100] In a specific embodiment, see Figure 8 The schematic diagram of the structure of a discriminator shown in FIG. 1 includes conv+ReLU layer, conv+BN+ReLU layer, concatenation layer, and sigmoid layer. The input data of the discriminator is the pseudo-classification data output by the generator and the real classification data, and the output is the similarity probability of the real and fake scene data. The discriminator is mainly composed of the encoding subnetwork, and the specific structure is shown in Table 1:

[0101] Table 1

[0102] Network layer structure Step Length Output size Input data_size*data_size*2 Conv+ReLU 2 data_size*data_size*1 Conv+BN+ReLU 2 data_size*data_size*1 Sigmoid 1 data_size*data_size*1 output 1

[0103] Among them, Conv stands for Convolutional and BN stands for Batch Normalization.

[0104] Based on the above network framework structure, the embodiment of the present invention also provides a model training method, see the following (1) to (2):

[0105] (1) Filter out the two-period target earth observation data corresponding to the specified sub-area in the study area, obtain the crop patches corresponding to the filtered two-period target earth observation data, and slice the filtered two-period target earth observation data and their corresponding crop patches to obtain a slice data set.

[0106] The designated sub-region may also be referred to as a sample region range. In one embodiment, using the target ground observation data M3 and the target ground observation data M4 and the manual interpretation vector of the same region, a sample set for BGAN network framework training is prepared within the sample region range to generate a BGAN deep learning model (BGAN_model) suitable for crop extraction.

[0107] For details, see Fig. 9 A crop extraction model training flowchart is shown, which performs slice processing on the target ground observation data M3 and the target ground observation data M4, as well as the manually interpreted crop patches, to obtain a slice data set; then, hyperparameters are set for the BGAN network framework, and network training and verification are performed to obtain a BGAN deep learning model (BGAN_model).

[0108] In the specific implementation, the target earth observation data M3 and the target earth observation data M4 are sliced ​​by using the manual interpretation vector map. Each slice sample contains three data, namely, the slice of the target earth observation data M3, the slice of the target earth observation data M4 and the corresponding annotation data. A total of m slice samples are sliced ​​to construct a slice data set E1. The m slice data sets are randomly split into training sets, validation sets and test sets according to the ratio of 0.6, 0.2, and 0.2. The training set is used to train the deep learning model, and the validation set and test set are used to verify the accuracy and stability of the model.

[0109] Set the hyperparameters of the BGAN network framework, namely, slice size data_size, batch training sample number batch_num, generator learning rate G_rate, discriminator generator D_rate, and training round epoch_num.

[0110] After epoch_num training, the model with the smallest loss and the highest accuracy is selected and recorded as BGAN_model.

[0111] (2) The improved generator and discriminator are trained separately using the sliced ​​dataset, and the trained improved generator is used as the crop extraction model. In one example, the improved generator in BGAN_model is the crop extraction model.

[0112] After the crop extraction model is trained, the crop extraction model can be used to predict the crop distribution in the study area. First, the two-period target ground observation data are recorded as the first target ground observation data M3 and the second target ground observation data M4, respectively. Then, the feature vector corresponding to the first target ground observation data M3 and the feature vector corresponding to the second target ground observation data M4 are extracted by the improved generator, and the data features of different growth periods are extracted according to the feature vectors, so as to generate the crop distribution prediction data corresponding to the study area based on the data features. For details, please refer to the following steps 1 to 3:

[0113] Step 1: Perform feature extraction on the first target earth observation data and the second target earth observation data through a multi-level feature extraction unit to obtain a feature vector output by each level of feature extraction unit.

[0114] In one example, for the first feature extraction unit, the first target earth observation data or the feature vector output by the previous feature extraction unit is downsampled through the downsampling layer in the first feature extraction unit, and the first target earth observation data after the downsampling is subjected to feature extraction through the Transformer layer in the first feature extraction unit to obtain the feature vector output by the first feature extraction unit;

[0115] In one example, for the second feature extraction unit, the second target ground observation data or the feature vector output by the previous feature extraction unit is downsampled through the downsampling layer in the second feature extraction unit, and feature extraction is performed on the downsampled second target ground observation data through the Transformer layer in the second feature extraction unit to obtain the feature vector output by the second feature extraction unit.

[0116] Please continue to see Figure 6 , Figure 6 The improved generator includes four first feature extraction units and second feature extraction units. Taking the first feature extraction unit as an example, the first downsampling layer downsamples the slice data T1, and inputs the downsampled data into the first Transformer layer, which outputs the corresponding feature vector; similarly, the second downsampling layer downsamples the feature vector output by the first Transformer layer, and inputs the downsampled data into the second Transformer layer, which outputs the corresponding feature vector, and repeats this process until the last Transformer layer outputs the feature vector.

[0117] Step 2: Through the spatiotemporal analysis layer, the feature vectors output by each level of feature extraction unit are subjected to 2D convolution operations, activation function operations, and batch normalization operations to generate data features of different growth periods.

[0118] Please continue to see Figure 6 The input of the spatiotemporal analysis layer includes the feature vectors output by the Transformer layer in the four first feature extraction units and the feature vectors output by the Transformer layer in the four second feature extraction units, and the output is the data features of different growth periods. In the specific implementation, the 2D convolution operation is performed through the Convolutional2D layer, the activation function operation is performed through the ReLU layer, and the batch normalization operation is performed through the Batch Normalization layer (BN).

[0119] Step 3: Generate crop distribution prediction data corresponding to the study area based on data features through multi-layer perceptron and activation function layer.

[0120] In the specific implementation, the entire area of ​​the target earth observation data M3 and the target earth observation data M4 is sliced, and then the slice data is predicted using the crop extraction model to generate predicted data of multiple slices, and then the predicted slice data is mosaicked to finally obtain the crop distribution prediction data M5 for the study area.

[0121] For the aforementioned step S106, the embodiment of the present invention provides an implementation method for optimizing the crop patches contained in the crop distribution prediction data to generate crop distribution results corresponding to the study area, see the following steps 1 to 3:

[0122] Step 1: Convert the crop patches contained in the crop distribution prediction data into vector patches.

[0123] In one example, the crop distribution prediction data M5 may be processed by raster-to-raster conversion to generate vector patches.

[0124] Step 2: For each vector graph spot, the current pixel and the next pixel in the vector graph spot are weighted with different weights to generate a smooth pixel between the current pixel and the next pixel, and the smooth pixel is used to remove the jagged information of the vector graph spot to obtain an optimized vector graph spot.

[0125] In one example, since raster data is composed of pixels, the process of generating vectors will inevitably cause jagged noise. In order to remove the jagged noise, the Chaikin smoothing algorithm is used to remove it. The process of Chaikin processing jagged spots is as follows:

[0126] Given a patch, the patch consists of n points:

[0127] {P0,P1,…,P n};

[0128] The patch boundaries are refined by generating new points:

[0129] {Q0,R0,Q1,R1,…,Q n-1 ,R n-1};

[0130]

[0131]

[0132] The jagged information of the image spot is removed by generating 2n new points. i is the current pixel, P i+1 is the next pixel, Q i , R i It is the smooth pixel between the current pixel and the next pixel.

[0133] Step three: Based on each optimized vector map patch, obtain the crop distribution results corresponding to the study area.

[0134] In one example, each vector image patch is processed by Chaikin smoothing algorithm to finally generate a crop distribution result S.

[0135] For ease of understanding, the present invention also provides an application example of a deep learning crop extraction method integrating key growth periods, see Fig.10 The overall process diagram of a deep learning crop extraction method integrating key growth periods is shown in the figure. First, the growth distribution characteristics of the monitored crops are analyzed and the key growth periods of the crops are extracted. Based on the key growth periods of the crops and in combination with the period of obtaining earth observation data, the initial earth observation remote sensing data covering the key growth periods of the crops are prioritized for two periods; the acquired initial earth observation remote sensing data are processed to generate target earth observation remote sensing data suitable for deep learning; the basic structure of DCGAN is referenced, and its network structure is modified to adapt to earth observation remote sensing data of different resolutions and different spectral channels, and the time characteristics of the key growth periods of crops can be learned, so as to construct a spatiotemporal deep learning network framework integrating key growth periods (i.e., a dual-phase generative adversarial network framework); based on the manually interpreted samples and target earth observation remote sensing data, a slice data set for deep learning training is prepared, and model training and verification are performed to generate the optimal crop extraction model; the crop extraction model is used to predict the two-period target earth observation data of the entire region to generate crop distribution prediction data; finally, the Chaikin smoothing algorithm is used to optimize the crop patches to generate crop distribution results.

[0136] Exemplary:

[0137] (1) Analysis of key growth periods of crops:

[0138] Winter wheat crop extraction objects were selected, and Henan Province, the main winter wheat producing area, was selected as the monitoring area. The three key growth periods of winter wheat in Henan Province, Q1, Q2 and Q3, are the jointing period in early April, the heading period in early to mid-May, and the maturity period in mid-to-late June. Sentinel2 was selected as the data source based on the actual situation. Since there is more rainfall in the north in mid-to-late June, the earth observation remote sensing data is not collected, so the sentinel2 data of Q1 and Q2 are used.

[0139] (2) Earth observation data processing:

[0140] The sentinel2 obtained in the Q1 and Q2 periods of Henan Province is processed with geometry, apparent reflectance calculation, resampling, and visual enhancement to generate two periods of data that are the same as deep learning training and prediction. They are denoted as M3 and M4. Since Henan Province is large, M3 and M4 are stored by city and county.

[0141] (3) Construction of deep learning network framework:

[0142] Because each period of sentinel2 has 8 bands, the number of bands of the BGAN input layer is set to 8 and data_size is set to 256.

[0143] (4) Deep learning model training:

[0144] Using M3 and M4 data and the manually interpreted vectors of samples distributed in Zhengzhou, Shangqiu, Zhumadian and Xinyang, a sample set for BGAN network framework training was created within the sample area. A total of 3054 patches were sliced ​​according to 256*256 to generate 5890 slice sample sets. 3534, 1178 and 1178 samples were randomly selected to construct the training set, validation set and test set respectively.

[0145] The hyperparameter batch training sample number batch_num of BGAN is 64, the generator learning rate G_rate is 0.001, the discriminator generator D_rate is 0.02, and the training round is epoch_num is 500. After 500 trainings, the training model accuracy is 94%, and the optimal model is BGAN_model.

[0146] (5) Crop distribution prediction:

[0147] Slice the entire area of ​​M3 and M4, and then use BGAN_model to predict the slice data to generate prediction data of multiple slices. Then mosaic the predicted slice data to finally obtain the crop distribution prediction result M5 of the monitoring area, such as Fig.11 A schematic diagram of crop distribution prediction results is shown.

[0148] (6) Result patch optimization:

[0149] M5 is processed by raster-to-raster conversion to generate vector patches, and Chaikin smoothing algorithm is used to remove sawtooth noise to obtain crop distribution results, such as Fig.12 A schematic diagram of crop distribution results is shown.

[0150] To sum up, the deep learning crop extraction method integrating key growth periods provided in the embodiments of the present invention is suitable for the extraction of major food crops such as wheat, corn, and rice. By constructing a spatiotemporal deep learning network framework integrating key growth periods, it is possible to not only extract crops using ground observation data of different resolutions and spectral channels, but also to make full use of the growth period of crops. It is possible to monitor the distribution of crops over a large area and accurately, has the advantages of fast speed, high precision, and ease of engineering application, and can provide basic support for food security supervision.

[0151] Based on the above embodiments, the present invention provides a deep learning crop extraction device integrating key growth periods, see Fig.13 A schematic diagram of the structure of a deep learning crop extraction device integrating key growth periods is shown, and the device mainly includes the following parts:

[0152] The acquisition module 1302 is used to acquire the initial earth observation data corresponding to the study area, and filter out two periods of target earth observation data from the initial earth observation data according to the crop growth distribution characteristics; wherein the crop growth distribution characteristics are used to characterize the key growth period of the crop;

[0153] A prediction module 1304 is used to perform crop distribution prediction based on two-phase target earth observation data using the crop extraction model obtained through training, so as to generate crop distribution prediction data corresponding to the study area; wherein the crop extraction model adopts a dual-temporal generative adversarial network framework, and the crop extraction model includes a spatiotemporal analysis layer, which is used to reduce the impact of the resolution of the target earth observation data on the crop distribution prediction data;

[0154] The optimization module 1306 is used to optimize the crop patches contained in the crop distribution prediction data to generate crop distribution results corresponding to the study area.

[0155] The deep learning crop extraction device integrating key growth periods provided in the embodiment of the present invention, by constructing a dual-phase generative adversarial network framework integrating key growth periods, can not only extract crops using ground observation data of different resolutions, but also make full use of the growth period of crops to accurately monitor the distribution of crops over a large area. It has the advantages of fast speed, high precision and ease of engineering application, and can provide basic support for food security supervision.

[0156] In one implementation, the acquisition module 1302 is further configured to:

[0157] Determine the key growth period of crops based on the growth distribution characteristics of crops; the key growth period of crops includes the period from sowing to germination, from jointing to heading, and from maturity to harvest;

[0158] Filter out one period of initial ground observation data corresponding to the sowing to germination period or the maturity to harvest period, and filter out another period of initial ground observation data corresponding to the jointing to heading period;

[0159] The two phases of initial earth observation data selected are preprocessed to obtain two phases of apparent reflectance data, the two phases of apparent reflectance data are visually enhanced using the standard deviation stretching algorithm to obtain two phases of visual enhancement data, and the two phases of visual enhancement data are mosaicked to obtain two phases of target earth observation data.

[0160] In one embodiment, the dual-temporal generative adversarial network framework includes an improved generator and a discriminator, the crop extraction model is the improved generator in the dual-temporal generative adversarial network framework, the improved generator includes multiple levels, and the two-phase target ground observation data are respectively the first target ground observation data and the second target ground observation data; the prediction module 1304 is also used for:

[0161] By improving the generator, the feature vectors corresponding to the first target ground observation data and the feature vectors corresponding to the second target ground observation data are extracted respectively, and the data features of different growth periods are extracted according to the feature vectors, so as to generate the crop distribution prediction data corresponding to the study area based on the data features.

[0162] In one embodiment, the improved generator includes a multi-level feature extraction unit, a spatiotemporal analysis layer, an upsampling layer, a multi-layer perceptron, and an activation function layer; the prediction module 1304 is also used to:

[0163] Performing feature extraction on the first target earth observation data and the second target earth observation data through a multi-level feature extraction unit to obtain a feature vector output by each level of feature extraction unit;

[0164] Through the spatiotemporal analysis layer, the feature vectors output by each level of feature extraction unit are subjected to 2D convolution, activation function and batch normalization operations to generate data features of different growth periods.

[0165] Through multi-layer perceptron and activation function layer, crop distribution prediction data corresponding to the study area is generated based on data characteristics.

[0166] In one implementation, the multi-level feature extraction unit is divided into a multi-level first feature extraction unit and a multi-level second feature extraction unit, and each level of the feature extraction unit includes a downsampling layer and a Transformer layer; the prediction module 1304 is also used for:

[0167] For the first feature extraction unit, downsampling the first target earth observation data or the feature vector output by the previous feature extraction unit is performed through the downsampling layer in the first feature extraction unit, and feature extraction is performed on the downsampled first target earth observation data through the Transformer layer in the first feature extraction unit to obtain the feature vector output by the first feature extraction unit;

[0168] And, for the second feature extraction unit, the second target ground observation data or the feature vector output by the previous feature extraction unit is downsampled through the downsampling layer in the second feature extraction unit, and feature extraction is performed on the downsampled second target ground observation data through the Transformer layer in the second feature extraction unit to obtain the feature vector output by the second feature extraction unit.

[0169] In one embodiment, a training module is further included for:

[0170] Filter out the two-period target earth observation data corresponding to the specified sub-area in the study area, and obtain the crop patches corresponding to the filtered two-period target earth observation data;

[0171] The selected two-phase target earth observation data and their corresponding crop patches are sliced ​​to obtain a slice data set;

[0172] The improved generator and discriminator are trained separately using the sliced ​​dataset, and the trained improved generator is used as the crop extraction model.

[0173] In one implementation, the optimization module 1306 is further configured to:

[0174] Convert the crop patches contained in the crop distribution prediction data into vector patches;

[0175] For each vector spot, the current pixel and the next pixel in the vector spot are weighted with different weights to generate a smooth pixel between the current pixel and the next pixel, and the sawtooth information of the vector spot is removed by the smooth pixel to obtain an optimized vector spot;

[0176] Based on each optimized vector map patch, the crop distribution results corresponding to the study area are obtained.

[0177] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0178] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0179] Fig.14 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 140, a memory 141, a bus 142 and a communication interface 143, wherein the processor 140, the communication interface 143 and the memory 141 are connected via the bus 142; the processor 140 is used to execute an executable module stored in the memory 141, such as a computer program.

[0180] The memory 141 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 143 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0181] The bus 142 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.14 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0182] Among them, the memory 141 is used to store programs, and the processor 140 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any of the embodiments of the present invention can be applied to the processor 140 or implemented by the processor 140.

[0183] The processor 140 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 140. The above processor 140 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 141, and the processor 140 reads the information in the memory 141 and completes the steps of the above method in combination with its hardware.

[0184] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0185] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0186] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A deep learning crop extraction method integrating key growth periods, characterized in that: include: Acquire initial earth observation data corresponding to the study area, and select two periods of target earth observation data from the initial earth observation data according to crop growth distribution characteristics; wherein the crop growth distribution characteristics are used to characterize the key growth period of the crop; The crop extraction model obtained through training is used to predict crop distribution based on the two phases of the target earth observation data to generate crop distribution prediction data corresponding to the study area; wherein the crop extraction model adopts a dual-temporal generative adversarial network framework, and the crop extraction model includes a spatiotemporal analysis layer, and the spatiotemporal analysis layer is used to reduce the impact of the resolution of the target earth observation data on the crop distribution prediction data; Optimizing the crop patches contained in the crop distribution prediction data to generate crop distribution results corresponding to the study area; The step of selecting two periods of target earth observation data from the initial earth observation data according to the crop growth distribution characteristics comprises: determining the key growth period of the crop according to the crop growth distribution characteristics; wherein the key growth period of the crop includes the sowing to germination period, the jointing to heading period, and the maturity to harvest period; selecting one period of initial earth observation data corresponding to the sowing to germination period or the maturity to harvest period, and selecting another period of initial earth observation data corresponding to the jointing to heading period; preprocessing the selected two periods of the initial earth observation data to obtain two periods of apparent reflectance data, performing visual enhancement processing on the two periods of the apparent reflectance data using a standard deviation stretching processing algorithm to obtain two periods of visual enhancement data, and performing mosaic processing on the two periods of the visual enhancement data to obtain two periods of target earth observation data; The dual-temporal generative adversarial network framework includes an improved generator and a discriminator, the crop extraction model is the improved generator in the dual-temporal generative adversarial network framework, the improved generator includes multiple levels, and the two phases of target ground observation data are respectively the first target ground observation data and the second target ground observation data; the crop extraction model obtained by training performs crop distribution prediction based on the two phases of target ground observation data to generate the crop distribution prediction data corresponding to the study area, comprising: extracting the feature vector corresponding to the first target ground observation data and the feature vector corresponding to the second target ground observation data respectively by the improved generator, extracting data features of different growth periods according to the feature vectors, so as to generate the crop distribution prediction data corresponding to the study area based on the data features; The improved generator includes a multi-level feature extraction unit, a spatiotemporal analysis layer, an upsampling layer, a multi-layer perceptron and an activation function layer; the step of extracting the feature vector corresponding to the first target earth observation data and the feature vector corresponding to the second target earth observation data respectively through the improved generator, extracting data features of different growth periods according to the feature vectors, and generating crop distribution prediction data corresponding to the study area based on the data features includes: performing feature extraction on the first target earth observation data and the second target earth observation data through the multi-level feature extraction unit to obtain feature vectors output by the feature extraction unit at each level; performing 2D convolution operation, activation function operation and batch normalization operation on the feature vector output by the feature extraction unit at each level through the spatiotemporal analysis layer to generate data features of different growth periods; generating crop distribution prediction data corresponding to the study area based on the data features through the multi-layer perceptron and the activation function layer; The multi-level feature extraction units are divided into multi-level first feature extraction units and multi-level second feature extraction units, and each level of the feature extraction units includes a downsampling layer and a Transformer layer; the step of performing feature extraction on the first target earth observation data and the second target earth observation data through the multi-level feature extraction units to obtain a feature vector output by each level of the feature extraction units includes: for the first feature extraction unit, performing downsampling processing on the first target earth observation data or the feature vector output by the previous level feature extraction unit through the downsampling layer in the first feature extraction unit, and performing feature extraction on the first target earth observation data after the downsampling processing through the Transformer layer in the first feature extraction unit to obtain the feature vector output by the first feature extraction unit; and, for the second feature extraction unit, performing downsampling processing on the second target earth observation data or the feature vector output by the previous level feature extraction unit through the downsampling layer in the second feature extraction unit, and performing feature extraction on the second target earth observation data after the downsampling processing through the Transformer layer in the second feature extraction unit to obtain the feature vector output by the second feature extraction unit.

2. The deep learning crop extraction method integrating key growth periods according to claim 1 is characterized in that: Before the step of using the crop extraction model obtained through training to predict crop distribution based on the target earth observation data of two periods to generate crop distribution prediction data corresponding to the study area, the method further includes: Filter out the two phases of the target earth observation data corresponding to the designated sub-region in the study area, and obtain the crop patches corresponding to the two phases of the filtered target earth observation data; Slicing the two phases of the selected target earth observation data and the corresponding crop patches to obtain a slice data set; The improved generator and the discriminator are trained respectively using the slice data set, and the trained improved generator is used as a crop extraction model.

3. The deep learning crop extraction method integrating key growth periods according to claim 1 is characterized in that: The step of optimizing the crop patches contained in the crop distribution prediction data to generate the crop distribution results corresponding to the study area includes: Converting the crop patches included in the crop distribution prediction data into vector patches; For each of the vector graphics spots, weighted processing is performed on the current pixel point and the next pixel point in the vector graphics spot with different weights to generate a smooth pixel point between the current pixel point and the next pixel point, and the sawtooth information of the vector graphics spot is removed by using the smooth pixel point to obtain an optimized vector graphics spot; Based on each optimized vector map patch, the crop distribution result corresponding to the study area is obtained.

4. A deep learning crop extraction device integrating key growth periods, characterized in that: include: An acquisition module is used to acquire initial earth observation data corresponding to the study area, and filter out two periods of target earth observation data from the initial earth observation data according to crop growth distribution characteristics; wherein the crop growth distribution characteristics are used to characterize the key growth period of the crop; A prediction module, which is used to perform crop distribution prediction based on the two phases of the target earth observation data using the crop extraction model obtained through training, so as to generate crop distribution prediction data corresponding to the study area; wherein the crop extraction model adopts a dual-temporal generative adversarial network framework, and the crop extraction model includes a spatiotemporal analysis layer, and the spatiotemporal analysis layer is used to reduce the impact of the resolution of the target earth observation data on the crop distribution prediction data; An optimization module, used for optimizing the crop patches contained in the crop distribution prediction data to generate a crop distribution result corresponding to the study area; The acquisition module is specifically used to: determine the key growth period of the crop according to the growth distribution characteristics of the crop; wherein the key growth period of the crop includes the sowing to germination period, the jointing to heading period, and the maturity to harvest period; screen out the initial ground observation data of one period corresponding to the sowing to germination period or the maturity to harvest period, and screen out the initial ground observation data of another period corresponding to the jointing to heading period; pre-process the screened two periods of the initial ground observation data to obtain two periods of apparent reflectance data, use the standard deviation stretching processing algorithm to perform visual enhancement processing on the two periods of the apparent reflectance data to obtain two periods of visual enhancement data, and perform mosaic processing on the two periods of the visual enhancement data to obtain two periods of target ground observation data; The dual-temporal generative adversarial network framework includes an improved generator and a discriminator, the crop extraction model is the improved generator in the dual-temporal generative adversarial network framework, the improved generator includes multiple stages, and the two phases of target ground observation data are respectively the first target ground observation data and the second target ground observation data; the prediction module is specifically used to: respectively extract the feature vector corresponding to the first target ground observation data and the feature vector corresponding to the second target ground observation data through the improved generator, extract data features of different growth periods according to the feature vectors, so as to generate crop distribution prediction data corresponding to the study area based on the data features; The improved generator includes a multi-level feature extraction unit, a spatiotemporal analysis layer, an upsampling layer, a multi-layer perceptron and an activation function layer; the prediction module is specifically used to: perform feature extraction on the first target ground observation data and the second target ground observation data through the multi-level feature extraction unit to obtain feature vectors output by the feature extraction unit at each level; perform 2D convolution operation, activation function operation and batch normalization operation on the feature vectors output by the feature extraction unit at each level through the spatiotemporal analysis layer to generate data features of different growth periods; generate crop distribution prediction data corresponding to the study area based on the data features through the multi-layer perceptron and the activation function layer; The multi-level feature extraction units are divided into multi-level first feature extraction units and multi-level second feature extraction units, and each level of the feature extraction units includes a downsampling layer and a Transformer layer; the prediction module is specifically used for: for the first feature extraction unit, downsampling the first target ground observation data or the feature vector output by the previous level feature extraction unit through the downsampling layer in the first feature extraction unit, and performing feature extraction on the first target ground observation data after the downsampling process through the Transformer layer in the first feature extraction unit to obtain the feature vector output by the first feature extraction unit; and, for the second feature extraction unit, downsampling the second target ground observation data or the feature vector output by the previous level feature extraction unit through the downsampling layer in the second feature extraction unit, and performing feature extraction on the second target ground observation data after the downsampling process through the Transformer layer in the second feature extraction unit to obtain the feature vector output by the second feature extraction unit.

5. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 3.

Images