Crop early season classification method and system based on multi-task cascade learning, and medium

Abstract

The invention discloses a crop early season classification method and system based on multi-task cascade learning and a medium, and the method comprises the steps: obtaining remote sensing time sequence data of early season crops, so as to construct multi-source remote sensing time sequence data; dividing a training set, a verification set and a test set according to the multi-source remote sensing time sequence data; constructing a mixed deep learning model based on multi-task cascade learning; training the mixed deep learning model by using the training set and the verification set to obtain a crop classification model; testing the crop classification model by using the test set to obtain a classification result; according to the confidence level of the classification result, dynamically adjusting the early season classification time point of the crop; wherein the hybrid deep learning model comprises a prediction module and a classification module, the prediction module takes a long short-term memory network as a core and combines an encoder-decoder structure and an attention mechanism, the classification module is a convolutional neural network, and the two modules are associated through a cascade learning mechanism. According to the invention, the accuracy of crop early season classification is obviously improved.

Description

Technical Field

[0001] The present invention relates to an early-season crop classification method, system and medium based on multi-task cascade learning, and belongs to the field of time series analysis in the deep learning branch of artificial intelligence. Background Art

[0002] Against the backdrop of growing food demand, timely and accurate crop classification is crucial for ensuring food security. Accurate early-season crop classification is crucial for agricultural applications and government decision-making. It not only provides reliable spatial information for risk management and crop insurance, but also enables precise intervention and effective management during the early stages of crop growth. Therefore, early-season crop classification is a crucial research topic.

[0003] There are three main approaches to early-season crop classification. The first is based on phenological knowledge, analyzing crop characteristics at different growth stages. Wang Jie analyzed time series of multiple vegetation indices to classify sugarcane based on its unique phenological characteristics, such as greening and senescence. The second approach is based on machine learning methods, such as classic methods like random forests (RF) and support vector machines (SVM). You et al. used a random forest classifier for early-season classification and concluded that the earliest identifiable times for rice, corn, and soybean are at transplanting, heading, and pod setting, respectively. The third approach is based on deep learning methods, such as the Transformer model, convolutional neural networks (CNN), and recurrent neural networks (RNN). Zhang et al. combined temporal random masking with spatial information from pixel sets to enhance the diversity of training data. They integrated a pixel aggregate encoder (PSAE) with an LSTM module to effectively aggregate time series information and better capture the temporal characteristics of crop growth. Mao et al. proposed a Dual-1DCNN model based on convolutional neural network (CNN) to effectively capture local features in time series and classify cotton and wheat.

[0004] Deep learning-based methods have made some progress by leveraging existing feature extraction techniques to effectively capture global and local information in time series. However, when classification is performed earlier, available temporal data becomes more limited. These methods struggle to understand contextual information from this limited data, resulting in insufficient extraction of key phenological information, ultimately preventing the model from achieving good classification results. Therefore, a method that effectively addresses this lack of key information extraction is crucial. Summary of the Invention

[0005] In view of this, the present invention provides an early season crop classification method, system, computer device and storage medium based on multi-task cascade learning, which constructs a prediction module based on long short-term memory network and a classification model based on convolutional neural network, and combines them into a hybrid deep learning model. The two modules are interconnected through cascade learning. By using early season time series data to predict future values, more key phenological information can be extracted from the complete time series data, thereby improving the classification accuracy. At the same time, when the classification performance is improved, the prediction accuracy can also be reversely optimized.

[0006] The first object of the present invention is to provide an early season crop classification method based on multi-task cascade learning.

[0007] The second object of the present invention is to provide an early season crop classification system based on multi-task cascade learning.

[0008] A third object of the present invention is to provide a computer device.

[0009] A fourth object of the present invention is to provide a computer-readable storage medium.

[0010] The first object of the present invention can be achieved by adopting the following technical solutions:

[0011] A method for early-season crop classification based on multi-task cascade learning, comprising:

[0012] Acquire remote sensing time series data of early season crops to construct multi-source remote sensing time series data;

[0013] Based on multi-source remote sensing time series data, the training set, validation set and test set are divided;

[0014] Build a hybrid deep learning model based on multi-task cascade learning;

[0015] The hybrid deep learning model is trained using the training set and validation set to obtain a crop classification model;

[0016] Use the test set to test the crop classification model and obtain the classification results;

[0017] Dynamically adjust the early season crop classification time point based on the confidence level of the classification results;

[0018] Among them, the hybrid deep learning model includes a prediction module and a classification module. The prediction module is based on the long short-term memory network and combines the encoder-decoder structure and attention mechanism. The classification module is a convolutional neural network. The prediction module and the classification module are associated through a cascade learning mechanism.

[0019] Furthermore, the encoder of the prediction module adopts a layer of long short-term memory network, which realizes the gradual deepening of feature extraction through series connection and outputs hidden state and cell state for initialization of the decoder.

[0020] Furthermore, the decoder of the prediction module introduces an attention mechanism, which includes a linear transformation layer for interactive calculation of the time step features output by the encoder and the hidden state of the decoder to obtain the attention score weight.

[0021] Furthermore, the classification module includes three cascaded convolution blocks.

[0022] Furthermore, each convolution block of the classification module includes a convolution layer and a ReLU activation layer, and a maximum pooling layer is added after the second convolution layer and the third convolution layer, and finally converted into a fully connected layer to output the classification result.

[0023] Furthermore, the loss function of the prediction module adopts a combination of mean square error loss and structural similarity index, the loss function of the classification module adopts cross entropy, and the multi-task cascade learning framework uses the GradNorm method to balance the learning of each task by dynamically adjusting the loss weights of different tasks, so that the hybrid deep learning model maintains a relatively balanced training state.

[0024] Furthermore, the method of dynamically adjusting the early season crop classification time point according to the confidence level of the classification results specifically includes:

[0025] Compare the confidence level of the classification result with a threshold;

[0026] If the confidence level is greater than the threshold, the input original multi-source remote sensing time series data is reduced to advance the prediction start time, and the next round of crop classification model testing is carried out, and the confidence level of the next round of testing is calculated;

[0027] If the confidence level is equal to the threshold, the predicted start time of the current round of testing is used as the earliest identifiable crop category time;

[0028] If the confidence level is less than the threshold, the predicted start time of the previous round of testing is used as the earliest identifiable crop category time.

[0029] The second object of the present invention can be achieved by adopting the following technical solutions:

[0030] A crop early season classification system based on multi-task cascade learning, the system comprising:

[0031] The first construction unit is used to obtain remote sensing time series data of early season crops to construct multi-source remote sensing time series data;

[0032] A partitioning unit is used to partition the multi-source remote sensing time series data into a training set, a validation set, and a test set;

[0033] The second building unit is used to build a hybrid deep learning model based on multi-task cascade learning;

[0034] A training unit is used to train the hybrid deep learning model using the training set and the validation set to obtain a crop classification model;

[0035] A testing unit is used to test the crop classification model using a test set to obtain classification results;

[0036] A dynamic adjustment unit is used to dynamically adjust the early season crop classification time point according to the confidence level of the classification results;

[0037] Among them, the hybrid deep learning model includes a prediction module and a classification module. The prediction module is based on the long short-term memory network and combines the encoder-decoder structure and attention mechanism. The classification module is a convolutional neural network. The prediction module and the classification module are associated through a cascade learning mechanism.

[0038] The third object of the present invention can be achieved by adopting the following technical solutions:

[0039] A computer device includes a processor and a memory for storing a program executable by the processor, wherein the processor implements the above-mentioned early season crop classification method when executing the program stored in the memory.

[0040] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0041] A computer-readable storage medium stores a program, which, when executed by a processor, implements the above-mentioned early-season crop classification method.

[0042] The present invention has the following beneficial effects compared to the prior art:

[0043] The present invention introduces cascade learning as its basis, associates the long short-term memory network based on the attention mechanism and the convolutional neural network in a sequential relationship, and uses the long short-term memory network based on the attention mechanism to predict crop remote sensing time series data, effectively making up for the information deficiency caused by the missing early season data, so that the classification model can extract more key phenological information; at the same time, the classification results of the convolutional neural network reversely optimize the performance of the long short-term memory network based on the attention mechanism, prompting the prediction module to generate remote sensing time series data that is more in line with crop growth. The bidirectional dependency between the two forms a synergistic effect, which ultimately significantly improves the accuracy of early season crop classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0045] Figure 1 This is a simplified flowchart of the early season crop classification method based on multi-task cascade learning according to Example 1 of the present invention.

[0046] Figure 2 This is a detailed flow chart of the early season crop classification method based on multi-task cascade learning according to Example 1 of the present invention.

[0047] Figure 3 Schematic diagram of the hybrid deep learning model of Example 1 of the present invention.

[0048] Figure 4 Schematic diagram of the prediction module in the hybrid deep learning model of Example 1 of the present invention.

[0049] Figure 5 Schematic diagram of the classification module in the hybrid deep learning model of Example 1 of the present invention.

[0050] Figure 6 Schematic diagram of the workflow of the loss weighting method of Example 1 of the present invention.

[0051] Figure 7 This is a structural block diagram of an early season crop classification system based on multi-task cascade learning according to Example 3 of the present invention.

[0052] Figure 8 This is a structural block diagram of a computer device according to embodiment 4 of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] Example 1:

[0055] like Figure 1 and Figure 2 As shown, this embodiment provides a method for early season crop classification based on multi-task cascade learning, which includes the following steps:

[0056] S201. Acquire remote sensing time series data of early season crops to construct multi-source remote sensing time series data.

[0057] This example uses Sentinel-1 SAR (Sentinel-1 C-band Synthetic Aperture Radar (SAR)), Sentinel-2, Landsat-7 / 8 TOA remote sensing images from 2017 to 2019, and MODIS LST products as the primary data sources. For Sentinel-2 (s2) and Landsat series data, the five basic bands of red, green, blue, infrared, and near-infrared are selected. Given the limitations of optical remote sensing images in cloud cover, haze, or rain and snow, this example incorporates Sentinel-1 Synthetic Aperture Radar (SAR) data. Sentinel-1 (s1) data consists of two polarization bands: VV and VH.

[0058] The following indices are calculated using the five basic bands of the Sentinel-2 (S2) and Landsat series:

[0059] Normalized Difference Vegetation Index (NDVI):

[0060] Enhanced Vegetation Index (EVI):

[0061] Ratio Vegetation Index (RVI):

[0062] Green vegetation cover index (GCVI):

[0063] Land Surface Water Index (LSWI):

[0064] Among them, Red represents the reflectivity of the red band, Green represents the reflectivity of the green band, Blue represents the reflectivity of the blue band, NIR represents the reflectivity of the near-infrared band, SWIR represents the reflectivity of the short-wave infrared band, G represents the gain factor, and c1 and c2 are coefficients used to correct the influence of aerosol scattering.

[0065] At the same time, the following indices are calculated using data from the radar sensor Sentinel-1:

[0066] Cross-Ratio:

[0067] Where VV and VH are the reflected signal intensities of the two polarization bands.

[0068] The multi-source remote sensing time series data of this embodiment is produced by integrating the 7 basic bands in S1 (Blue, Green, Red, NIR, SWIR1, VV, VH) and the 6 indices (NDVI, EVI, RVI, GCVI, LSWI) calculated above, totaling 13 variables.

[0069] S202. Divide the multi-source remote sensing time series data into a training set, a validation set, and a test set.

[0070] In this embodiment, the multi-source remote sensing time series data is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0071] S203. Build a hybrid deep learning model based on multi-task cascade learning.

[0072] like Figure 3 As shown, the hybrid deep learning model of this embodiment includes a prediction module and a classification module. The prediction module is based on the long short-term memory (LSTM) network and combines the encoder-decoder structure and the attention mechanism. It can be called the long short-term memory network based on the attention mechanism (Attention LSTM, AtLSTM for short). The classification module is a convolutional neural network (CNN). The prediction module and the classification module are associated through a cascade learning mechanism.

[0073] In this embodiment, the early season time series data is first input into the prediction module to perform future data prediction; then the predicted data is spliced ​​with the original early season data to form the full-year time series data; finally, it is input into the classification module to perform crop classification.

[0074] In this embodiment, the encoder of the prediction module is the input layer, which is used to extract high-level global features from the input time series and capture the long-term and short-term dependencies in the time series, such as Figure 4 As shown in Figure 1, the encoder uses a layer of long short-term memory network, which realizes the gradual deepening of feature extraction through series connection and outputs hidden state and cell state for initialization of the decoder.

[0075] Furthermore, the number of feature mapping channels of the encoder input sample feature data is 14, and the output feature generated is x∈R stride×14 (where stride represents the time step), the long short-term memory network is used to memorize the input feature data. It adaptively determines the extent to which the long short-term memory network retains the previous state and memorizes the current information through the gating mechanism, and outputs each encoded hidden feature h through the tanh activation function. t(which contains information from the previous t-1 time steps).

[0076] In this embodiment, the decoder of the prediction module introduces an attention mechanism to extract and amplify the features most relevant to the current prediction, enrich the context information, and generate a coherent and accurate output, such as Figure 4 As shown in Figure 2, the attention mechanism includes a linear transformation layer for interactive calculation of the time-step features output by the encoder and the hidden state of the decoder to obtain the attention score weight.

[0077] Furthermore, the attention mechanism first concatenates the output feature x in the encoder's long short-term memory network layer with the hidden feature h, then passes it through the fully connected layer, and finally uses the softmax activation function to normalize the trainable parameters v and output the probability distribution a. t , assigning an “attention score” between 0 and 1 to each element in the encoding.

[0078] Furthermore, the decoder uses a long short-term memory network to input the multiplication and addition results of the attention score and the encoder output features at each time step, and outputs the predicted crop growth information; specifically, the matrix c formed by concatenating the output features and the hidden features is combined with the attention score matrix a t The matrix obtained by matrix multiplication is passed to the long short-term memory network layer and the tanh activation function is used. The final result output by the regression head fully connected layer is the predicted growth information of early season crops.

[0079] like Figure 5 As shown in the figure, the classification module consists of three cascaded Conv1d convolution blocks. Each Conv1d convolution block includes a Conv1d convolution layer (the number of channels is 32, 64, and 128, respectively) and a ReLU activation layer. A MaxPool layer (size is 2, the number of channels is the number of channels of the corresponding convolution layer) is added after the second and third convolution layers. Finally, it is converted to a Classification Head fully connected layer to output the classification result. The specific process is as follows:

[0080] The input growth information matrix first passes through the first Conv1d convolution block and outputs a matrix x1∈R with nonlinear features N×32 , and then pass through the second Conv1d convolution block, the output matrix x2∈R N×64 , then the length of the feature matrix is ​​compressed through the MaxPool maximum pooling layer to obtain the matrix Then pass through a Conv1d convolution block and a MaxPool maximum pooling layer to obtain the matrix Finally, the feature matrix x4 is expanded and input into the Classification Head fully connected layer to output the final classification result.

[0081] In this embodiment, the cascade learning mechanism refers to the joint learning mechanism of the prediction module and the classification module. The predicted remote sensing time series data enables the classification module to obtain more time series data of the crop growing season and extract more key phenological information, thereby effectively improving the accuracy of classification; at the same time, the results of classification and prediction further optimize the performance of the entire hybrid deep learning for crop classification.

[0082] S204: Use the training set and the validation set to train the hybrid deep learning model to obtain a crop classification model.

[0083] In this embodiment, the hybrid deep learning model is trained using the training set and validation set divided in step S202 to obtain a crop classification model. During the training process, the loss function of the prediction module adopts a combination of the traditional mean square error loss (MSE) and the structural similarity index (SSIM), and the loss function of the classification module adopts the traditional cross entropy (CCE).

[0084] Furthermore, the loss function of the prediction module is as follows:

[0085] L pred =MSE(h(x),y)+(1-SSIM(h(x),y))

[0086] Among them, h(x) represents the predicted time series of the prediction module, and y represents the actual time series.

[0087] Furthermore, the loss function of the classification module is as follows:

[0088]

[0089] in, represents the true crop label, and z represents the classification result of the classification module.

[0090] The framework of multi-task cascade learning uses the GradNorm method, such as Figure 6 As shown in the figure, by dynamically adjusting the loss weights of different tasks to balance the learning of each task, the hybrid deep learning model can maintain a relatively balanced training state. Therefore, the loss function of the hybrid deep learning model is as follows:

[0091]

[0092] The specific steps are as follows:

[0093] First, in order to adjust the gradients of the loss functions of the prediction and classification modules to a common scale, the gradients are normalized. For task i, the gradient norm is as follows:

[0094]

[0095] Among them, i=1 represents the prediction module task, i=2 represents the classification module task, L pred ,L c , represents the gradient norm of the i-th task at the t-th epoch, Indicates the derivative of the model parameter W, w i (t) represents the weight of task i, L i (t) is the loss of the t-th epoch for the i-th task.

[0096] Secondly, the average gradient norm of all tasks is calculated as the benchmark of the shared scale, as follows:

[0097]

[0098] in,, is the average gradient norm E of all tasks task .

[0099] Again, by adjusting the size of the back propagation gradient of each task, the task training rate is relatively balanced. The training rate of task i is calculated as follows:

[0100]

[0101] in, represents the ratio of the loss of task i in the tth epoch to the initial loss, L i (0) represents the loss value of task i at the initial moment, r i (t) is the relative training rate of task i and defines the target gradient norm of task i:

[0102] Finally, in order to achieve gradient adjustment, a gradient loss function L is introduced grad The gradient loss is calculated using L1norm, which calculates the L1 loss for the output value and target value of each task, and then sums all the tasks to get the total gradient loss, as shown below:

[0103]

[0104] Finally, the Classification Head fully connected layer, the Regression Head fully connected layer and the Shared Layers shared layer can be balanced and optimized.

[0105] S205. Use the test set to test the crop classification model to obtain the classification result.

[0106] In this embodiment, the test set divided in step S202 is used as the input of the prediction module, and the classification result is output after passing through the early-season crop classification model.

[0107] S206. Dynamically adjust the early-season classification time point of the crop according to the confidence level of the classification result.

[0108] Calculate the confidence level of the classification result as follows:

[0109]

[0110] where Precision is the precision rate, Recall is the recall rate, Event A represents that a certain crop is predicted as the positive class, and event B represents that a certain crop is actually the positive class.

[0111] Set the confidence level threshold p0 = 80%, and compare the confidence level F1_score of the classification result with the threshold p0:

[0112] 1) If the confidence level is greater than the threshold (F1-score > p0), then reduce the input original multi-source remote sensing time series data, advance the prediction start time, and perform the next round of crop classification model testing (return to step S204), and calculate the confidence level of the next round of testing.

[0113] In 2), if the confidence level is equal to the threshold (F1-score = p0), then use the prediction start time of the current round of testing as the earliest recognizable crop category time.

[0114] 3) If the confidence level is less than the threshold (F1-score < p0), then use the prediction start time of the previous round of testing as the earliest recognizable crop category time.

[0115] It should be noted that although the above method operations are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the shown operations must be performed to achieve the desired result. On the contrary, the described steps can be changed in the execution order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0116] Embodiment 2:

[0117] In order to verify the effect of the hybrid deep learning model in Example 1 above, this embodiment sets up a simulation experiment to verify the superiority of the hybrid deep learning model:

[0118] 1) Simulation conditions

[0119] The hardware platform is: 12th Gen Intel(R)Core(TM)i5-12600KF, NVIDIAGeForce RTX3090.

[0120] The software platform is: Pycharm, and the model is built using PyTorch as the framework.

[0121] 2) Simulation method

[0122] The method of the above-mentioned embodiment 1 is used to obtain remote sensing time series data; wherein the remote sensing data includes remote sensing time series data and its corresponding real crop labels; multi-source remote sensing time series data is constructed, and the data is divided into a training set and a validation set according to a certain ratio to train the hybrid deep learning model, and finally the test set is used to classify crops.

[0123] 3) Simulation content and simulation results

[0124] The crop types selected for the simulation experiment mainly include soybeans, corn, rice, and other crops, a total of four crop types. This embodiment constructs three classifiers for soybeans and non-soybeans, corn and non-corn, and rice and non-rice for simulation experiments. Soybeans and non-soybeans, corn and non-corn, and rice and non-rice are represented by {0, 1}, respectively, and their corresponding real crop labels.

[0125] In the simulation experiment, a total of 17,870 sample data points were collected and divided into training set, validation set and test set in a ratio of 8:1:1. The early season crop classification accuracy on the test set is shown in Table 1.

[0126] Table 1 Classification accuracy of crops in early season

[0127] DOY 140 150 160 170 180 190 200 210 220 230 Corn 0.6614 0.6699 0.6656 0.6762 0.7248 0.7818 0.8333 0.8621 0.8816 0.8985 Soybean 0.6266 0.6645 0.6705 0.6728 0.7343 0.8098 0.8571 0.8786 0.8949 0.8855 DOY 80 90 100 110 120 130 140 150 160 170 Rice 0.6332 0.6847 0.6948 0.7774 0.8012 0.8882 0.9354 0.9486 0.9541 0.9544

[0128] As can be seen from Table 1, in the remote sensing segmentation scenario, the hybrid deep learning model provided in this embodiment shows robust and excellent classification accuracy in early-season crop classification, especially in the classification of crops before the key phenological period, and has achieved very good results. It provides a baseline for another early-season crop classification model in addition to CNN, Transformer, RNN and RF for the field of early-season crop classification, and provides an effective solution to the existing problems of existing early-season classification methods.

[0129] Example 3:

[0130] like Figure 7 As shown, this embodiment provides an early season crop classification system based on multi-task cascade learning. The system includes a first construction unit 701, a division unit 702, a second construction unit 703, a training unit 704, a testing unit 705, and a dynamic adjustment unit 706. The specific functions of each unit are as follows:

[0131] The first construction unit 701 is used to obtain remote sensing time series data of early season crops to construct multi-source remote sensing time series data;

[0132] A division unit 702 is used to divide the multi-source remote sensing time series data into a training set, a validation set, and a test set;

[0133] A second construction unit 703 is configured to construct a hybrid deep learning model based on multi-task cascade learning;

[0134] A training unit 704 is used to train the hybrid deep learning model using the training set and the validation set to obtain a crop classification model;

[0135] A testing unit 705 is used to test the crop classification model using a test set to obtain a classification result;

[0136] A dynamic adjustment unit 706 is used to dynamically adjust the early season crop classification time point according to the confidence level of the classification result;

[0137] Among them, the hybrid deep learning model includes a prediction module and a classification module. The prediction module is based on the long short-term memory network and combines the encoder-decoder structure and attention mechanism. The classification module is a convolutional neural network. The prediction module and the classification module are associated through a cascade learning mechanism.

[0138] The specific implementation of each unit in this embodiment can be found in the above-mentioned embodiment 1, and will not be described one by one here; it should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional units. In actual applications, the above-mentioned functions can be assigned to different functional units as needed, that is, the internal structure can be divided into different functional units to complete all or part of the functions described above.

[0139] Example 4:

[0140] This embodiment provides a computer device, such as Figure 8As shown, it includes a processor 802, a memory, an input device 803, a display device 804, and a network interface 805 connected via a device bus 801. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 806 and an internal memory 807. The non-volatile storage medium 806 stores an operating device, a computer program, and a database. The internal memory 807 provides an environment for the operation of the operating device and the computer program in the non-volatile storage medium. When the processor 802 executes the computer program stored in the memory, the early season crop classification method of the above-mentioned embodiment 1 is implemented as follows:

[0141] Remote sensing time series data of early season crops are obtained to construct multi-source remote sensing time series data; a training set, a validation set and a test set are divided according to the multi-source remote sensing time series data; a hybrid deep learning model is constructed based on multi-task cascade learning; the hybrid deep learning model is trained with the training set and the validation set to obtain a crop classification model; the crop classification model is tested with the test set to obtain a classification result; and the early season crop classification time point is dynamically adjusted according to the confidence level of the classification result; wherein the hybrid deep learning model includes a prediction module and a classification module, the prediction module is based on a long short-term memory network and combines an encoder-decoder structure and an attention mechanism, the classification module is a convolutional neural network, and the prediction module and the classification module are associated through a cascade learning mechanism.

[0142] Example 5:

[0143] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the early-season crop classification method of the above-mentioned embodiment 1 is implemented as follows:

[0144] Remote sensing time series data of early season crops are obtained to construct multi-source remote sensing time series data; a training set, a validation set and a test set are divided according to the multi-source remote sensing time series data; a hybrid deep learning model is constructed based on multi-task cascade learning; the hybrid deep learning model is trained with the training set and the validation set to obtain a crop classification model; the crop classification model is tested with the test set to obtain a classification result; and the early season crop classification time point is dynamically adjusted according to the confidence level of the classification result; wherein the hybrid deep learning model includes a prediction module and a classification module, the prediction module is based on a long short-term memory network and combines an encoder-decoder structure and an attention mechanism, the classification module is a convolutional neural network, and the prediction module and the classification module are associated through a cascade learning mechanism.

[0145] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0146] In this embodiment, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this embodiment, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0147] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to implement the computer program of the present embodiment, including object-oriented programming languages ​​such as Java, Python, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect via the Internet).

[0148] In summary, the present invention introduces cascade learning as the basis, associates the long short-term memory network based on the attention mechanism and the convolutional neural network in a sequential relationship, and uses the long short-term memory network based on the attention mechanism to predict crop remote sensing time series data, which effectively makes up for the information deficiency caused by the lack of early season data, so that the classification model can extract more key phenological information; at the same time, the classification results of the convolutional neural network reversely optimize the performance of the long short-term memory network based on the attention mechanism, prompting the prediction module to generate remote sensing time series data that is more in line with crop growth. The bidirectional dependency relationship between the two forms a synergistic effect, which ultimately significantly improves the accuracy of early season crop classification.

[0149] The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes based on the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which falls within the scope of protection of the present invention.

Claims

1. A crop early season classification method based on multi-task cascade learning, characterized by: The method comprises: Acquire remote sensing time series data of early season crops to construct multi-source remote sensing time series data; Based on multi-source remote sensing time series data, the training set, validation set and test set are divided; Build a hybrid deep learning model based on multi-task cascade learning; The hybrid deep learning model is trained using the training set and validation set to obtain a crop classification model; Use the test set to test the crop classification model and obtain the classification results; Dynamically adjust the early season crop classification time point based on the confidence level of the classification results; Among them, the hybrid deep learning model includes a prediction module and a classification module. The prediction module is based on the long short-term memory network and combines the encoder-decoder structure and attention mechanism. The classification module is a convolutional neural network. The prediction module and the classification module are associated through a cascade learning mechanism.

2. The early season crop classification method according to claim 1, characterized in that: The encoder of the prediction module adopts a layer of long short-term memory network, which realizes the gradual deepening of feature extraction through series connection and outputs hidden state and cell state for initialization of the decoder.

3. The early season crop classification method according to claim 1, characterized in that: The decoder of the prediction module introduces an attention mechanism, which includes a linear transformation layer for interactive calculation of the time step features output by the encoder and the hidden state of the decoder to obtain the attention score weight.

4. The early season crop classification method according to claim 1, characterized in that: The classification module consists of three cascaded convolutional blocks.

5. The early season crop classification method according to claim 4, characterized in that: Each convolution block of the classification module includes a convolution layer and a ReLU activation layer, and a maximum pooling layer is added after the second convolution layer and the third convolution layer, and finally converted into a fully connected layer to output the classification result.

6. The early season crop classification method according to claim 1, characterized in that: The loss function of the prediction module adopts a combination of mean square error loss and structural similarity index, the loss function of the classification module adopts cross entropy, and the multi-task cascade learning framework uses the GradNorm method to balance the learning of each task by dynamically adjusting the loss weights of different tasks, so that the hybrid deep learning model maintains a relatively balanced training state.

7. The early season crop classification method according to any one of claims 1 to 6, characterized in that: The method of dynamically adjusting the early season crop classification time point based on the confidence level of the classification results specifically includes: Compare the confidence level of the classification result with a threshold; If the confidence level is greater than the threshold, the input original multi-source remote sensing time series data is reduced to advance the prediction start time, and the next round of crop classification model testing is carried out, and the confidence level of the next round of testing is calculated; If the confidence level is equal to the threshold, the predicted start time of the current round of testing is used as the earliest identifiable crop category time; If the confidence level is less than the threshold, the predicted start time of the previous round of testing is used as the earliest identifiable crop category time.

8. An early season crop classification system based on multi-task cascade learning, characterized by: The system comprises: The first construction unit is used to obtain remote sensing time series data of early season crops to construct multi-source remote sensing time series data; A partitioning unit is used to partition the multi-source remote sensing time series data into a training set, a validation set, and a test set; The second building unit is used to build a hybrid deep learning model based on multi-task cascade learning; A training unit is used to train the hybrid deep learning model using the training set and the validation set to obtain a crop classification model; A testing unit is used to test the crop classification model using a test set to obtain classification results; A dynamic adjustment unit is used to dynamically adjust the early season crop classification time point according to the confidence level of the classification results; Among them, the hybrid deep learning model includes a prediction module and a classification module. The prediction module is based on the long short-term memory network and combines the encoder-decoder structure and attention mechanism. The classification module is a convolutional neural network. The prediction module and the classification module are associated through a cascade learning mechanism.

9. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the early season crop classification method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the early season crop classification method according to any one of claims 1 to 7 is implemented.

Images