Crop identification method and device for agricultural irrigation ditch area, terminal, medium and product

Abstract

The invention discloses a crop identification method and device for an agricultural irrigation ditch area, a terminal, a storage medium and a computer program product, and the method comprises the steps: building a self-adaptive early classification framework based on an MTTNet network and a decision network; historical satellite time sequence data are collected and preprocessed, and sample data are obtained and divided into a training set and a test set; performing iterative training on the adaptive early classification framework by using the training set and the test set to obtain an adaptive early classification model; the method comprises the following steps: collecting satellite time sequence data to be classified, preprocessing the satellite time sequence data, and classifying the satellite time sequence data by using a self-adaptive early classification model to obtain a classification result and a time node corresponding to the satellite time sequence data to be classified so as to realize identification of the type and the growth time node of crops in an agricultural irrigation area to be classified. According to the scheme, the model is obtained by setting the self-adaptive early classification framework and training the historical data, so that early recognition of a large area is conveniently realized, and the accuracy is good.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a method, device, terminal, storage medium and computer program product for crop recognition in agricultural irrigation canal areas. Background Art

[0002] Early crop recognition refers to predicting crop types using limited satellite observation data in the early stage of the crop growing season. Early recognition of crop types plays a crucial role in modern agricultural management and global food security. Early monitoring can also help assess the impact of agricultural activities on the ecosystem and provide a more accurate basis for risk assessment for agricultural insurance companies. Finally, early recognition provides key data for national and regional food security planning, supporting decision-makers in resource allocation and policy-making. Therefore, in recent years, there has been an increasing interest in developing effective crop classification methods, especially using satellite image time series classification, as it can capture the dynamic changes of crops throughout the growing season and over a large area.

[0003] Early crop recognition refers to predicting crop types using limited satellite observation data in the early stage of the crop growing season. Timely and accurate identification of crop types is crucial for crop yield prediction, disaster warning and food security. However, in cross-regional scenarios where the crop time spectral characteristics vary greatly in different regions, crop early mapping methods in large areas often face significant challenges. Specific vegetation indices or deep learning methods are used to solve the problem of early crop recognition, but these methods usually conduct early mapping for specific crops in specific regions or use incremental strategies to determine the optimal recognition period of crops, which at least leads to difficulties in early mapping of large-area crops.

[0004] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The object of the present invention is to provide a method, device, terminal, storage medium and computer program product for crop recognition in agricultural irrigation canal areas, so as to solve the problem that early mapping for specific crops in specific regions or using incremental strategies to determine the optimal recognition period of crops at least leads to difficulties in early mapping of large-area crops, and achieve the effect of facilitating early recognition of large-area crops with good accuracy by setting an adaptive early classification framework and training a model using historical data.

[0006] The present invention provides a method for identifying crops in an agricultural irrigation canal area, which is applied to identify the types and growth time nodes of crops in an agricultural irrigation area. The method for identifying crops in the agricultural irrigation canal area includes: building an adaptive early classification framework based on the MTTNet network and the decision network; collecting historical satellite time series data of crops in the agricultural irrigation area, denoted as historical satellite time series data; preprocessing the historical satellite time series data to obtain sample data; and dividing the sample data into a training set and a test set; iteratively training the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model; collecting satellite time series data of crops in the agricultural irrigation area to be classified, denoted as satellite time series data to be classified; after preprocessing the satellite time series data to be classified, using the adaptive early classification model for classification to obtain the classification result and time node corresponding to the satellite time series data to be classified, so as to realize the identification of the types and growth time nodes of crops in the agricultural irrigation area to be classified.

[0007] In some embodiments, building an adaptive early classification framework based on the MTTNet network and the decision network includes: setting the MTTNet network based on the TCN model and the Transformer self-attention mechanism; fusing the decision network with reinforcement learning based on the MTTNet network to obtain the required adaptive early classification framework.

[0008] In some embodiments, setting the MTTNet network based on the TCN model and the Transformer self-attention mechanism includes: based on the TCN model and the Transformer self-attention mechanism, using depthwise separable convolution and introducing the GELU activation function to set up a first convolutional network; adding a residual structure based on the first convolutional network and using at least one of layer normalization technology and dropout technology to set up a second convolutional network as the required MTTNet network.

[0009] In some embodiments, the satellite time series data in collecting historical satellite time series data of crops in the agricultural irrigation area and collecting satellite time series data of crops in the agricultural irrigation area to be classified includes: collecting data of the bottom-of-atmosphere reflectance image of Sentinel-2 Level-2A, and selecting the median of this data to synthesize the final time series data at a set number of days interval as the collected satellite time series data.

[0010] In some embodiments, the preprocessing in preprocessing the historical satellite time series data and preprocessing the satellite time series data to be classified includes: for the collected satellite time series data, using the linear interpolation method, obtaining an image covering the entire time period with a time series of a set number of days, and obtaining the required time series.

[0011] In some embodiments, iteratively training the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model includes: training the adaptive early classification framework using the training set and the test set, which is the training model of the adaptive early classification framework; testing the training model of the adaptive early classification framework using the test set to obtain a test result; updating the training model of the adaptive early classification framework according to the test result to obtain an updated model of the training model of the adaptive early classification framework; and iterating in this way until the error of the updated model of the training model of the adaptive early classification framework is less than a set threshold or the number of iterations reaches a set number, and taking the updated model of the training model of the adaptive early classification framework as the adaptive early classification model.

[0012] Matched with the above method, on the other hand, the present invention provides a crop recognition device for an agricultural irrigation canal area, which is applied to recognize the types and growth time nodes of crops in an agricultural irrigation area. The crop recognition device for the agricultural irrigation canal area includes: a control unit configured to build an adaptive early classification framework based on the MTTNet network and a decision network; an acquisition unit configured to collect historical satellite time series data of crops in the agricultural irrigation area, denoted as historical satellite time series data; the control unit is further configured to preprocess the historical satellite time series data to obtain sample data; and divide the sample data into a training set and a test set; the control unit is further configured to iteratively train the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model; the acquisition unit is further configured to collect satellite time series data of crops in the agricultural irrigation area to be classified, denoted as satellite time series data to be classified; the control unit is further configured to preprocess the satellite time series data to be classified and then classify it using the adaptive early classification model to obtain a classification result and a time node corresponding to the satellite time series data to be classified, so as to realize the recognition of the types and growth time nodes of crops in the agricultural irrigation area to be classified.

[0013] Matched with the above device, on yet another aspect, the present invention provides a terminal, including: the above-mentioned crop recognition device for the agricultural irrigation canal area.

[0014] Matched with the above method, on the other hand, the present invention provides a storage medium, which includes a stored program. When the program runs, it controls the device where the storage medium is located to execute the steps of the crop recognition method in the agricultural irrigation canal area described above.

[0015] Matched with the above method, on the other hand, the present invention provides a computer program product, including a computer program, which realizes the steps of the crop recognition method in the agricultural irrigation canal area described above when executed by a processor.

[0016] Thus, in the solution of the present invention, an adaptive early classification (i.e., GEO-EARLIEST) framework composed of a basic classification network MTTNet and a decision network Agent is set up, where the MTTNet network is composed of multi-scale TCN blocks and Transformer blocks; after collecting historical satellite observation data (such as the median of Sentinel-2 observation data to synthesize the final time series data at 10-day intervals) and performing data preprocessing, sample data is obtained; the sample data is divided into a training set and a test set, and the GEO-EARLIEST framework is trained using the training set to obtain a training model of the GEO-EARLIEST framework. The training model of the GEO-EARLIEST framework is tested and updated using the test set to obtain a test model of the GEO-EARLIEST framework, which is used as the GEO-EARLIEST model to classify the satellite time series to be classified, and classification results and time nodes are obtained; thus, by setting up the GEO-EARLIEST framework and training with historical data to obtain a model, it is convenient to achieve early recognition of large areas, such as accurately recognizing the classification results and the earliest recognizable time nodes of irrigation areas, which is conducive to water conservation for targeted irrigation.

[0017] Other features and advantages of the present invention will be described in the following specification, and some will be obvious from the specification or understood by implementing the present invention.

[0018] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0019] Figure 1 It is a schematic flowchart of an embodiment of the crop recognition method in the agricultural irrigation canal area of the present invention;

[0020] Figure 2 It is a schematic flowchart of an embodiment of building an adaptive early classification framework in the method of the present invention;

[0021] Figure 3 It is a schematic flowchart of an embodiment of setting up the MTTNet network in the method of the present invention;

[0022] Figure 4 Schematic flow chart of an embodiment for obtaining an adaptive early classification model in the method of the present invention;

[0023] Figure 5 Schematic structural diagram of an embodiment of the crop recognition device in the agricultural irrigation canal area of the present invention;

[0024] Figure 6 Schematic image of crop distribution information, where (a) represents the CDL crop cover product released by the US Department of Agriculture in 2021 [i.e., Geographical positioning of the six study sites overlayed on the 2021 Cropland Data Layer (CDL)], and (b) represents the crop distribution information of the selected 5 study areas (i.e., Study area information);

[0025] Figure 7 Schematic image of the planting, growth, and harvesting cycles of the main crops, representing the growth calendars of different crops (i.e., Major crop’s typical planting, growing, and harvest period in USA);

[0026] Figure 8 Table showing the number of crop samples in each county in 2021 (i.e., The number of crop samples in each county in 2021), namely Table 1;

[0027] Figure 9 Schematic overall flow chart of the crop early recognition method in the agricultural irrigation canal area;

[0028] Figure 10 Schematic structural diagram of the GEO-EARLIEST model (i.e., GEO-EARLIEST model);

[0029] Figure 11 Schematic diagram of the MTTNet architecture (i.e., MTTNet Architecture);

[0030] Figure 12 Schematic structural diagram of the TCN module (i.e., The dilated causal convolutions);

[0031] Figure 13 Schematic structural diagram of the self-attention mechanism (i.e., Masked time attention mechanism);

[0032] Figure 14 Schematic diagram of spectral characteristics of crops in different counties;

[0033] Figure 15 Schematic diagram of the earliest recognizable time of different crops;

[0034] Figure 16 Schematic diagram of the distribution of different crops;

[0035] Figure 17 Schematic diagram of early stages of multiple crops;

[0036] Figure 18 Schematic diagram of feature interpretability analysis;

[0037] Figure 19 Comparison table of different models, i.e., Table 2.

[0038] In combination with the accompanying drawings, the reference numerals in the embodiments of the present invention are as follows:

[0039] 102 - Acquisition unit; 104 - Control unit. Detailed implementation manners

[0040] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in combination with specific embodiments of the present invention and the corresponding accompanying drawings. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] Considering that specific vegetation indices or deep learning methods are used to solve the problem of early crop identification, but these methods usually perform early mapping for specific crops in specific regions or use incremental strategies to determine the optimal identification period of crops, resulting in difficulties in early mapping of large-area crops and a large amount of data storage requirements brought by incremental strategies.

[0042] Crop mapping has been well achieved through machine learning and deep learning methods. Especially using deep learning methods, complex change patterns can be extracted from satellite image time series. However, these methods often perform classification only once at the end of the crop growing season. Unfortunately, when faced with tasks that require timely acquisition of crop distribution maps, this method that relies on data throughout the growing season fails to provide timely support for in-season agricultural management and decision-making. To address this challenge and improve the timeliness of crop mapping, researchers have increasingly focused on identifying crop types earlier within the growing season. Although promising results have been obtained, these methods are often applicable to specific regions and specific crops, which means they are suitable for target study areas with similar agro-ecological conditions to the source domain. Even for the same crop, due to different climate and environmental conditions, the spectral and temporal characteristics of crops in different regions vary. When directly applied to different regions, their adaptability may be severely challenged. Early crop mapping using time series remote sensing data faces many challenges. For example: (1) Spectral features are not obvious in the early stages of the crop growth period, making it difficult to distinguish different crops; (2) The limitations of multiple crops and large areas; (3) Large-scale storage, calculation, and preprocessing of remote sensing time series data require large storage space and efficient computing resources. The above methods are difficult to clearly alleviate the problem of early crop identification in different regions.

[0043] In recent years, the strategies for early crop identification have mainly been divided into two types. One is to determine features for specific time periods with significant differences for different objects. For example, for rapeseed mapping, Zang Yunze et al. used the enhanced regional yellowness index to enhance the weak yellow signal during the flowering period. You Nanshan et al. found that the reflectance of the red edge band of corn and soybeans differed during the heading stage of corn and the podding stage of soybeans. This strategy is usually designed for specific crops and lacks generalization ability across different crops and regions. For different crops, it may be necessary to redesign features or adjust the classification strategy. The other is the incremental classification strategy. These methods are usually based on machine learning and deep learning models such as support vector machine (SVM) and random forest (RF). Each time new time series data (such as new satellite images) is available, the classifier is retrained or updated. Researchers will record the changes in model accuracy as new data is added and determine the earliest recognizable time. This incremental method has certain limitations. The computational complexity is relatively high. Each time new data is available, the model needs to be retrained or updated, resulting in high computational costs, especially for large-scale datasets, where the efficiency of this method is low. In addition, it lacks an intelligent stopping mechanism. These methods usually rely on artificially set accuracy thresholds or time thresholds and cannot dynamically and adaptively select the optimal recognition time.

[0044] In recent years, deep learning methods have shown greater advantages in processing time series data. Among them, Convolutional Neural Networks (CNNs) extract features through convolutional operations. Due to the fixed size of the convolutional kernels, CNNs perform well in capturing local information, but their ability to obtain global information is relatively limited. On the other hand, Recurrent Neural Networks (RNNs) are designed specifically for time series data modeling and can process each time step sequentially through recurrent units. In particular, Long Short-Term Memory (LSTM), as a commonly used RNN method, has been widely used in land cover classification based on time series images. Different from RNNs, Transformers (a neural network architecture based on the attention mechanism) adopt multi-head self-attention mechanisms and positional encoding, and can process all time steps of sequence data simultaneously. At the same time, reinforcement learning focuses on solving sequential decision-making problems in the interaction between agents and the environment. In particular, the application of deep reinforcement learning eliminates the need for manual design of the state space and enables direct processing of unstructured input data. In many fields, reinforcement learning algorithms have been successfully applied.

[0045] Considering the need to determine the crop spatial distribution in a timely manner, the solution of the present invention develops an adaptive early classification framework that combines deep learning and reinforcement learning (i.e., GEO-EARLIEST). The adaptive early classification framework is used for early crop mapping within the crop season based on Sentinel 2 data (Adaptive Early Classification Framework for Early Crop Mapping Based on Sentinel 2 Data), which not only maintains a leading position in recognition accuracy but also can significantly advance the identification of crop species and determine the optimal identification timing.

[0046] According to an embodiment of the present invention, a method for crop identification in an agricultural irrigation canal area is provided, as Figure 1 shown in the schematic flowchart of an embodiment of the method of the present invention. The method for crop identification in the agricultural irrigation canal area is applied to identify the types and growth time nodes of crops in the agricultural irrigation area. The method for crop identification in the agricultural irrigation canal area includes: steps S110 to S160.

[0047] At step S110, an adaptive early classification framework, i.e., the GEO-EARLIEST framework, is pre-built based on the MTTNet network and the decision network.

[0048] At step S120, historical satellite time series data of crops in the agricultural irrigation area is pre-collected and denoted as historical satellite time series data.

[0049] At step S130, preprocess the historical satellite time series data to obtain sample data; and divide the sample data into a training set and a test set.

[0050] At step S140, iteratively train the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model, such as the GEO-EARLIEST model.

[0051] At step S150, collect the satellite time series data of the crops in the agricultural irrigation area to be classified, denoted as the satellite time series data to be classified.

[0052] At step S160, after preprocessing the satellite time series data to be classified, classify it using the adaptive early classification model to obtain the classification result and time node corresponding to the satellite time series data to be classified, so as to realize the identification of the types and growth time nodes of the crops in the agricultural irrigation area to be classified.

[0053] The solution of the present invention develops an adaptive early classification framework (i.e., GEO-EARLIEST) combining deep learning and reinforcement learning, uses the time series and samples of the whole year of 2021 to train the adaptive early classification framework, and determines the earliest recognizable time of each crop. Specifically, the solution of the present invention uses the data of historical years to train and obtain the earliest recognizable time of each crop, and then applies the classification model in the framework to the images at the beginning of the 2022 growing season to obtain the crop distribution map of 2022. The solution of the present invention compares the early mapping results with other methods using full-length sequences. The main contributions of the solution of the present invention are: (1) A general early recognition framework for large-area scales is proposed, breaking through the limitation that existing methods are only applicable to specific regions or crops; (2) Compared with various state-of-the-art models, it shows excellent performance in terms of classification accuracy, mapping result quality, and earliness; (3) Through experiments in different spatial regions, the model of the solution of the present invention shows good adaptability in a large-area range.

[0054] The solution of the present invention, through the proposed GEO-EARLIEST framework, specifically solves the difficulties in early mapping of crops in large areas and the large data storage requirements brought by the incremental strategy. For the problem of difficult early mapping of crops in large areas, the GEO-EARLIEST framework integrates the basic classification network (MTTNet) and the decision-making network (Agent), extracts the time-series features of historical satellite observation data through multi-scale TCN blocks and Transformer blocks, and can achieve unified early classification of crops in a large area without relying on prior knowledge of specific regions or crops. For example, for an area covering multiple crops such as wheat, corn, and rice, this method can generate early mapping results of each crop under the same framework without separate modeling, thus significantly improving the efficiency. To solve the large data storage requirements caused by the incremental strategy.

[0055] In some embodiments, for the specific process of building an adaptive early classification framework based on the MTTNet network and the decision-making network in step S110, refer to the following exemplary description.

[0056] The following combines Figure 2 The schematic flowchart of an embodiment of building an adaptive early classification framework in the method of the present invention shown in the figure further illustrates the specific process of building an adaptive early classification framework in step S110, including: step S210 to step S220.

[0057] Step S210, set the MTTNet network based on the TCN model and the Transformer self-attention mechanism.

[0058] Step S220, based on the MTTNet network, fuse the decision-making network with reinforcement learning to obtain the required adaptive early classification framework.

[0059] In the solution of the present invention, building an adaptive early classification framework based on the MTTNet network and the decision-making network not only maintains a leading position in recognition accuracy but also can significantly advance the recognition of crop types and determine their optimal recognition timing.

[0060] Preferably, for the specific process of setting the MTTNet network based on the TCN model and the Transformer self-attention mechanism in step S210, refer to the following exemplary description.

[0061] The following combines Figure 3 The schematic flowchart of an embodiment of setting the MTTNet network in the method of the present invention shown in the figure further illustrates the specific process of setting the MTTNet network in step S210, including: step S310 to step S320.

[0062] Step S310: Based on the TCN model and the Transformer self-attention mechanism, use depthwise separable convolution, introduce the GELU activation function, and set up to obtain the first convolutional network.

[0063] Step S320: Based on the first convolutional network, add a residual structure, and use at least one of layer normalization technology and dropout technology, and set up to obtain the second convolutional network as the required MTTNet network.

[0064] In the solution of the present invention, the approach overview is as follows. The solution of the present invention proposes an early classification framework GEO-EARLIEST (Hartvigsen et al., 2019) that combines TCN, Transformer, and reinforcement learning ideas, with the focus on extracting spatio-temporal information to predict crop types as early as possible from time series data. Figure 10 It is a schematic diagram of the structure of the GEO-EARLIEST model (i.e., the GEO-EARLIEST model). Figure 10 It is a schematic diagram that describes the proposed framework. The GEO-EARLIEST framework logically consists of two parts: the basic classification network MTTNet and the decision network Agent, where the MTTNet network is composed of multi-scale TCN blocks and Transformer blocks. Figure 10 It represents the designed early recognition model. Specifically, input time series represents the input time series, which undergoes feature extraction by MTTNet, and finally the decision agent decides whether to stop. The model finally outputs the label of each time series and the earliest recognizable time t-stop. The GEO-EARLIEST framework proposed in the solution of the present invention classifies according to the features extracted from the MTTNet network. MTTNet can obtain multi-scale time features from time series, where the TCN blocks help capture local features, while the Transformer blocks help capture global dependency features; then these features are fed back to the decision network Agent, which learns an early stopping strategy based on the observed information; finally, the predicted category of each time series and the earliest recognizable time node are output.

[0065] In the solution of the present invention, the relevant description of the MTTNet Architecture is as follows. Figure 11It is a schematic diagram of the MTTNet architecture (i.e., MTTNet Architecture). MTTNet is responsible for extracting multi-scale features of time series and mapping them to a vector representation St ∈ Rk, where k is an adjustable hyperparameter of the hidden dimension. The feature St at each time step is called the hidden state, representing the sequence dynamic information of all time steps before the current time step. The MTTNet network is composed of multi-scale TCN blocks and Transformer blocks, as Figure 11 shown. Figure 11 It represents the structure of NTTNet, where input time series represents the input time series. After extracting multi-scale local features through a multi-scale temporal convolutional network and performing feature fusion, it is input into the transformer attention mechanism to extract global features, and finally the entire extracted features are output. TCN and Feature fusion represent the temporal convolutional block and the feature fusion module respectively. First, the time series data is fed into the multi-scale TCN module to effectively extract local features. To ensure that the output at time step j is only related to the features at the current time step and previous time steps, the solution of the present invention uses dilated causal convolutions in each TCN block. Secondly, the multi-scale features extracted by the TCN layer are fused and then fed into the Transformer block to extract global features, and finally the time series features extracted by the model are output.

[0066] In the solution of the present invention, the relevant description of the TCN module (TCN block) is as follows. Figure 12 It is a schematic diagram of the structure of the TCN module (i.e., Thedilated causal convolutions). Figure 12 represents the temporal convolutional network, d represents different scales, and Hidden represents the hidden state. To avoid the problems of long-term dependence and difficulty in parallelization of RNN, Bai Shaojie proposed a temporal convolutional network with temporal processing ability. Aiming at the complexity of the original TCN structure, the solution of the present invention proposes a new improved structure. First, depthwise separable convolution is adopted, and the specific structure is shown in Figure 12 . This convolution method can effectively reduce the model parameters and computational complexity. At the same time, the GELU (Gaussian Error Linear Units) activation function is introduced. Compared with traditional activation functions, GELU provides better non-linear processing ability. In addition, layer normalization (LN) is adopted in this structure to stabilize the training process and reduce the risk of overfitting through the dropout technique. Finally, a residual structure is added to help solve the problem of gradient disappearance in deep networks. Such a design not only simplifies the model structure but also optimizes the learning ability of the model.

[0067] Specifically, in the solution of the present invention, first, in order to reduce the computational complexity of the model and optimize the original TCN structure, depthwise separable convolution is adopted. By dividing the convolution operation into depthwise convolution and pointwise convolution, the number of parameters is significantly reduced and the computational efficiency is improved. On this basis, the GELU (Gaussian Error Linear Unit) activation function is introduced. Compared with the traditional ReLU activation function, GELU enhances the feature expression ability of the model through a smooth non-linear transformation. At the same time, in order to further stabilize the training process, layer normalization (LN) is adopted. By normalizing the input features of each layer, the problems of gradient explosion or gradient disappearance are avoided, and the adaptability of the model to different input distributions is enhanced. To reduce the risk of overfitting of the model, the dropout technique is also added. By randomly masking some neurons, the generalization ability of the model is improved. In addition, to solve the problem of gradient disappearance in deep networks, a residual connection is introduced into the structure to ensure the effective transmission of deep features, thereby improving the overall performance of the model. These improvement steps complement each other and jointly achieve the design goal of the improved TCN module by reducing complexity, enhancing feature expression ability, improving training stability and generalization ability, and optimizing gradient transmission.

[0068] In the solution of the present invention, the relevant description of the Transformer module (i.e., Transformer block) is as follows. The Transformer block is responsible for extracting global features. In the traditional self-attention mechanism, the features at each time step are associated with the features of all time steps in the sequence data. For the early classification problem of time series, where each timestamp can only observe the information of the current time step and the previous ones, the solution of the present invention designs a masked self-attention mechanism, such that the features at each time step are only associated with the features of the data observed at the current moment, as Figure 13 shown. Figure 13 It is a schematic diagram of the structure of the self-attention mechanism (i.e., Maskedtime attention mechanism). Figure 13It represents the masked attention mechanism. Q represents the query vector, which represents the input feature at the current time step or the vector that needs to be correlated with other time steps. K represents the key vector, which is used to calculate the reference feature for the attention distribution. V represents the value vector, which is used to generate the final output feature representation. Mask represents the masking operation, which is used to avoid information leakage or mask certain unnecessary parts (such as masking future time steps in an autoregressive model). dk represents the dimension of the key vector K, which is used to scale the dot product to avoid gradient problems caused by overly large values. SoftMax represents the normalization function, which is used to normalize the attention distribution into a probability distribution.

[0069] Among them, the masked self-attention mechanism obtains the attention scores by calculating the dot product of the query vector Q and the key vector K, as shown in formula (1). To avoid leaking future information, the attention scores in the upper triangular part are set to negative infinity. Then, these scores are normalized through the SoftMax function, and the masked attention scores are multiplied by the value vector V to finally obtain the result input to the feed-forward layer. The feed-forward layer is a multi-layer perceptron that obtains the feature representation for each timestamp.

[0070]

[0071] It represents the masked attention mechanism. Q represents the query vector, which represents the input feature at the current time step or the vector that needs to be correlated with other time steps. K represents the key vector, which is used to calculate the reference feature for the attention distribution. V represents the value vector, which is used to generate the final output feature representation. Mask represents the masking operation, which is used to avoid information leakage or mask certain unnecessary parts (such as masking future time steps in an autoregressive model). dk represents the dimension of the key vector K, which is used to scale the dot product to avoid gradient problems caused by overly large values. SoftMax represents the normalization function, which is used to normalize the attention distribution into a probability distribution.

[0072] In the solution of the present invention, the relevant description of the decision-making network Agent is as follows. The control agent is a key component for controlling the workflow, which decides whether to continue observing the next time feature or activate the discriminator to generate the classification result of the sequence. To achieve this goal, the solution of the present invention adopts reinforcement learning technology to solve the partially observable Markov decision process (POMDP). The features extracted by the input backbone network from the satellite time series select the actions for each state through the use of a well-learned policy and obtain rewards based on the quality of the selected actions. The control agent is trained by a policy gradient-based method, and its goal is to optimize the long-term reward according to the performance of the discriminator. Among them, the key components of the given reinforcement learning framework are described as follows.

[0073] In reinforcement learning, the state is a representation of the current environment, describing the current situation. In the case of the solution of the present invention, the state is a set of feature vectors at the current moment, which is essentially the feature representation St of the currently observed time step t extracted from the satellite time series based on the MTTNet network. It should be noted that each vector St summarizes the time series information existing in X{0,…,t}, and the feature of the current time step is the core information that determines the action selection according to the learning strategy.

[0074] The policy is a strategy for controlling the agent to determine the next action according to the current state. A good way to reduce the number of states is to use a neural network-based policy, where the input is the state and the output is the action. Here, the policy selects an action, that is, a t by the current state S t =π θ (S t ). As mentioned above, S t is a low-dimensional hidden state encoding internal and interdependencies, used to represent the observed time series signal. Among them, the solution of the present invention uses a fully connected neural network to approximate this policy function. The policy here is a function that maps the current state S t to a parameterized distribution of a set of actions, as shown in formula (2).

[0075] π t =(1 - α)×σ(W π ·S t +b π ).

[0076] Among them, π t : represents the probability distribution function of selecting a certain action at time t. S t represents the observed time series signal, W π and b π are the weight matrix and bias vector parameters that need to be learned during the training process respectively. σ is the sigmoid activation function, used to fix the output between 0 and 1. α is a small positive number between 0 and 1, which is a small value for adjusting the parameterized distribution. ∈: random noise term, used to introduce exploratory behavior to prevent the network from falling into local optima.

[0077] The action is an operation that the control agent can perform in each state. The action controls the entire process of the discriminator: if a t =0, the agent will advance to the next observation and obtain the hidden state at the next time point. On the other hand, if a t =1, the agent selects to interrupt the observation, and then notifies the discriminator to predict the label by feeding the hidden vector state S t . Then, the approximate probability π tApplicable to the Bernoulli distribution to sample actions according to P(a t = 1)=π t Once sampled according to the output of the Bernoulli distribution, when the agent chooses to interrupt the observation or the observation series ends (t = T), t is regarded as the interruption time point τ, as shown in Equation (3).

[0078] a t = Bernoulli(p = π t ) (3).

[0079] a t : represents the action variable at time t, taking values 0 or 1. Bernoulli(π t ): indicates that the action a t is randomly sampled according to the Bernoulli distribution with parameter π t .

[0080] The reward is used to quantify the effect of the current policy. To promote the cooperation between the control agent and the discriminator, the agent needs to measure its success based on the feedback of the discriminator. Therefore, when the discriminator gives the correct label, the reward r t of the current policy is r t = 1 / T; conversely, when the prediction is incorrect, it is punished, and the reward r t is r t = -1 / T. The solution of the present invention designs a reward function, as shown in the formula. For example, if the control agent stops at a certain time point (t = 5) and outputs the correct answer, the reward will be 5 / T. In this case, if the control agent produces an incorrect answer, the reward will be -5 / T. Generally speaking, the goal of the control agent is to maximize the total reward, as shown in Equation (4).

[0081]

[0082] where R represents the sum of all rewards, and r represents the reward obtained at each time step.

[0083] In the solution of the present invention, the implementation details are as follows. In the training phase, the goal of the solution of the present invention is to minimize the prediction error of the MTTNet network while maximizing the reward obtained by the control agent. The solution of the present invention uses a gradient-based update method to optimize the accuracy of the proposed framework and optimizes the framework through a combination of accuracy loss and time decision penalty. MTTNet is optimized according to the cross-entropy loss, as shown in Equation (5):

[0084]

[0085] where θc denotes the parameters of the MTTNet network, and k denotes the number of categories. represents the corresponding log probability. The decision-making agent needs to find the optimal parameters to obtain the maximum reward. Therefore, the general loss function cannot be effectively optimized. The solution of the present invention needs to convert the original 0-1 loss function into a threshold loss function

[25] ,

[26] , as shown in formula (6).

[0086]

[0087] where θ r denotes the parameters of the agent. In addition, minimizing l r will lead to unstable gradient estimation, resulting in high variance of policy updates because each instance is a separate case. Therefore, the solution of the present invention adopts an additional baseline loss to estimate the observed reward, as shown in formula (7):

[0088]

[0089] where, θ b denotes the parameters of the baseline network, and b t is the estimated reward prediction value at each time stamp. The baseline is widely used to reduce the variance of gradient estimation while keeping the bias unchanged.

[0090] Therefore, the final loss function is as shown in the formula. Since the premature penalty loss minimizes the log probability, it corresponds to increasing the stopping probability. Therefore, the solution of the present invention adds a parameter β to balance accuracy and earliness. Increasing β means that the solution of the present invention emphasizes the penalty loss, resulting in early stopping, as shown in formula (8).

[0091] l(θ) = l c (θ c ) + βl r (θ r ) + l b (θ b ) (8).

[0092] Among them, these three terms respectively represent l c (θ c ) classification loss, l r (θ r ) reward loss, and l b (θ b ) baseline loss

[0093] To address the difficulties in early mapping of specific crops in a specific area or using an incremental strategy to determine the optimal crop recognition period, which leads to difficulties in early mapping of crops in a large area and the limitation of a large amount of data storage requirements brought by the incremental strategy, the solution of the present invention develops a framework for early crop recognition in a large area, which can cope with the challenges of crop phenological differences in different regions and find the optimal recognition time window for crops in different regions. This framework for early crop recognition in a large area combines the transformer attention mechanism and a reinforcement learning decision agent, and balances the early prediction and accuracy through the design of a new reward function, demonstrating its excellent early crop recognition ability on the main crop data of five different geographical counties in the United States. The results show that GEO-EARLIEST not only leads in recognition accuracy but also can significantly advance the recognition of crop species and determine their optimal recognition timing. Specifically, the overall accuracy (OA) of MTTNet (MTTnet is a high-performance.NET library for MTT-based communication) in Harvey County (HV, Kansas) is 0.924, and the F1 (i.e., recall rate) is 0.886. The OA in Traill County, North Dakota is 0.964, and the F1 is 0.933. The average earliest recognizable time of the crops is early July. It provides timely information support for crop management and realizes the early recognition of crops in a large area using Sentinel-2 satellite time series data and deep learning algorithms.

[0094] In some embodiments, the satellite time series data in collecting the satellite time series data of historical agricultural irrigation area crops in step S120 and collecting the satellite time series data of agricultural irrigation area crops to be classified in step S150 includes: collecting the data of the bottom-of-atmosphere reflectance image of Sentinel-2 Level-2A, and selecting the median composite of this data for the final time series data at a set number of days interval as the collected satellite time series data.

[0095] In some embodiments, the preprocessing in preprocessing the historical satellite time series data in step S130 and preprocessing the satellite time series data to be classified in step S160 includes: for the collected satellite time series data, using the linear interpolation method to obtain an image covering the entire time period with a time series of a set number of days to obtain the required time series.

[0096] In the solution of the present invention, the relevant descriptions of the study area and materials are as follows. Study area: In the early crop identification task, five geographically widespread counties in the United States were selected as the study area. Figure 6 It is a schematic image of crop distribution information. Among them, (a) represents the CDL crop cover product released by the US Department of Agriculture in 2021 [i.e., Geographical positioning of the six study sites overlayed on the 2021 Cropland Data Layer (CDL)], and (b) represents the crop distribution information of the selected five study areas (i.e., Study area information). As Figure 6 shown, these counties are Harvey County in Kansas, Haskell County in Texas, Randolph County in Indiana, Traill County in North Dakota, and Sherman County in Oregon respectively. These areas are located in different climate zones and geographical regions of the United States, with significant agricultural diversity. Harvey County is located in the central United States, with a temperate continental climate, and the main crops are corn, soybeans and wheat. Haskell County is located in the semi-arid area in the western part of Texas, and agricultural production depends on irrigation. The main crops are cotton and wheat. Randolph County is located in the eastern part of Indiana, with a mild and humid climate, and the main crops are corn and soybeans. Traill County is located in North Dakota, with a dry and cold climate, and the main crops include corn and beets. Its production cycle is restricted by the severe cold climate. Sherman County is located in the semi-arid area of Oregon, and agriculture is mainly based on wheat cultivation. The selected study sites cover a variety of climate conditions and planting systems, which provides an opportunity to evaluate the scalability of crop mapping methods in a large area.

[0097] The diverse climate conditions and crop types in these counties provide a wide range of test scenarios for the solution of the present invention, which helps to verify the applicability and robustness of the classification model in different environments. In addition, in the United States, except for winter wheat, the sowing season of the main crops usually starts from April and ends in June every year. On the other hand, the harvest is generally carried out between September and November, Figure 7 showing the typical sowing, growth and harvest periods of several main crops in the study area. By covering a wide range of growth conditions, these five counties provide a good opportunity to evaluate the generalization ability of the classification model in a large area. Figure 7It is a schematic image of the planting, growth, and harvest cycles of major crops, representing the growth calendars of different crops (i.e., Major crop’s typical planting, growing, and harvest period in USA).

[0098] Sentinel data is generally divided into 5 levels, namely Level-0, Level-1A, Level-1B, Level-1C, and Level-2A. The specific processing of each level is as follows: Level-0: raw data; Level-1A: geometrically coarsely corrected product containing meta-information; Level-1B: radiance product, embedded with a geometric model optimized by GCP but without corresponding geometric correction; Level-1C: atmospherically apparent reflectance product after orthorectification and sub-pixel level geometric fine correction; Level-2A: mainly contains bottom-of-atmosphere reflectance data after atmospheric correction. In the solution of the present invention, the relevant descriptions of the data and processing of Materials (such as Sentinel-2 Data and Processing) are as follows. The Sentinel-2 multispectral data used comes from the European Space Agency (ESA). Sentinel 2 includes Sentinel 2A and Sentinel 2B. The revisit period of a single satellite is 10 days, and it is increased to 5 days for the two satellites. The spatial resolution is 10m; Sentinel 2 data has 13 bands, covering the visible light to near-infrared part. In addition to the commonly used visible light and near-infrared bands, there are 4 red-edge bands, and the band width is small, with a high spectral resolution. Among them, the resolutions of Band 1, Band 9, and Band 10 are insufficient (such as less than 60m), and their contributions to the fine classification of crops are small; therefore, the solution of the present invention selects the remaining 10 bands as spectral features, including 3 visible light bands, 1 near-infrared band, 4 red-edge bands, and 2 short-wave infrared bands. In addition, the solution of the present invention uses the bottom-of-atmosphere reflectance images of Sentinel-2 Level-2A, which have been corrected for atmosphere and orthorectification, helping to provide more accurate and reliable surface reflectance data, thus better monitoring the agricultural situation.

[0099] The solution of the present invention collected all available images with cloud cover ≤ 20% during the period from January 1 to December 31, 2021, and removed the pixels contaminated by clouds. The solution of the present invention resampled the spatial resolution to 30m to match the spatial resolution of CDL. In addition, in order to eliminate the influence of the discontinuous time interval of Sentinel-2 data caused by different orbital observation dates of Sentinel-2, the median of Sentinel-2 observation data was selected to synthesize the final time series data with a 10-day interval. Linear interpolation was used to obtain images covering the entire time period using the 10-day time series, and a total of 36 time series were obtained. All necessary steps of data collection and preprocessing were performed using a platform (Google Earth Engine) dedicated to processing satellite images and other Earth observation data in the cloud.

[0100] In the solution of the present invention, the relevant description of the Ground Reference Data is as follows. CDL is used as the ground reference data in the solution of the present invention. CDL is a map of crop-specific land cover in the continental United States produced annually by the USDA NASS, providing more than 100 detailed category labels covering crops, pastures, forests, water, developed areas, etc. It is generated using medium-resolution satellite images and extensive agricultural ground truth with a spatial resolution of 30m. In addition to land cover categories, CDL also provides identification of cultivated land and non-cultivated land and the prediction confidence of a given classification. The solution of the present invention uses the CDL map of 2021 to collect labeled crop samples, and sets a confidence level of 90% to filter the CDL map to improve the sampling quality. Then, the solution of the present invention randomly sampled 10,000 sample points in each study area, extracted the corresponding crop types from the CDL map, and extracted the relevant spectral features from the time series images. The distribution of crop samples is as Figure 8 shown in Table 1 below. Specifically, crop types accounting for less than 5% of the total number of samples were merged into the "other" category. Figure 8 Table 1 is the table of the number of crop samples in each county in 2021 (i.e., The number of crop samples in each county in 2021).

[0101] In some embodiments, the specific process of iteratively training the adaptive early classification framework using the training set and the test set in step S140 to obtain the adaptive early classification model is as follows in the following exemplary description.

[0102] The following combines Figure 4The flowchart of an embodiment for obtaining the adaptive early classification model in the method of the present invention is shown, further illustrating the specific process of obtaining the adaptive early classification model in step S140, including: steps S410 to S440.

[0103] Step S410: Train the adaptive early classification framework using the training set and the test set, which is the training model of the adaptive early classification framework.

[0104] Step S420: Test the training model of the adaptive early classification framework using the test set to obtain the test results.

[0105] Step S430: Update the training model of the adaptive early classification framework according to the test results to obtain the updated model of the training model of the adaptive early classification framework.

[0106] Step S440: Iterate in this way until the error of the updated model of the training model of the adaptive early classification framework is less than the set threshold or the number of iterations reaches the set number, and use the updated model of the training model of the adaptive early classification framework as the adaptive early classification model, such as the GEO-EARLIEST model.

[0107] Figure 9 It is the overall flowchart of the method for early crop recognition in the agricultural irrigation canal area. In the solution of the present invention, the overall flowchart of the method (Methodology) for crop recognition in the agricultural irrigation canal area using the adaptive early classification framework is as Figure 9 shown. As Figure 9 shown, the method for crop recognition in the agricultural irrigation canal area using the adaptive early classification framework includes:

[0108] Step S1: Input the satellite time series data for training, perform data preprocessing on the satellite time series data, and divide the preprocessed satellite time series data into training set data and validation set data.

[0109] Step S2: Combine the TCN model and the self-attention mechanism as the basic classification network, fuse the reinforcement learning agent, and establish the GEO-EARLIEST model.

[0110] Step S3: Train the GEO-EARLIEST model using the training set data and the validation set data to obtain the trained GEO-EARLIEST model. Among them, when the error is less than the given threshold or the maximum number of iterations is satisfied, it is considered that the model training is completed.

[0111] Step S4: Use the trained GEO-EARLIEST model to predict the satellite time series data to be classified. Specifically, input the satellite time series to be classified into the GEO-EARLIEST model, and output the classification result and the earliest identifiable time node.

[0112] In the solution of the present invention, the relevant description of the growth characteristics of crop samples is as follows. Figure 14 It is a schematic diagram of the spectral characteristics of crops in different counties. For several major crops, their time series spectral curves are as Figure 14 shown, and each color represents a crop. Figure 14 It shows the spectral characteristic changes of different crops in different counties on Sentinel-2 time series images. By analyzing the data of 5 counties (Haskell, Harvey, Traill, Randolph, and Sherman), the solution of the present invention finds that there are obvious differences in the spectral performance of the same crop in different counties. For example, the cotton in Haskell and Randolph counties shows significant differences in reflectance in the short-wave infrared (SWIR) band, especially in the SWIR 1 band, where the reflectance in Haskell County is higher and that in Mississippi County is lower, indicating that the SWIR band is important for distinguishing crops in different regions. However, in the Red Edge band and the near-infrared (NIR) band, the spectral curves of cotton in Haskell and Mississippi counties show a large overlap, indicating that these bands are more effective in capturing the common characteristics of crops. On the other hand, even in the same county, different crops may show similarities in certain bands. For example, in Traill and Randolph counties, the NIR band spectral curves of corn and soybeans almost overlap, indicating that it is challenging to distinguish these two crops in these bands. Generally speaking, the SWIR band has a strong ability to distinguish different crops in different counties, while the NIR and Red Edge bands are more suitable for describing the common characteristics of crops. Therefore, the model needs to combine multiple band and time step features to improve the classification accuracy, especially when dealing with crops in different counties.

[0113] In the solution of the present invention, the relevant description of the earliest identifiable time of different crops is as follows. Figure 15 It is a schematic diagram of the earliest identifiable time of different crops, Figure 15 which describes that the stop dates of different crop types are different, and the classification of all crops stops during the agricultural relevant period between sowing and harvesting. Figure 15The earliest recognizable time for each crop in Trail County (upper left), and the left lower represents the confusion matrix of classification. The right side represents the recognition time map and classification confusion matrix of Harvey County. The average classification time for most crops (i.e., soybeans, winter wheat, corn, sugar beets) is in the middle of the season. All crops are classified before the harvest period, and in the case of understanding the local crop calendar, domain experts usually regard the harvest period as the end date of the series applicable only to the accuracy classifier.

[0114] In the early identification of crops, the growth cycle and phenological characteristics of each crop will affect the change of its spectral reflectance, thus determining the timing of model identification. Taking corn, wheat and soybeans in Harvey County as examples: For cotton during its leaf expansion stage, the leaf area increases rapidly, and the canopy structure becomes more complex. Especially the reflectance in the near-infrared band increases significantly, which enables the model to use these spectral characteristics to identify cotton earlier. Due to its relatively short growth cycle, wheat shows significant spectral changes during the early tillering and jointing stages. In particular, the rapid increase in the value of the normalized difference vegetation index (NDVI) and the reflectance in the red-edge band enable the model to distinguish wheat from other crops earlier at this time. In contrast, sorghum has a relatively long growth cycle. Its early spectral characteristics are similar to those of other crops, but with the arrival of the jointing stage, the canopy coverage increases, and the reflectance in the near-infrared and short-wave infrared bands changes significantly. The model can more accurately identify sorghum at this stage. The changes in these spectral characteristics are closely related to the growth cycle and physiological characteristics of the crops. By capturing the spectral differences at these critical periods, the model can achieve early identification during different growth stages of the crops.

[0115] In the solution of the present invention, the relevant description of Early-Season Crop Mapping for Study area is as follows. Based on the results of the earliest recognizable time of crops obtained from the early classification framework, the solution of the present invention finds that there are obvious differences in the earliest recognizable time of different crops. This inspires the solution of the present invention to design two early mapping strategies: one is for a single major crop, making full use of its earliest recognizable time for fine mapping; the other is for a multi-crop area, selecting a unified time point for mapping to balance practicality. These two strategies respectively correspond to different needs in practical applications: some areas need to focus on the early identification of a certain major crop, while some areas need to monitor the distribution of multiple crops simultaneously.

[0116] In the solution of the present invention, the relevant description of Single-Crop Mapping is as follows. Figure 16 Schematic diagrams of the distributions of different crops. Figure 16The day of year (DOY) representing different crop selections in Harvey County, the early mapping results, predicted represents the predicted result map, and Difference map represents the error map of the classification results and distribution products. The solution of the present invention takes Harvey County as an example to verify the fine mapping strategy for specific crops. Based on the results of the early classification framework, the solution of the present invention uses the classification models in the framework to map winter wheat, corn, and soybeans with different cut-off times respectively: winter wheat uses DOY 150, corn uses DOY 180, and soybeans use DOY 180. It should be noted that the model training only uses the time series data before the respective cut-off dates in 2022. The mapping results are as Figure 16 shown, and satisfactory classification accuracies can be obtained at these early time points. Specifically, the accuracy of winter wheat reached 93.1%, the accuracy of corn reached 96.2%, and the accuracy of soybeans reached 94.3%. As can be seen from Figure 16 , the model can accurately identify the spatial distribution of various crops, with fewer and more scattered misclassified areas. The growth cycle of winter wheat starts from the autumn of the previous year. Since the data input in the solution of the present invention starts from January of the survey year, the earliest recognizable time of winter wheat is delayed. Compared with the mapping results using the data of the complete growing season, the early mapping strategy of the solution of the present invention reaches a similar accuracy level only using the data in the early stage of the growing season. This mapping strategy based on specific crop times has significant advantages: First, it maximally advances the acquisition time of crop distribution information; Second, by selecting the optimal time points for different crops, it ensures a high classification accuracy; Finally, this strategy is particularly suitable for areas that need to prioritize the monitoring of a certain crop and can provide timely support for agricultural management decisions.

[0117] In the solution of the present invention, the relevant description of multi-crop mapping is as follows. Figure 17 It is a schematic diagram of multiple crops in the early stage. Figure 17 The day of year (DOY) representing the average earliest recognizable time in Traill County, the mapping results of different models, predicted represents the predicted result map, and Difference map represents the error map of the classification results and distribution products. In agricultural monitoring, there are some areas that need to know the distribution of multiple crops at the same time. To meet this demand, the solution of the present invention designs a multi-crop mapping strategy at a unified time point, and selects a time node that comprehensively considers the recognizability of each crop for synchronous mapping. Taking Traill County as an example, the solution of the present invention selects the day of year (DOY) 210 as the mapping time point and uses the time series data before this time point in 2022 for mapping. The mapping results are as Figure 17As shown, at day of year (DOY) 210, the vast majority of crops can be accurately identified, achieving good classification accuracy. However, due to the relatively late phenological development process of sugar beet, it needs to wait until DOY 230 to reach a high recognition accuracy. Although this strategy does not reach the earliest recognizable time for each crop, it realizes the synchronous monitoring of most crops and meets the demand for synchronous understanding of the distribution of multiple crops in practical applications.

[0118] In the solution of the present invention, the relevant description of geological exploration interpretation (GEO - EARLIEST interpretation) is as follows. Figure 18 It is a schematic diagram for feature interpretability analysis. Figure 18 In it, a(1)-a(6) represent the model feature importance of different crops in different Traill counties, while b(1)-b(3) represent the model feature importance of various crops in Harvey county. The MTTNet architecture of the solution of the present invention effectively captures the local changes in the early stage of crop growth through multi - scale TCN blocks, and at the same time, the Transformer block further extracts the long - term dependencies in the time series, which enables the model to accurately identify crop types at the beginning of the growing season. Compared with traditional RNN or LSTM, the combination of TCN and Transformer shows stronger time - series feature extraction ability when dealing with long time series, thus obtaining significant discrimination in the early stage of crops. In the time domain, the features of the crop growth period are more concerned, especially the features close to the maturity stage. In addition, the solution of the present invention also notes that the important time window of winter wheat is within 90 - 180 DOY (day of year), earlier than other crops. This time shift is mainly because winter wheat is different from other crops. Other crops are usually sown in late spring, while winter wheat is sown in autumn before sowing and harvested in summer of the same year. The decision agent of the solution of the present invention selects to predict winter wheat at an earlier time of the year. On the other hand, the important time - series features of corn and soybeans are both between 150 - 250 DOY, which may be due to the similar phenological periods of these two crops, and the decision agent selects to classify corn and soybeans around the 190th DOY. More importantly, the solution of the present invention finds that compared with other crops, the important time period of cotton has a longer span, which may be because the cotton planting / harvest time varies greatly in different counties (as Figure 18 shown), and the common features are not very concentrated during the cotton growth period, which may be the reason for the model to determine different recognition times for cotton in different regions.

[0119] In the solution of the present invention, the relevant description of the comparison to non-early classification models is as follows. The solution of the present invention compares the GEO-EARLIEST model with various existing models (such as Random Forest, TempCNN, MS-ResNet, InceptionTime, LSTM, Transformer, etc.). Taking the results of Trail County as an example, the classification performance is measured by accuracy and kappa score, while the early prediction performance is evaluated by the t / T index, that is, what proportion of the data in the entire time series is used by the model for prediction. The kappa value is an index to measure the consistency of two judges, and its value range is from 0 to 1. The traditional "only accuracy" model always classifies at the end of the time series (t = T), while GEO-EARLIEST can significantly reduce the required sequence length while maintaining the same accuracy. The results show that the prediction ratio of the GEO-EARLIEST model is t / T = 0.54 ± 0.07, which means that only 54% ± 7% of the time series data is required to achieve accurate classification. This not only improves the advance of prediction but also saves the time cost of data download, storage, and processing. In contrast, other methods need to use the complete time series (until December 30th) to complete the prediction, while GEO-EARLIEST can complete the prediction at 6th June ± 28 days, showing higher efficiency and stability, as Figure 19 shown in Table 2. Figure 19 Table 2 is a comparison table for different models. Figure 19 It represents the comparison between the model GEO-Earliest of the present invention and other models. ERALINESS represents the earliness, that is, the required time series length, and average date represents the specific date.

[0120] In the solution of the present invention, the related description of model adaptability is as follows. Differences in climate conditions, light duration, temperature changes, etc. in different regions will significantly affect the growth patterns and spectral characteristics of crops. Such differences pose challenges to crop classification models, which need to have sufficient adaptability to maintain stable classification performance in different regions. By combining the Temporal Convolutional Network (TCN) and the Transformer model, GEO-EARLIEST can flexibly handle these regional differences and capture local and global time series features during the crop growth process. The reinforcement learning agent can dynamically adjust the classification stopping time according to the specific crop growth conditions in each region, thereby optimizing the classification effect. To verify the cross-regional adaptability of the model, the solution of the present invention selected two significantly different study areas, Harvey County and Traill County, for experiments. These two regions differ in climate conditions, light duration, and agricultural management methods. The experimental results show that GEO-EARLIEST can maintain a high classification accuracy in both regions and successfully adapt to the growth patterns of crops in different regions. This indicates that the model has strong cross-regional self-adaptability and can maintain robust classification performance under different environmental conditions.

[0121] In the solution of the present invention, the related description of limitations and future work is as follows. Although GEO-EARLIEST shows potential in early crop identification, it still faces some challenges. First, the model's dependence on high-quality labeled data may cause application difficulties in regions with scarce data. Second, the current method may have limitations in dealing with crop species with sparse category labels. For example, in Harvey County, the number of cotton samples is very small, resulting in difficulties in early identification because deep learning models usually impose less penalty on categories with sparse labels. In addition, remote sensing data often faces problems such as data missing caused by cloud cover, atmospheric interference, etc., which may significantly affect the model performance. In particular, data missing in key growth stages may cause the model to miss the best decision-making opportunity, and the robustness of the model to such data missing needs to be further improved. Future research should focus on solving these problems to enhance the applicability and reliability of the model in actual agricultural scenarios.

[0122] In summary, in the solution of the present invention, early crop identification is crucial for scientific agricultural planning and management. In this work, the solution of the present invention proposes an adaptive early classification framework GEO-EARLIEST based on the idea of deep reinforcement learning. The solution of the present invention obtains the earliest recognizable time of each crop by training with 21-year time series images, and generates the crop coverage map of 2022 only using part of the sequence in the early stage of crop growth. The solution of the present invention solves the unsupervised nature of early classification through a neural network-based method by reinforcement learning. GEO-EARLIEST directly models multiple objectives of accuracy and earliness, allowing them to be jointly optimized under conflicting trends. During the classification process, the feature extraction network of the solution of the present invention learns the multi-scale feature representation of the multivariate time series, and then uses it to inform the early stopping decision and prediction label. When it observes a signal, GEO-EARLIEST effectively learns to stop, otherwise it waits, thus effectively fine-tuning the reactive signal capture. In the early stage of crop growth, the model usually makes a waiting action until an important phenological event or a reliable time node is observed, and then makes a stopping action to output the final classification result. The solution of the present invention applies the proposed model to classify the main crops in five counties in different regions of the United States to test its robustness. The solution of the present invention evaluates the model by comparing it with several other methods (including Random Forest, Temp CNN, and LSTM, etc.). The results show that the model of the solution of the present invention shows good accuracy and earliness in multiple regions. It shows that the model can generate accurate and timely crop maps on a large scale.

[0123] Adopting the technical solution of this embodiment, an adaptive early classification (i.e., GEO-EARLIEST) framework composed of a basic classification network MTTNet and a decision network Agent is set up, where the MTTNet network is composed of multi-scale TCN blocks and Transformer blocks; after collecting historical satellite observation data (such as the median of Sentinel-2 observation data to synthesize the final time series data at 10-day intervals) and performing data preprocessing, sample data is obtained; the sample data is divided into a training set and a test set, and the GEO-EARLIEST framework is trained using the training set to obtain a training model of the GEO-EARLIEST framework. The training model of the GEO-EARLIEST framework is tested and updated using the test set to obtain a test model of the GEO-EARLIEST framework, which is used as the GEO-EARLIEST model to classify the satellite time series to be classified, and classification results and time nodes are obtained; thus, by setting up the GEO-EARLIEST framework and training with historical data to obtain a model, it is convenient to achieve early identification of large areas, such as accurately identifying the classification results and the earliest recognizable time nodes of irrigation areas, so that targeted irrigation is beneficial to water conservation.

[0124] In the solution of the present invention, by processing historical satellite observation data (such as Sentinel-2 data), the model can identify irrigation areas and provide the earliest recognizable time nodes. This function helps decision-makers timely grasp the water demand of irrigation areas and reduce the over-supply of water resources. The classification results clearly indicate the areas that need irrigation, avoiding unnecessary comprehensive irrigation, thereby reducing water resource waste. The earliest recognizable time node allows for the early deployment of irrigation plans to provide water support during the critical period when crops need water. Suppose a certain area uses this solution for irrigation management. By processing historical satellite observation data, the system identifies that some areas will enter the water demand period within the next two weeks and provides specific time nodes. Irrigation managers can arrange irrigation plans precisely based on this information and supply water only to the areas that need water. For example, in the dry season, if the irrigation area is reduced by 20% due to model prediction, and each irrigation requires 100 cubic meters of water per mu, for 1000 mu of land, the water-saving effect can reach 20,000 cubic meters.

[0125] According to an embodiment of the present invention, there is also provided a crop recognition device for an agricultural irrigation canal area corresponding to a crop recognition method for an agricultural irrigation canal area. Refer to Figure 5 The structural schematic diagram of an embodiment of the device of the present invention shown. The crop recognition device for an agricultural irrigation canal area is applied to recognize the types and growth time nodes of crops in an agricultural irrigation area. The crop recognition device for an agricultural irrigation canal area includes: an acquisition unit 102 and a control unit 104.

[0126] Among them, the control unit 104 is configured to pre-build an adaptive early classification framework, namely the GEO-EARLIEST framework, based on the MTTNet network and the decision network. The specific functions and processes of the control unit 104 are described in step S110.

[0127] The acquisition unit 102 is configured to pre-collect the satellite time series data of historical agricultural irrigation area crops, denoted as historical satellite time series data. The specific functions and processes of the acquisition unit 102 are described in step S120.

[0128] The control unit 104 is further configured to preprocess the historical satellite time series data to obtain sample data; and divide the sample data into a training set and a test set. The specific functions and processes of the control unit 104 are also described in step S130.

[0129] The control unit 104 is further configured to iteratively train the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model, such as the GEO-EARLIEST model. The specific functions and processes of the control unit 104 are also described in step S140.

[0130] The acquisition unit 102 is further configured to collect the satellite time series data of the agricultural irrigation area crops to be classified, denoted as the satellite time series data to be classified. The specific functions and processes of the acquisition unit 102 are also described in step S150.

[0131] The control unit 104 is further configured to preprocess the satellite time series data to be classified and then classify it using the adaptive early classification model to obtain the classification result and time node corresponding to the satellite time series data to be classified, so as to realize the identification of the type and growth time node of the agricultural irrigation area crops to be classified. The specific functions and processes of the control unit 104 are also described in step S160.

[0132] In the solution of the present invention, an adaptive early classification framework combining deep learning and reinforcement learning (i.e., GEO-EARLIEST) is developed, and time series and samples for the whole year of 2021 are used to train the adaptive early classification framework to determine the earliest recognizable time for each crop. Specifically, the solution of the present invention uses the data of historical years to train and obtain the earliest recognizable time for each crop, and then applies the classification model in the framework to the images at the beginning of the growing season in 2022 to obtain the crop distribution map in 2022. The solution of the present invention compares the early mapping results with other methods using full-length sequences. The main contributions of the solution of the present invention are: (1) A general early recognition framework for large regional scales is proposed, breaking through the limitation that existing methods are only applicable to specific regions or crops; (2) Compared with various state-of-the-art models, it shows excellent performance in terms of classification accuracy, mapping result quality, and earliness; (3) Through experiments in different spatial regions, the model of the solution of the present invention shows good adaptability within a large regional scope.

[0133] Since the processing and functions implemented by the device in this embodiment are basically corresponding to the embodiments, principles, and examples of the foregoing method, for the details not described in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and details will not be repeated here.

[0134] According to an embodiment of the present invention, a terminal corresponding to the crop recognition device for agricultural irrigation canal areas is further provided. The terminal may include: the crop recognition device for agricultural irrigation canal areas described above.

[0135] Since the processing and functions implemented by the terminal in this embodiment are basically corresponding to the embodiments, principles, and examples of the foregoing device, for the details not described in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and details will not be repeated here.

[0136] According to an embodiment of the present invention, a computer program product corresponding to the crop recognition method for agricultural irrigation canal areas is further provided, including a computer program, and when the computer program is executed by a processor, the steps of the crop recognition method for agricultural irrigation canal areas described above are implemented.

[0137] Since the processing and functions implemented by the product in this embodiment are basically corresponding to the embodiments, principles, and examples of the foregoing method, for the details not described in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and details will not be repeated here.

[0138] According to an embodiment of the present invention, a storage medium corresponding to the crop recognition method for agricultural irrigation canal areas is further provided. The storage medium includes a stored program, and when the program runs, it controls the device where the storage medium is located to execute the steps of the crop recognition method for agricultural irrigation canal areas described above.

[0139] Since the processing and functions implemented by the storage medium of this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing method, for the parts not described in detail in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and details are not repeated here.

[0140] In summary, it is easy for those skilled in the art to understand that, on the premise of no conflict, the above-mentioned advantageous methods can be freely combined and superimposed.

[0141] The above are only the embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for crop identification in an agricultural irrigation canal area, characterized in that: The method is applied to identify the types and growth time nodes of crops in agricultural irrigation areas. The crop identification method in agricultural irrigation canal areas includes: Based on the MTTNet network and decision network, an adaptive early classification framework is built; Collect historical satellite time series data of crops in agricultural irrigation areas, recorded as historical satellite time series data; Preprocessing the historical satellite time series data to obtain sample data; and dividing the sample data into a training set and a test set; Iteratively training the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model; Collect satellite time series data of crops in agricultural irrigation areas to be classified, and record them as satellite time series data to be classified; After preprocessing the satellite time series data to be classified, the adaptive early classification model is used to perform classification, and the classification results and time nodes corresponding to the satellite time series data to be classified are obtained, so as to realize the identification of the types and growth time nodes of the crops in the agricultural irrigation area to be classified.

2. The crop identification method in an agricultural irrigation canal area according to claim 1, characterized in that: Based on the MTTNet network and decision network, an adaptive early classification framework is built, including: Based on the TCN model and Transformer self-attention mechanism, set up the MTTNet network; Based on the MTTNet network, the decision network described in reinforcement learning is integrated to obtain the required adaptive early classification framework.

3. The crop identification method in an agricultural irrigation canal area according to claim 2, characterized in that: Based on the TCN model and Transformer self-attention mechanism, set up the MTTNet network, including: Based on the TCN model and Transformer self-attention mechanism, we use deep separable convolution and introduce the GELU activation function to obtain the first convolutional network. Based on the first convolutional network, a residual structure is added, and at least one of a layer normalization technique and a random dropout technique is adopted to obtain a second convolutional network as the required MTTNet network.

4. The crop identification method in an agricultural irrigation canal area according to claim 1, characterized in that: Collect satellite time series data of crops in historical agricultural irrigation areas and satellite time series data of crops in agricultural irrigation areas to be classified, including: The data of the atmospheric bottom reflectance image of Sentinel-2Level-2A are collected, and the median value of the data is selected to synthesize the final time series data of the set day interval as the collected satellite time series data.

5. The crop identification method in an agricultural irrigation canal area according to claim 1 or 4, characterized in that: The preprocessing of the historical satellite time series data and the preprocessing of the satellite time series data to be classified includes: For the collected satellite time series data, the linear interpolation method is used to obtain images covering the entire time period using the time series of the set number of days to obtain the required time series.

6. The crop identification method in an agricultural irrigation canal area according to any one of claims 1 to 5, characterized in that: The adaptive early classification framework is iteratively trained using the training set and the test set to obtain an adaptive early classification model, including: Using the training set and the test set to train the adaptive early classification framework, the training model of the adaptive early classification framework; Using the test set to test the training model of the adaptive early classification framework to obtain a test result; The training model of the adaptive early classification framework is updated according to the test result to obtain an updated model of the training model of the adaptive early classification framework; In this way, the update model of the training model of the adaptive early classification framework is used as the adaptive early classification model by iterating until the error of the update model of the training model of the adaptive early classification framework is less than a set threshold or the number of iterations reaches a set number.

7. A crop identification device for an agricultural irrigation canal area, characterized in that: The device is used to identify the types and growth time nodes of crops in agricultural irrigation areas. The crop identification device in the agricultural irrigation canal area includes: A control unit, configured to build an adaptive early classification framework based on the MTTNet network and the decision network; An acquisition unit is configured to collect historical satellite time series data of crops in agricultural irrigation areas, recorded as historical satellite time series data; The control unit is further configured to pre-process the historical satellite time series data to obtain sample data; and divide the sample data into a training set and a test set; The control unit is further configured to iteratively train the adaptive early classification framework using the training set and the test set to obtain an adaptive early classification model; The acquisition unit is further configured to collect satellite time series data of crops in the agricultural irrigation area to be classified, recorded as satellite time series data to be classified; The control unit is further configured to pre-process the satellite time series data to be classified, and then classify the data using the adaptive early classification model to obtain classification results and time nodes corresponding to the satellite time series data to be classified, so as to identify the types and growth time nodes of crops in the agricultural irrigation area to be classified.

8. A terminal, characterized in that: include: The crop identification device for an agricultural irrigation canal area as claimed in claim 7.

9. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the crop identification method for the agricultural irrigation canal area according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the crop identification method for an agricultural irrigation canal area according to any one of claims 1 to 6 are implemented.

Images