Crop early classification method, device, equipment and medium

Abstract

The invention relates to a crop early classification method, device and equipment and a medium, and the method comprises the steps: dividing remote sensing waveband time sequence data corresponding to a historical observation year into two parts in a time dimension according to an observation point e of a target observation year, so as to determine first remote sensing waveband time sequence data and second waveband time sequence data; inputting the third remote sensing wave band time sequence data with the time dimension of [0, e] in the target observation year into the optimal crop growth prediction model to generate fourth remote sensing wave band time sequence data with the time dimension of [e, 36] in the target observation year; and splicing the third remote sensing wave band time sequence data and the fourth remote sensing wave band time sequence data in a time dimension to determine spliced remote sensing wave band time sequence data, and inputting the spliced remote sensing wave band time sequence data into a crop classification model to determine a crop category of the target crop and a corresponding phenological period thereof. The problem that early-stage observation wave band feature information is insufficient can be effectively solved.

Description

Technical Field

[0001] The present application relates to the field of remote sensing agricultural information, and in particular to an early classification method for crops, a corresponding device, an electronic device and a computer-readable storage medium. Background Art

[0002] Early classification of crops plays a central role in modern agriculture, and its importance is reflected in multiple dimensions. First, early classification helps improve the accuracy of crop management. By timely identifying and classifying different crops in the early stages of crop growth, farmers can implement corresponding management measures in a targeted manner, such as precision fertilization, irrigation, and pest and disease control. This approach can not only effectively improve crop yield and quality, but also reduce resource waste and achieve sustainable development. Second, early classification provides reliable data support for agricultural decision-making. In the context of responding to climate change and market fluctuations, decision makers need to make flexible adjustments based on real-time data. Early classification can monitor crop growth in real time, enabling policymakers to optimize resource allocation in a timely manner and enhance the risk resistance of agricultural production. In addition, early classification plays a key role in promoting precision agriculture. Using remote sensing technology and big data analysis, early classification can not only improve the level of agricultural informatization, but also promote the efficient use of resources and maximize production benefits. In the field of agricultural insurance, accurate crop classification can improve the accuracy of risk assessment, thereby formulating reasonable insurance strategies.

[0003] At present, due to differences in the geographical and climatic environments in which crops grow, there may be large differences in the growth patterns of crops. In the early classification research tasks of crops based on deep learning, there are problems such as too little band feature information in the early observation period, insufficient effective identification features, resulting in low classification accuracy, late identification date, and poor stability of the classification model. In order to achieve earlyness, traditional technology can only improve earlyness by losing the step size of the time dimension, and the classification accuracy will also drop significantly. This balance problem limits the effectiveness of traditional early classification methods based on multi-temporal data.

[0004] To sum up, in the early classification research tasks of crops based on deep learning in the existing technology, there are problems such as too little band feature information in the early stage of observation, insufficient effective identification features, resulting in low classification accuracy, late identification date and poor stability of the classification model. The applicant has made corresponding explorations to solve this problem. Summary of the invention

[0005] The purpose of the present application is to solve the above-mentioned problems and to provide a method for early classification of crops, a corresponding device, an electronic device and a computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A method for early classification of crops proposed to meet one of the purposes of this application, including:

[0008] Obtain the annual time-series remote sensing images containing the target crop within the historical observation years to extract the time-series data of remote sensing bands of the target crop in multiple phenological periods within the historical observation years. Among them, the time dimension of the time-series data of remote sensing bands is [0, 36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days long;

[0009] According to the observation point e in the target observation year, divide the time-series data of remote sensing bands corresponding to the historical observation years into two parts in the time dimension to determine the first time-series data of remote sensing bands and the second time-series data of remote sensing bands. Among them, the time dimension of the first time-series data of remote sensing bands is [0, e], and the time dimension of the second time-series data of remote sensing bands is [e, 36];

[0010] Use the first time-series data of remote sensing bands as the training set and the second time-series data of remote sensing bands as the labels to train the optimized crop growth prediction model until the model reaches the convergence state to determine the best crop growth prediction model;

[0011] Input the third time-series data of remote sensing bands with the time dimension of [0, e] in the target observation year into the best crop growth prediction model to generate the fourth time-series data of remote sensing bands with the time dimension of [e, 36] in the target observation year;

[0012] Stitch the third time-series data of remote sensing bands and the fourth time-series data of remote sensing bands in the time dimension to determine the stitched time-series data of remote sensing bands, and input the stitched time-series data of remote sensing bands into the crop classification model that has been trained to the convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period to complete the early classification of the target crop.

[0013] Optionally, the time-series data of remote sensing bands include the basic bands of optical sensors, the basic bands of radar sensors, and vegetation spectral indices. The basic bands of optical sensors include the red light band, the green light band, the blue light band, the near-infrared band, and the short-wave infrared band. The basic bands of radar sensors include the vertical polarization parameter and the horizontal polarization parameter. The vegetation spectral indices include the normalized difference vegetation index, the enhanced vegetation index, the green canopy vegetation index, the soil moisture index, and the radar vegetation index;

[0014] The basic network architecture of the optimized crop growth prediction model includes a Linear model with a time series data decomposition module, a convolutional neural network with a deep temporal convolutional network and causal convolution, a long short-term memory network with a self-attention module and a residual connection module, or a Transformer model with an autoregressive module;

[0015] The basic network architecture of the crop classification model includes any one of a convolutional neural network, a long short-term memory network, or a Transformer model;

[0016] The target crops include one or more of rice, corn, and soybeans, and the phenological periods include one or more of the sowing period, tillering period, seedling period, jointing period, branching period, heading period, filling period, pod-setting period, and maturity period.

[0017] Optionally, for the Linear model with a time series data decomposition module, the original input sequence is decomposed into a seasonal component and a trend component. The moving average method is used to extract the seasonal and trend components in the data, and the seasonal and trend components are predicted separately through a linear layer to improve the ability to capture long-term trends and periodic changes;

[0018] For the convolutional neural network with a deep temporal convolutional network and causal convolution, the causal convolution ensures that the model only uses historical information for prediction, and the temporal module and the deep temporal convolutional network are used to effectively capture multi-scale temporal features. Finally, the prediction is made through a linear layer to improve the accuracy of crop growth prediction;

[0019] For the long short-term memory network with a self-attention module and a residual connection module, the self-attention module enhances the ability to capture long-term dependencies by dynamically adjusting the weights of time steps, and the gradient disappearance is alleviated through the residual connection module;

[0020] For the Transformer model with an autoregressive module, multiple prediction results are gradually generated and the prediction of the previous step is used as the input for the next time step. Each prediction step depends on the previous output. After being mapped to a high dimension through an expansion layer, it further provides an estimate of future moments.

[0021] Optionally, the steps for determining the normalized difference vegetation index include:

[0022] Obtain the first TOA reflectance value corresponding to the near-infrared band and the second TOA reflectance value corresponding to the red band in the time series remote sensing image;

[0023] Calculate and determine a first difference between the first TOA reflectance value and the second TOA reflectance value, and calculate and determine a first sum value between the first TOA reflectance value and the second TOA reflectance value;

[0024] Determine the normalized difference vegetation index according to a first ratio between the first difference and the first sum value.

[0025] Optionally, the step of determining the enhanced vegetation index includes:

[0026] Obtain a first TOA reflectance value corresponding to the near-infrared band, a second TOA reflectance value corresponding to the red band, and a third TOA reflectance value corresponding to the blue band in the time-series remote sensing image;

[0027] Calculate and determine a first product between the first TOA reflectance value and the number 6, and calculate and determine a second product between the third TOA reflectance value and the number 7.5;

[0028] Calculate and determine a second sum value between the first TOA reflectance value and the first product, calculate and determine a second difference between the second sum value and the second product, and calculate and determine a third sum value between the second difference and the number 1;

[0029] Calculate and determine a first difference between the first TOA reflectance value and the second TOA reflectance value, calculate and determine a second ratio between the first difference and the third sum value, and determine the enhanced vegetation index according to a third product between the second ratio and the number 2.5.

[0030] Optionally, the step of determining the green canopy vegetation index and the soil moisture index includes:

[0031] Obtain a first TOA reflectance value corresponding to the near-infrared band, a fourth TOA reflectance value corresponding to the green band, and a fifth TOA reflectance value corresponding to the short-wave infrared band in the time-series remote sensing image;

[0032] Calculate and determine a third ratio between the first TOA reflectance value and the fourth TOA reflectance value, and determine the green canopy vegetation index according to a third difference between the third ratio and the number 1; calculate and determine a fourth difference between the first TOA reflectance value and the fifth TOA reflectance value, calculate and determine a fourth sum value between the first TOA reflectance value and the fifth TOA reflectance value, and determine the soil moisture index according to a fourth ratio between the fourth difference and the fourth sum value.

[0033] Optionally, the step of determining the radar vegetation index includes:

[0034] Obtain the first polarization backscattering coefficient for horizontal transmission and vertical reception, the second polarization backscattering coefficient for horizontal transmission and horizontal reception, and the third polarization backscattering coefficient for vertical transmission and vertical reception;

[0035] Calculate and determine the fourth product between the first polarization backscattering coefficient and the value 8, calculate and determine the fifth product between the first polarization backscattering coefficient and the value 2, and calculate and determine the fifth sum value among the fifth product, the second polarization backscattering coefficient, and the third polarization backscattering coefficient;

[0036] Determine the radar vegetation index according to the fifth ratio between the fourth product and the fifth sum value.

[0037] A crop early classification device provided for another purpose of this application, comprising:

[0038] A historical remote sensing band acquisition module, which acquires the annual time-series remote sensing images containing the target crop within the historical observation year to extract the time-series data of the remote sensing bands of the target crop in multiple phenological periods within the historical observation year. Among them, the time dimension of the time-series data of the remote sensing bands is [0, 36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days;

[0039] A remote sensing band data division module, which is set to divide the time-series data of the remote sensing bands corresponding to the historical observation year into two parts in the time dimension according to the observation point e of the target observation year to determine the first time-series data of the remote sensing bands and the second time-series data of the remote sensing bands. Among them, the time dimension of the first time-series data of the remote sensing bands is [0, e], and the time dimension of the second time-series data of the remote sensing bands is [e, 36];

[0040] A growth prediction model training module, which is set to use the first time-series data of the remote sensing bands as the training set and the second time-series data of the remote sensing bands as the label to train the optimized crop growth prediction model until the model reaches the convergence state to determine the optimal crop growth prediction model;

[0041] A remote sensing band data prediction module, which inputs the third time-series data of the remote sensing bands with the time dimension of [0, e] within the target observation year into the optimal crop growth prediction model to generate the fourth time-series data of the remote sensing bands with the time dimension of [e, 36] within the target observation year;

[0042] The crop early classification module is configured to splice the third remote sensing band time series data with the fourth remote sensing band time series data in the time dimension to determine the spliced ​​remote sensing band time series data, and input the spliced ​​remote sensing band time series data into a crop classification model that has been trained to a convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period, so as to complete the early classification of the target crop.

[0043] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the early crop classification method described in the present application.

[0044] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the early classification method of crops in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0045] Compared with the prior art, this application aims at the problems in the prior art of early classification research tasks of crops based on deep learning, such as too little band feature information in the early observation period, insufficient effective recognition features, resulting in low classification accuracy, late recognition date, and poor stability of the classification model. This application includes but is not limited to the following beneficial effects:

[0046] First, the application can significantly improve the classification accuracy. By generating more band features, the application effectively solves the problem of insufficient band feature information in the early observation. In traditional methods, due to the limitation of band feature dimensions, the early classification accuracy is low, and the application can generate richer feature information based on the existing bands, thereby improving the accuracy of early crop classification.

[0047] Secondly, this application solves the problem of delayed classification date. Traditional early classification methods for crops often need to sacrifice time steps to improve earlyness, which leads to reduced classification accuracy. This application uses a time series model to conduct in-depth analysis of spectral trends and the application of feature screening mechanisms, which can not only generate continuous time series spectral features, but also will not delay the classification date. This can significantly improve the accuracy of classification while ensuring the earlyness of classification;

[0048] Third, this application enhances the stability and adaptability of the model. Traditional models are usually trained based on data from specific regions or seasons, which can easily lead to poor generalization of the model and inability to adapt to crop growth patterns under different geographical and climatic conditions. The multi-source remote sensing time series data generation framework provided by this application can effectively improve the adaptability and stability of the model to crop classification tasks in different environments by deeply processing data at multiple time points;

[0049] Fourthly, the present application makes full use of multi-source remote sensing time-series data, which can not only generate the trend of remote sensing time-series data for any time period in the target observation year, but also flexibly adapt to different climate environments and crop growth patterns. This method can maintain a high recognition accuracy in a changing environment and has a strong application breadth.

[0050] Furthermore, the present application can further improve the accuracy and stability of the crop classification model without delaying the classification date, and can flexibly predict the trend of multi-source remote sensing time-series data for any time period in the target observation year. Description of the Drawings

[0051] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0052] Figure 1 is a schematic flowchart of the method for early crop classification in the embodiments of the present application;

[0053] Figure 2 is an exemplary structure of the early crop classification system in the embodiments of the present application;

[0054] Figure 3 is a schematic diagram showing the distribution of the dataset samples of the method for early crop classification in the embodiments of the present application in the time dimension and the space dimension;

[0055] Figure 4 is a schematic diagram of 12-band data of three main crops, namely rice, soybean and corn, in the embodiments of the present application;

[0056] Figure 5 is a schematic diagram of the phenological period distribution maps of three main crops, namely rice, soybean and corn, in the embodiments of the present application;

[0057] Figure 6 is a schematic diagram of the overall process of the method for early crop classification in the embodiments of the present application in the stage of generating band data;

[0058] Figure 7 is a schematic diagram of determining the prediction step X in the stage of generating band data of the method for early crop classification in the embodiments of the present application;

[0059] Figure 8 is a schematic diagram of the comparison of the classification effects between the method for early crop classification in the embodiments of the present application and the traditional method for early crop classification;

[0060] Figure 9 is a schematic flowchart of the web application interface of the method for early crop classification in the embodiments of the present application;

[0061] Figure 10 It is a schematic diagram of the heat map of the model generation fitting degree at the long prediction step X in the band data generation stage of the crop early classification method in the embodiment of the present application;

[0062] Figure 11 It is a schematic diagram of the radar chart of the change in the early crop classification accuracy after the fusion of band data generation and original data at the long prediction step X in the embodiment of the present application;

[0063] Figure 12 It is a principle block diagram of the crop early classification device in the embodiment of the present application;

[0064] Figure 13 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. Detailed implementation manners

[0065] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as a limitation of the present application.

[0066] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present application means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0067] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0068] Those skilled in the art can understand that the "client", "terminal", and "terminal device" used herein include both devices with a wireless signal receiver that only has the ability to receive and no ability to transmit, and devices with receiving and transmitting hardware that can perform two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, etc., which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; conventional laptop and / or palm-top computers or other devices, which are conventional laptop and / or palm-top computers or other devices with and / or including a radio frequency receiver. The "client", "terminal", and "terminal device" used herein can be portable, transportable, installed in a vehicle (air, sea, and / or land), or suitable for and / or configured to run locally, and / or run in a distributed manner at any other location on the earth and / or in space. The "client", "terminal", and "terminal device" used herein can also be a communication terminal, an Internet access terminal, a music / video playback terminal, for example, it can be a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback function, or it can also be a smart TV, a set-top box, etc.

[0069] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer, which is a hardware device with the necessary components disclosed by the von Neumann principle, including a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input / output devices to complete specific functions.

[0070] It should be noted that the concept of "server" in this application can similarly be extended to the case applicable to a server cluster. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. Physically, these servers can either be independent of each other but can be invoked through an interface, or can be integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility and should not be restricted by this when implementing the network deployment method of this application.

[0071] One or several technical features of this application, unless expressly specified, can either be deployed on a server for implementation and accessed by a client remotely invoking the online service interface provided by the server, or can be directly deployed and run on the client for implementation and access.

[0072] The neural network models cited or possibly cited in this application, unless expressly specified, can either be deployed on a remote server and remotely invoked on the client, or can be deployed on a client with sufficient device capabilities for direct invocation. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0073] All kinds of data involved in this application, unless expressly specified, can either be remotely stored on a server or stored on a local terminal device, as long as it is suitable for being invoked by the technical solution of this application.

[0074] Those skilled in the art should be aware of this: Although the various methods of this application are described based on the same concept and thus show commonality with each other, unless otherwise specified, these methods can all be executed independently. Similarly, for each embodiment disclosed in this application, they are all proposed based on the same inventive concept. Therefore, for the same expressed concepts, as well as concepts that are only appropriately transformed for convenience although the concept expressions are different, they should be equivalently understood.

[0075] For each embodiment to be disclosed in this application, unless expressly pointed out that there is a mutually exclusive relationship between them, otherwise, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct new embodiments, as long as this combination does not deviate from the creative spirit of this application and can meet the needs in the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0076] Please refer to Figure 1 and Figure 2 , in one embodiment of the crop early classification method of this application, it includes:

[0077] Step S10: Obtain the annual time-series remote sensing images containing the target crop in the historical observation year to extract the remote sensing band time-series data of the target crop in multiple phenological periods in the historical observation year. Among them, the time dimension of the remote sensing band time-series data is [0, 36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days long;

[0078] The early crop classification system in the terminal device can obtain the annual time-series remote sensing images containing the target crop in the historical observation year to extract the remote sensing band time-series data of the target crop in multiple phenological periods in the historical observation year. Among them, the time dimension of the remote sensing band time-series data is [0, 36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days long; Among them, the historical observation year includes 2017, 2018 or 2019, etc.,

[0079] In some embodiments, the remote sensing band time-series data includes the basic bands of optical sensors, the basic bands of radar sensors, and vegetation spectral indices. The basic bands of optical sensors include the red light band, the green light band, the blue light band, the near-infrared band, and the short-wave infrared band. The basic bands of radar sensors include vertical polarization parameters and horizontal polarization parameters. The vegetation spectral indices include the normalized difference vegetation index, the enhanced vegetation index, the green canopy vegetation index, the soil moisture index, and the radar vegetation index; The target crop includes one or any combination of rice, corn, and soybeans, and the phenological periods include one or any combination of the sowing period, the tillering period, the seedling period, the jointing period, the branching period, the heading period, the filling period, the pod-setting period, and the maturity period.

[0080] Specifically, the annual time-series remote sensing images of Landsat-7 / 8, Sentinel-1, Sentinel-2, and MODIS LST satellite sensors in each historical observation year from 2017 to 2019 can be selected as the main data source. The time-series remote sensing images can include 12-band time-series data of 11 land cover types, and its main land cover types include rice, corn, soybeans, wheat, grassland, wetland, forest, water body, and built-up area. Samples that do not belong to the above land cover types are uniformly classified as "other crops" and "other land". The 12-band remote sensing time-series data includes the basic bands of optical sensors, the basic bands of radar sensors, and vegetation spectral indices. The basic bands of optical sensors include the red band, green band, blue band, near-infrared (NIR) band, and short-wave infrared (SWIR) band. The basic bands of radar sensors include the vertical polarization (VV) parameter and the horizontal polarization (VH) parameter. The vegetation spectral indices include the normalized difference vegetation index (NDVI), enhanced vegetation index (EVI), green canopy vegetation index (GCVI), land surface water index (LSWI), and radar vegetation index (RVI);

[0081] Furthermore, the top-of-atmosphere reflectance (TOA) data of Landsat-7 ETM+ and Landsat-8 OLI are used. These data are from the Landsat Collection 2 dataset with a processing level of level 1 to ensure high standards of image quality. The dataset contains images with less cloud cover and excellent quality, and has been radiometrically calibrated and orthorectified with high precision. To further ensure data quality, the "QA_PIXEL" band generated by the CFMask algorithm is used to identify and remove low-quality pixels such as clouds, shadows, snow, or ice. When processing Landsat-7 images, strip errors are effectively removed through masking technology, significantly improving data accuracy. At the same time, the TOA data of Sentinel-2A / B are selected in this application and radiometrically calibrated and geometrically corrected to ensure data accuracy and consistency. Given the limitations of optical remote sensing images in cloudy, hazy, rainy, or snowy weather, the synthetic aperture radar (SAR) data of Sentinel-1 are combined with the optical images. Sentinel-1 SAR data has high spatial and temporal resolutions and is suitable for crop classification and growth monitoring in large-scale agricultural areas. Finally, the land surface temperature data provided by MODIS LST images are used as a supplement to the thermodynamic characteristics during the crop growth process.

[0082] To ensure the accuracy of early crop classification tasks, a sufficient number of ground truth samples must be provided. The ground truth data was collected through on-site surveys in a certain area from 2017 to 2019. During the survey, a mobile GIS device was used to record the location and crop type of each sample along the route. A total of 17,870 sample data points of 11 land cover types were collected. The distribution of the sample points is as Figure 3 shown. The main land cover types include rice, corn, soybeans, wheat, grassland, wetland, forest, water body, built-up area, other crops, and other land. After the field survey, all ground samples were visually interpreted using high-resolution images from Google Earth and optical data (Sentinel-2 and Landsat), and samples with obvious errors were excluded. Finally, the sample data was divided as follows:

[0083] The data from 2017 and 2018 can be selected as the historical observation year samples, and the data from 2019 as the target observation year samples. In the spectral feature prediction generation stage, the historical observation year sample data is used for model training and validation, and the target observation year sample data is used for testing; in the early crop classification stage, the historical observation year sample data is used for training, and the target observation year sample data is used for model validation.

[0084] In some embodiments, the step of determining the normalized difference vegetation index includes:

[0085] Step S11: Obtain a first TOA reflectance value corresponding to the near-infrared band and a second TOA reflectance value corresponding to the red band in the time-series remote sensing image;

[0086] Step S12: Calculate and determine a first difference between the first TOA reflectance value and the second TOA reflectance value, and calculate and determine a first sum value between the first TOA reflectance value and the second TOA reflectance value;

[0087] Step S13: Determine the normalized difference vegetation index according to a first ratio between the first difference and the first sum value.

[0088] Specifically, in order to be consistent with the 10m spatial resolution of Sentinel-2, this application uses bicubic interpolation to uniformly resample the spatial resolutions of Landsat, Sentinel-1 / 2, and MODIS LST images to 10 meters to achieve spatial matching of different data sources. Although the original spatial resolution of MODIS LST images is relatively low, considering the inherent consistency of land surface temperature data over a large area, the spatial error caused by resampling has a limited impact on crop identification and is within a reasonable range. In order to make full use of the data advantages of different sensors, this application pairs and integrates each data source according to the band order and time series, and selects five bands, namely the red band, green band, blue band, near-infrared (NIR) band, and short-wave infrared (SWIR) band, to calculate spectral indices, which include the normalized difference vegetation index (NDVI), enhanced vegetation index (EVI), green canopy vegetation index (GCVI), land surface water index (LSWI), and radar vegetation index (RVI). The normalized difference vegetation index (NDVI) is expressed as:

[0089]

[0090] where NDVI represents the normalized difference vegetation index, and ρ NIR represents the first TOA reflectance value corresponding to the near-infrared band, and ρ red represents the second TOA reflectance value corresponding to the red band.

[0091] In a further embodiment, the steps for determining the enhanced vegetation index include:

[0092] Step S101: Obtain the first TOA reflectance value corresponding to the near-infrared band, the second TOA reflectance value corresponding to the red band, and the third TOA reflectance value corresponding to the blue band in the time-series remote sensing image;

[0093] Step S102: Calculate and determine the first product between the first TOA reflectance value and the number 6, and calculate and determine the second product between the third TOA reflectance value and the number 7.5;

[0094] Step S103: Calculate and determine the second sum value between the first TOA reflectance value and the first product, calculate and determine the second difference value between the second sum value and the second product, and calculate and determine the third sum value between the second difference value and the number 1;

[0095] Step S104: Calculate and determine the first difference between the first TOA reflectance value and the second TOA reflectance value, calculate and determine the second ratio between the first difference and the third sum value, and determine the enhanced vegetation index according to the third product between the second ratio and the value 2.5.

[0096] Specifically, the calculation formula of the enhanced vegetation index is expressed as:

[0097]

[0098] where EVI represents the enhanced vegetation index, ρ NIR represents the first TOA reflectance value corresponding to the near-infrared band, ρ red represents the second TOA reflectance value corresponding to the red light band, ρ blue represents the third TOA reflectance value corresponding to the blue light band;

[0099] In a further embodiment, the steps of determining the green canopy vegetation index and the soil moisture index include:

[0100] Step S1001: Obtain the first TOA reflectance value corresponding to the near-infrared band, the fourth TOA reflectance value corresponding to the green light band, and the fifth TOA reflectance value corresponding to the short-wave infrared band in the time-series remote sensing image;

[0101] Step S1002: Calculate and determine the third ratio between the first TOA reflectance value and the fourth TOA reflectance value, and determine the green canopy vegetation index according to the third difference between the third ratio and the value 1;

[0102] Step S1003: Calculate and determine the fourth difference between the first TOA reflectance value and the fifth TOA reflectance value, calculate and determine the fourth sum value between the first TOA reflectance value and the fifth TOA reflectance value, and determine the soil moisture index according to the fourth ratio between the fourth difference and the fourth sum value.

[0103] Specifically, the calculation formula of the green canopy vegetation index is expressed as:

[0104]

[0105] where GCVI represents the green canopy vegetation index, ρ NIR represents the first TOA reflectance value corresponding to the near-infrared band, ρ green represents the fourth TOA reflectance value corresponding to the green light band;

[0106] The calculation formula of the soil moisture index is expressed as:

[0107]

[0108] Among them, LSWI represents the soil moisture index, and ρ SWIR represents the fifth TOA reflectance value corresponding to the short-wave infrared band.

[0109] In a further embodiment, the steps of determining the radar vegetation index include:

[0110] Step S10001: Obtain the first polarization backscattering coefficient of horizontal transmission and vertical reception, the second polarization backscattering coefficient of horizontal transmission and horizontal reception, and the third polarization backscattering coefficient of vertical transmission and vertical reception;

[0111] Step S10002: Calculate and determine the fourth product between the first polarization backscattering coefficient and the value 8, calculate and determine the fifth product between the first polarization backscattering coefficient and the value 2, and calculate and determine the fifth sum value among the fifth product, the second polarization backscattering coefficient, and the third polarization backscattering coefficient;

[0112] Step S10003: Determine the radar vegetation index according to the fifth ratio between the fourth product and the fifth sum value.

[0113] Specifically, the radar vegetation index is expressed as:

[0114]

[0115] Among them, RVI represents the radar vegetation index, and V HV represents the first polarization backscattering coefficient of horizontal transmission and vertical reception, and σ HH represents the second polarization backscattering coefficient of horizontal transmission and horizontal reception, and σ VV represents the third polarization backscattering coefficient of vertical transmission and vertical reception.

[0116] Given that remote sensing time-series data is large in volume and highly redundant, directly using this data for model training may lead to waste of computing resources. The Green Canopy Vegetation Index (GCVI) has advantages in evaluating the green leaf content of vegetation, can more accurately reflect the health and growth status of vegetation, and effectively overcomes the saturation problem of the Normalized Difference Vegetation Index (NDVI). It is particularly suitable for crop growth period monitoring. Therefore, in this study, the Green Canopy Vegetation Index (GCVI) was selected as the main vegetation index, and the maximum GCVI value images were extracted every ten days for synthesis processing. This not only retains the information of the critical growth period but also reduces the redundancy of time-series data, providing more accurate and concise input data. To further improve the matching of optical time-series remote sensing images and environmental data, this application combines the minimum VH polarization value of Sentinel-1 and the minimum value of MODIS LST for ten-day interval synthesis to capture the dynamic changes during crop harvesting and the impact of temperature on crop growth.

[0117] In summary, finally, generate Figure 4 the time-series curves of the spectral bands and vegetation indices VIs of paddy fields, soybean fields, and corn fields in the three northeastern provinces of China from 2017 to 2019 as shown to illustrate their potential for crop type classification.

[0118] In some embodiments, refer to Figure 5 , Figure 5 which is a schematic diagram of the phenological period distribution maps of the three main crops of rice, soybean, and corn in the embodiments of this application.

[0119] Step S20: According to the observation point e of the target observation year, divide the remote sensing band time-series data corresponding to the historical observation year into two parts in the time dimension to determine the first remote sensing band time-series data and the second band time-series data. Among them, the time dimension of the first remote sensing band time-series data is [0, e], and the time dimension of the second remote sensing band time-series data is [e, 36];

[0120] After obtaining the annual time-series remote sensing images containing the target crop in the historical observation year and extracting the remote sensing band time-series data of the target crop in multiple phenological periods in the historical observation year, according to the observation point e of the target observation year, divide the remote sensing band time-series data corresponding to the historical observation year into two parts in the time dimension to determine the first remote sensing band time-series data and the second band time-series data. Among them, the time dimension of the first remote sensing band time-series data is [0, e], and the time dimension of the second remote sensing band time-series data is [e, 36];

[0121] Step S30: Use the first remote sensing band time-series data as the training set and the second remote sensing band time-series data as the label to train the optimized crop growth prediction model until the model reaches the convergence state to determine the best crop growth prediction model;

[0122] According to the observation point e in the target observation year, the remote sensing band time series data corresponding to the historical observation year is divided into two parts in the time dimension. After determining the first remote sensing band time series data and the second band time series data, the first remote sensing band time series data is used as the training set, and the second remote sensing band time series data is used as the label to train the optimized crop growth prediction model until the model reaches a convergence state to determine the best crop growth prediction model; wherein, the basic network architecture of the optimized crop growth prediction model includes a Linear model with a time series data decomposition module introduced, a convolutional neural network with a deep time series convolutional network and causal convolution introduced, a long short-term memory network with a self-attention module and a residual connection module introduced, or a Transformer model with an autoregressive module introduced;

[0123] In some embodiments, the basic network architecture of the crop growth prediction model includes a Linear model with a time series data decomposition module introduced, a convolutional neural network with a deep time series convolutional network and causal convolution introduced, a long short-term memory network with a self-attention module and a residual connection module introduced, or a Transformer model with an autoregressive module introduced; for the Linear model with a time series data decomposition module introduced, the original input sequence is decomposed into a seasonal component and a trend component, the moving average method is used to extract the seasonal and trend components in the data, and the seasonal and trend components are predicted separately through a linear layer to improve the ability to capture long-term trends and periodic changes; for the convolutional neural network with a deep time series convolutional network and causal convolution introduced, the causal convolution ensures that the model only uses historical information for prediction, and the time series module and the deep time series convolutional network are used to effectively capture multi-scale time series features, and finally the prediction is made through a linear layer to improve the accuracy of crop growth prediction; for the long short-term memory network with a self-attention module and a residual connection module introduced, the self-attention module enhances the ability to capture long-term dependencies by dynamically adjusting the weights of time steps, and the gradient disappearance is alleviated through the residual connection module; for the Transformer model with an autoregressive module introduced, multiple prediction results are generated step by step and the prediction of the previous step is used as the input for the next time step. Each prediction step depends on the previous output, and after being mapped to a high dimension through an expansion layer, it further provides an estimate of future moments.

[0124] Specifically, for the Linear model, by introducing a time series data decomposition (series_decomp) module, the original input sequence is decomposed into seasonal and trend components, the moving average method is used to extract the seasonal and trend components in the data, and the seasonal and trend components are predicted separately through a linear layer, thereby improving the ability to capture long-term trends and periodic fluctuations;

[0125] For the convolutional neural network (CNN), the Deep Temporal Convolutional Network (DeepTCN) is introduced. Causal convolution (CausalConv1d) is used to ensure that the model only uses historical information for prediction. Temporal features at multiple scales are effectively captured through the Temporal Block and the Temporal Convolutional Network. Finally, prediction is made through a linear layer, improving the accuracy of crop growth prediction.

[0126] For the long short-term memory network (LSTM), the Self-Attention module and the Residual Connection module are introduced. The Self-Attention module enhances the model's ability to capture long-term dependencies by dynamically adjusting the weights at each time step, while the Residual Connection module alleviates the vanishing gradient problem in deep networks, improving the training stability of the model. For the Transformer model, the Autoregression module is introduced. By gradually generating multiple prediction results and using the previous prediction as the input for the next moment, each prediction step depends on the previous output. The prediction results are mapped to a high-dimensional space through the fc_expand layer, further improving the accurate estimation of future moments and enhancing the accuracy and prediction stability of the model.

[0127] Specifically, the best crop growth prediction model trained to convergence is called. For soybeans and corn, prediction starts at the sowing stage, and the range of e is e ∈ [12, 25]. Since rice is an aquatic crop and the land needs to maintain a high water content before sowing, and the near-infrared (NIR) and short-wave infrared (SWIR) bands in remote sensing images are highly sensitive to soil moisture changes, it is possible to preliminarily identify the rice planting characteristics in this area about 10 to 20 days before the rice sowing stage (about the 120th day of the year). Therefore, for rice, early prediction is carried out about 10 to 20 days before sowing, and the range of e is e ∈ [10, 25]. Three crops are segmented into T t The time periods are: [0, 10), [0, 11)... [0, 25), a total of 16 groups.

[0128] The prediction interval T x The length X ∈ [1, 36 – e + 1]. If the sum of the time steps of T t and T x is less than 36, the data after T x is recorded as T d and is not used as valid data for model training.

[0129] Please refer to Figure 6, in the multi-source remote sensing band generation stage, using the 16 groups of multi-remote sensing band time series data generated in the above embodiments as the original band data, dividing the sample data of the historical observation years into a training set and a validation set according to a ratio of 8:2, and using the sample data of the target observation year as the test set. In the training set and the validation set, the time step is T t of the remote sensing band time series data as the input data, and the time step is T x of the remote sensing band time series data as the prediction label. In the test set, the time step is T t of the remote sensing band time series data as the input data to generate the remote sensing band time series data with a time step of T p wherein the second remote sensing band time series data is the remote sensing band time series data with a time step of T p of the remote sensing band time series data.

[0130] In some embodiments, the hyperparameters of the crop growth prediction model are set as follows: the initial learning rate is 0.0001, it decays to 0.1 times the original every 10 epochs, batchsize = 256, and epoch = 100.

[0131] Regarding the fitting accuracy of the crop growth prediction model in the training, validation, and testing links, the following four indicators are used for evaluation, including the mean absolute error (MAE), the root mean square error (RMSE), the symmetric mean absolute percentage error (sMAPE), and the coefficient of determination (R 2 ). Among them, MAE is used to measure the linear measure of the model prediction error of the crop growth prediction model, RMSE is mainly used to evaluate the performance of the model when dealing with outliers, sMAPE is used to evaluate the robustness of the model when the amplitude difference between the predicted value and the true value is large, and R 2 reflects the fitting degree of the model to the real data. The specific calculation formulas of each indicator are as follows:

[0132]

[0133] where, y i is the true value, is the predicted value, is the mean of the true values, and n is the number of samples.

[0134] Step S40: Input the third remote sensing band time series data with a time dimension of [0, e] within the target observation year into the best crop growth prediction model to generate the fourth remote sensing band time series data with a time dimension of [e, 36] within the target observation year;

[0135] Based on any of the above embodiments, the first remote sensing band time series data is used as the training set, and the second remote sensing band time series data is used as the label to train the optimized crop growth prediction model until the model reaches a convergence state. After determining the optimal crop growth prediction model, the third remote sensing band time series data with a time dimension of [0, e] within the target observation year is input into the optimal crop growth prediction model to generate the fourth remote sensing band time series data with a time dimension of [e, 36] within the target observation year.

[0136] Step S50: Concatenate the third remote sensing band time series data and the fourth remote sensing band time series data in the time dimension to determine the concatenated remote sensing band time series data. Input the concatenated remote sensing band time series data into the crop classification model that has been trained to a convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period, so as to complete the early classification of the target crop.

[0137] After inputting the third remote sensing band time series data with a time dimension of [0, e] within the target observation year into the optimal crop growth prediction model to generate the fourth remote sensing band time series data with a time dimension of [e, 36] within the target observation year, concatenate the third remote sensing band time series data with a time dimension of [0, e] within the target observation year and the fourth remote sensing band time series data with a time dimension of [e, 36] within the target observation year in the time dimension to determine the concatenated remote sensing band time series data. Input the concatenated remote sensing band time series data into the crop classification model that has been trained to a convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period, so as to complete the early classification of the target crop; wherein, the basic network architecture of the crop classification model includes any one of a convolutional neural network (CNN), a long short-term memory network (LSTM), or a Transformer model.

[0138] Based on any of the above embodiments, the third remote sensing band time series data with a time dimension of [0, e] before the observation point e in the target observation year is input into the optimal crop growth prediction model to generate the fourth remote sensing band time series data with a time dimension of [e, 36] after the observation point e in the target observation year. Concatenate the third remote sensing band time series data with a time dimension of [0, e] within the target observation year and the fourth remote sensing band time series data with a time dimension of [e, 36] within the target observation year in the time dimension to determine the concatenated remote sensing band time series data, so as to complete data fusion.

[0139] Specifically, the third remote sensing band time series data has a time step of T tFor the time-series data of the remote sensing bands, the time-series data of the third remote sensing band with a time dimension of [0, e] within the target observation year is spliced with the time-series data of the fourth remote sensing band with a time dimension of [e, 36] within the target observation year in the time dimension to determine the spliced time-series data of the remote sensing bands, so as to complete data fusion. The spliced time-series data of the remote sensing bands is used as the training set of the crop classification model for the early classification and recognition task of the crop classification model of this application.

[0140] During the training and validation process of the crop growth prediction model, the initial prediction step size X = 1 is selected for model training. After that, X is incremented by 1 each time, and the model training and evaluation process is repeated. If the prediction accuracy index of the current batch is higher than or close to the evaluation result of the previous time, then continue to increment X; if the prediction accuracy index continuously decreases during the increment of X, then stop incrementing X. After the crop growth prediction model has gone through the training and validation steps, it predicts the test set samples of the target observation year, and the model output is at time step T p for 12 kinds of time-series data of various crops within. Finally, the generated time-series data of the fourth remote sensing band with a time step of T p is appended to the time-series data of the third remote sensing band with a time step of T t in the time dimension to determine the spliced time-series data of the remote sensing bands, supplementing 12 kinds of time-series data of various crops within the time step of T p in the time dimension, increasing the number of features of the time-series data of the remote sensing bands. The final result is to supplement the predicted band data in the time dimension to enrich the feature information. The value result of the prediction step size X is as Figure 7 shown.

[0141] Specifically, the time-series data of the third remote sensing band is the time-series data of the remote sensing band with a time step of T t The time-series data of the third remote sensing band with a time dimension of [0, e] within the target observation year is spliced with the time-series data of the fourth remote sensing band with a time dimension of [e, 36] within the target observation year in the time dimension to determine the spliced time-series data of the remote sensing bands, so as to complete data fusion. After the data fusion is completed, the spliced time-series data of the remote sensing bands is used as the training set of the crop classification model. After the crop classification model is trained to the convergence state, it can be used to detect the crop category corresponding to the target crop in the time-series remote sensing image and its corresponding phenological period.

[0142] Furthermore, the performance of three deep learning classification models, namely Convolutional Neural Network (CNN), Long Short-Term Memory Network (LSTM), or Transformer model, in the early crop classification accuracy using the fused remote sensing band time series data is evaluated using the following two metrics, including Overall Accuracy (OA) and Kappa coefficient (Cohen's Kappa). Among them, OA is the proportion of correctly classified samples in the classification results to the total samples, indicating the overall accuracy of the model. When measuring the accuracy of the Kappa coefficient, the possibility of random classification is considered. Its value ranges from -1 to 1, and the higher the value, the higher the classification accuracy. The specific calculation formulas for each metric are as follows:

[0143]

[0144] Among them, P o is the observed accuracy (actual classification result), and P e is the expected accuracy in the case of randomness.

[0145] A set of control experiments was designed. The training set of Experiment 1 uses the sample data of historical observation years and the band features where T t ∈[0,e] for training, and the band features where the target observation year T t ∈[0,e] are used to verify the classification results. The training set of Experiment 2 uses the sample data of historical observation years and the band features where T t ∈[0,e+X] for training, and the fused data after Step4 is used for verification in the validation set. The accuracy metrics of Experiment 1 and Experiment 2 in the classification task were evaluated, and the evaluation results are as Figure 8 shown.

[0146] In some embodiments, the improved network is deployed, and a web application interface is built to implement the early crop classification method. The specific process is as follows: Import the remote sensing band data within the time period from this year to the current date, generate a workflow through the remote sensing band data in this article, output the deep learning model weight file (.pt format) for early crop classification and recognition, and the size of the prediction step X can also be manually set in the remote sensing band data generation workflow in this article. The web application interface is as Figure 9 shown.

[0147] In some embodiments, please refer to Figure 10 and Figure 11 , where Figure 10 is a schematic diagram of the heat map of the model generation fitting degree for the long prediction step X in the band data generation stage of the early crop classification method in the embodiments of this application; Figure 11Schematic diagram of a radar chart showing the change in the early classification accuracy of crops after the generation of band data and fusion with the original data under the long prediction step X in the embodiments of the present application.

[0148] As can be seen from the above embodiments, compared with the prior art, in the research task of early crop classification based on deep learning in the prior art, there are problems such as too little band feature information in the early observation period, insufficient effective recognition features, resulting in low classification accuracy, late recognition date, and poor stability of the classification model. The present application includes but is not limited to the following beneficial effects:

[0149] First, the present application can significantly improve the classification accuracy. By generating more band features, the present application effectively solves the problem of insufficient band feature information in the early observation period. In traditional methods, due to the limitation of the dimension of band features, the early classification accuracy is low, while the present application can generate richer feature information on the basis of the existing bands, thereby improving the accuracy of early crop classification.

[0150] Second, the present application solves the problem of lagging classification date. Traditional early crop classification methods often need to sacrifice time steps to improve earlyness, which leads to a decrease in classification accuracy. Through in-depth analysis of spectral trends by the time series model and the application of the feature screening mechanism in the present application, not only can continuous time series spectral features be generated, but the classification date will not be postponed. In this way, while ensuring the earlyness of classification, the accuracy of classification can be significantly improved.

[0151] Third, the present application enhances the stability and adaptability of the model. Traditional models are usually trained based on data in specific regions or seasons, which easily leads to poor generalization ability of the model and inability to adapt to crop growth patterns under different geographical and climatic conditions. The multi-source remote sensing time series data generation framework provided by the present application can effectively improve the adaptability and stability of the model for crop classification tasks in different environments through in-depth processing of data at multiple time points.

[0152] Fourth, the present application makes full use of multi-source remote sensing time series data. It can not only generate the trend of remote sensing time series data for any time period in the target observation year, but also flexibly adapt to different climatic environments and crop growth patterns. This method can maintain a high recognition accuracy in a changing environment and has a strong application breadth.

[0153] Furthermore, the present application can further improve the accuracy and stability of the crop classification model without postponing the classification date, and can flexibly predict the trend of multi-source remote sensing time series data for any time period in the target observation year.

[0154] Please refer to Figure 12, A crop early classification device provided to meet one of the purposes of this application, including a historical remote sensing band acquisition module 1100, a remote sensing band data division module 1200, a growth prediction model training module 1300, a remote sensing band data prediction module 1400, and a crop early classification module 1500. Among them, the historical remote sensing band acquisition module 1100 acquires the annual time-series remote sensing images containing the target crop within the historical observation years to extract the remote sensing band time-series data of the target crop in multiple phenological periods within the historical observation years. Among them, the time dimension of the remote sensing band time-series data is [0, 36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days; the remote sensing band data division module 1200 is set to divide the remote sensing band time-series data corresponding to the historical observation years into two parts in the time dimension according to the observation point e of the target observation year to determine the first remote sensing band time-series data and the second band time-series data. Among them, the time dimension of the first remote sensing band time-series data is [0, e], and the time dimension of the second remote sensing band time-series data is [e, 36]; the growth prediction model training module 1300 is set to use the first remote sensing band time-series data as the training set and the second remote sensing band time-series data as the label to train the optimized crop growth prediction model until the model reaches the convergence state to determine the best crop growth prediction model; the remote sensing band data prediction module 1400 inputs the third remote sensing band time-series data with the time dimension of [0, e] within the target observation year into the best crop growth prediction model to generate the fourth remote sensing band time-series data with the time dimension of [e, 36] within the target observation year; the crop early classification module 1500 is set to splice the third remote sensing band time-series data and the fourth remote sensing band time-series data in the time dimension to determine the spliced remote sensing band time-series data, and input the spliced remote sensing band time-series data into the crop classification model that has been trained to the convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period to complete the early classification of the target crop.

[0155] Based on any embodiment of this application, please refer to Figure 13 , Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as Figure 13As shown, it is a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database can store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a method for early classification of crops. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device can store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the method for early classification of crops of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 13 The structure shown is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0156] In this embodiment, the processor is used to execute Figure 12 the specific functions of each module in. The memory stores the program code and various types of data required to execute the above modules. The network interface is used for data transmission between the user terminal and the server. The memory in this embodiment stores the program code and data required to execute all modules in the crop early classification device of the present application. The server can call the program code and data of the server to execute the functions of all modules.

[0157] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more processors are caused to execute the steps of the method for early classification of crops according to any embodiment of the present application.

[0158] The present application also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by one or more processors, the steps of the method for early classification of crops according to any embodiment of the present application are implemented.

[0159] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments of the present application can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disc, a Read-Only Memory (ROM), or a Random Access Memory (RAM), etc.

[0160] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for early classification of crops, characterized in that: include: Acquire the full-year time series remote sensing images containing the target crop in the historical observation year to extract the remote sensing band time series data of the target crop in multiple phenological periods in the historical observation year, wherein the time dimension of the remote sensing band time series data is [0,36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days; According to the observation point e of the target observation year, the remote sensing band time series data corresponding to the historical observation year is divided into two parts in the time dimension to determine the first remote sensing band time series data and the second band time series data, wherein the time dimension of the first remote sensing band time series data is [0, e], and the time dimension of the second remote sensing band time series data is [e, 36]; The optimized crop growth prediction model is trained by using the first remote sensing band time series data as a training set and the second remote sensing band time series data as a label until the model reaches a convergence state, so as to determine the best crop growth prediction model; Inputting the third remote sensing band time series data with a time dimension of [0, e] in the target observation year into the optimal crop growth prediction model to generate the fourth remote sensing band time series data with a time dimension of [e, 36] in the target observation year; The third remote sensing band time series data and the fourth remote sensing band time series data are spliced ​​in the time dimension to determine the spliced ​​remote sensing band time series data, and the spliced ​​remote sensing band time series data are input into a crop classification model that has been trained to a convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period, so as to complete the early classification of the target crop.

2. The method for early classification of crops according to claim 1, characterized in that: The remote sensing band time series data includes basic bands of optical sensors, basic bands of radar sensors and vegetation spectral indexes, the basic bands of optical sensors include red light bands, green light bands, blue light bands, near infrared bands and short-wave infrared bands, the basic bands of radar sensors include vertical polarization parameters and horizontal polarization parameters, and the vegetation spectral indexes include normalized vegetation index, enhanced vegetation index, green canopy vegetation index, soil moisture index and radar vegetation index; The basic network architecture of the optimized crop growth prediction model includes a Linear model that introduces a time series data decomposition module, a convolutional neural network that introduces a deep time series convolutional network and a causal convolution, a long short-term memory network that introduces a self-attention module and a residual connection module, or a Transformer model that introduces an autoregressive module; The basic network architecture of the crop classification model includes any one of a convolutional neural network, a long short-term memory network or a Transformer model; The target crops include one or more of rice, corn and soybean, and the phenological period includes one or more of sowing period, tillering period, seedling period, jointing period, branching period, heading period, filling period, bud formation period and maturity period.

3. The method for early classification of crops according to claim 2, characterized in that: For the Linear model that introduces the time series data decomposition module, the original input sequence is decomposed into seasonal components and trend components. The moving average method is used to extract the seasonal and trend components in the data, and the seasonal and trend components are predicted separately through the linear layer to improve the ability to capture long-term trends and cyclical changes. For the convolutional neural network that introduces the deep temporal convolutional network and causal convolution, the causal convolution ensures that the model only uses historical information for prediction, and effectively captures multi-scale temporal features through the temporal module and the deep temporal convolutional network, and finally predicts through the linear layer to improve the accuracy of crop growth prediction; For the long short-term memory network that introduces the self-attention module and the residual connection module, the self-attention module enhances the ability to capture long-term dependencies by dynamically adjusting the weights of the time steps, and the residual connection module alleviates the gradient disappearance; For the Transformer model that introduces the autoregressive module, by gradually generating multiple prediction results and using the prediction of the previous step as the input of the next time step, each prediction step depends on the previous output, and after being mapped to a high dimension through the expansion layer, it further provides estimates of future moments.

4. The method for early classification of crops according to claim 2, characterized in that: The step of determining the normalized vegetation index comprises: Acquire a first TOA reflectivity value corresponding to the near-infrared band and a second TOA reflectivity value corresponding to the red light band in the time series remote sensing image; Calculate and determine a first difference between the first TOA reflectivity value and the second TOA reflectivity value, and calculate and determine a first sum between the first TOA reflectivity value and the second TOA reflectivity value; The normalized vegetation index is determined according to a first ratio between the first difference and the first sum.

5. The method for early classification of crops according to claim 2, characterized in that: The step of determining the enhanced vegetation index comprises: Acquire a first TOA reflectivity value corresponding to the near-infrared band, a second TOA reflectivity value corresponding to the red light band, and a third TOA reflectivity value corresponding to the blue light band in the time series remote sensing image; Calculate and determine a first product between the first TOA reflectivity value and a value of 6, and calculate and determine a second product between the third TOA reflectivity value and a value of 7.5; Calculate and determine a second sum value between the first TOA reflectivity value and the first product, calculate and determine a second difference value between the second sum value and the second product, and calculate and determine a third sum value between the second difference value and a value of 1; A first difference between the first TOA reflectivity value and the second TOA reflectivity value is calculated and determined, a second ratio between the first difference and the third sum is calculated and determined, and the enhanced vegetation index is determined based on a third product between the second ratio and a value of 2.

5.

6. The method for early classification of crops according to claim 2, characterized in that: The step of determining the green canopy vegetation index and the soil moisture index comprises: Acquire a first TOA reflectivity value corresponding to the near-infrared band, a fourth TOA reflectivity value corresponding to the green light band, and a fifth TOA reflectivity value corresponding to the short-wave infrared band in the time series remote sensing image; A third ratio between the first TOA reflectivity value and the fourth TOA reflectivity value is calculated and determined, and the green canopy vegetation index is determined according to a third difference between the third ratio and the value 1; a fourth difference between the first TOA reflectivity value and the fifth TOA reflectivity value is calculated and determined, and a fourth sum between the first TOA reflectivity value and the fifth TOA reflectivity value is calculated and determined, and the soil moisture index is determined according to a fourth ratio between the fourth difference and the fourth sum.

7. The method for early classification of crops according to claim 2, characterized in that: The step of determining the radar vegetation index comprises: Acquire a first polarization backscatter coefficient for horizontal transmission and vertical reception, a second polarization backscatter coefficient for horizontal transmission and horizontal reception, and a third polarization backscatter coefficient for vertical transmission and vertical reception; Calculate and determine a fourth product between the first polarization backscatter coefficient and a value of 8, calculate and determine a fifth product between the first polarization backscatter coefficient and a value of 2, and calculate and determine a fifth sum between the fifth product, the second polarization backscatter coefficient, and the third polarization backscatter coefficient; The radar vegetation index is determined according to a fifth ratio between the fourth product and the fifth sum.

8. An early crop classification device, characterized in that: include: The historical remote sensing band acquisition module acquires the time-series remote sensing images of the target crop in the historical observation year to extract the remote sensing band time-series data of the target crop in multiple phenological periods in the historical observation year, wherein the time dimension of the remote sensing band time-series data is [0,36], which represents that the whole year is divided into 36 time steps according to the growth cycle of the target crop, and each time step is 10 days; A remote sensing band data division module is configured to divide the remote sensing band time series data corresponding to the historical observation year into two parts in the time dimension according to the observation point e of the target observation year, so as to determine the first remote sensing band time series data and the second band time series data, wherein the time dimension of the first remote sensing band time series data is [0, e], and the time dimension of the second remote sensing band time series data is [e, 36]; A growth prediction model training module is configured to train the optimized crop growth prediction model using the first remote sensing band time series data as a training set and the second remote sensing band time series data as a label until the model reaches a convergence state, so as to determine an optimal crop growth prediction model; A remote sensing band data prediction module inputs the third remote sensing band time series data with a time dimension of [0, e] in the target observation year into the optimal crop growth prediction model to generate the fourth remote sensing band time series data with a time dimension of [e, 36] in the target observation year; The crop early classification module is configured to splice the third remote sensing band time series data with the fourth remote sensing band time series data in the time dimension to determine the spliced ​​remote sensing band time series data, and input the spliced ​​remote sensing band time series data into a crop classification model that has been trained to a convergence state to determine the crop category corresponding to the target crop and its corresponding phenological period, so as to complete the early classification of the target crop.

9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images