Crop growth monitoring method and system based on multi-source images

Abstract

The present invention discloses a crop growth monitoring method and system based on multi-source images, including: partitioning the crops in the study area and performing different irrigation and fertilization treatments; obtaining meteorological information, remote sensing image data and thermal infrared image data of crops in different partitions; cutting and data augmentation processing of thermal infrared images and remote sensing images to construct a crop data set; constructing a crop growth monitoring deep learning model, taking the meteorological information of the study area as input, optimizing the crop growth deep learning model using the data set to obtain an optimized crop growth deep learning model; regressing the relationship between BBCH value and irrigation and fertilization; using the optimized crop growth monitoring deep learning model to predict the BBCH value of the crop to be fertilized, and formulating a corresponding irrigation and fertilization plan according to the predicted BBCH value. The present invention can monitor the crop growth situation on a large scale and quickly, and provide guidance for crop irrigation and fertilization to prevent crop yield reduction.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop growth, and in particular relates to a crop growth monitoring method and system based on multi-source images. Background Art

[0002] With the development of science and technology and its application in agriculture, the methods for monitoring crop growth processes have become more accurate and diversified. Deep learning methods are widely used in smart agriculture fields such as pest and disease monitoring, crop classification, and crop monitoring. Deep learning can use a limited number of samples to abstract the underlying features of the data into more specific and representative features, and can better cooperate with agricultural hardware equipment to improve the intelligence of the equipment. At present, remote sensing images have become an important source of data for agricultural deep learning research. Remote sensing data are mainly divided into near-surface measurement data and satellite remote sensing data. Land surface phenology (LSP) is collected by ground and near-surface sensors, mainly including digital cameras and drones. PhenoCam is a near-surface digital camera located above the canopy. These cameras are distributed in different regions and different ecosystem environments. The camera detects leaf phenological changes by analyzing color changes over time. By quantifying the red, green, and blue (RGB) color channels, the green and red channels can be used to estimate phenomena such as leaf budding and senescence, respectively.

[0003] Most satellite-based phenology studies rely on satellite sensors with medium to low spatial resolution, such as MODIS and Landsat, or fuse them to improve temporal and spatial resolution. However, the large observation range of satellites limits the resolution of their images. At the same time, there is a gap in observation scale and characteristics between ground observations and satellite remote sensing observations, which may not correctly reflect the phenology of crops. In the field of remote sensing combined with deep learning, many scholars have proposed their own methods. For example, Zhao et al. developed a spatial perception scheme based on deep learning to integrate SAR and optical time series data to overcome the differences in perspective and range between the two to achieve accurate crop phenology monitoring. However, this method has high requirements for time series data and requires a continuous and stable data source. Satellites are affected by clouds and it is difficult to provide continuous, high-quality image data, which will have a certain impact on accuracy.

[0004] Different water and fertilizer conditions before and after the flowering period of crops will change the development process. Accurately monitoring the growth and development stage of crops can guide farmers to replenish water and fertilizer in time, ensure the normal growth process of crops during the critical growth period, and is of great significance for avoiding crop yield reduction. The traditional acquisition of crop growth and development stages is mainly obtained through field observations. The observation results are relatively accurate, but it takes a long time, requires a lot of manpower, and has a small coverage area. Most of the crop growth and development monitoring methods based on remote sensing images currently constructed using deep learning methods have low spatial resolution and limited accuracy, and cannot be subdivided at the level of specific growth stages; secondly, most remote sensing monitoring methods are used for large-area fields, ignoring the differences in physical conditions between fields. Therefore, it is necessary to develop a new method for crop growth monitoring to guide farmers to replenish water and fertilizer in time. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the prior art and provide a crop growth monitoring method based on multi-source images, which can monitor crop growth over a large area and quickly, and provide guidance for crop irrigation and fertilization based on the monitored crop growth conditions to prevent crop yield reduction.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A crop growth monitoring method based on multi-source images comprises the following steps:

[0008] Step 1: Divide the crops in the study area into different zones and apply different irrigation and fertilization treatments to the crops in different zones;

[0009] Step 2: Obtain meteorological information of the study area, remote sensing image data and thermal infrared image data of crops in different sub-areas, and pre-process the obtained remote sensing image data;

[0010] Step 3: Crop and data augment the thermal infrared images and preprocessed remote sensing images, annotate the processed images with phenological information and corresponding irrigation and fertilization information to obtain a crop dataset, and divide the dataset into a training set, a validation set, and a test set;

[0011] Step 4: Construct a crop growth monitoring deep learning model for predicting crop BBCH values. Take the meteorological information of the study area as input, use the training set to train the model parameters of the crop growth deep learning model and verify it on the validation set, then use the test set to test the accuracy, and select the crop growth deep learning model with the required accuracy as the optimized crop growth deep learning model;

[0012] Step 5: Use the data in the data set to regress the relationship between BBCH value and fertigation;

[0013] Step 6: Use the optimized crop growth monitoring deep learning model to predict the BBCH value of the crops to be fertilized, and formulate a corresponding irrigation and fertilization plan based on the predicted BBCH value and the relationship obtained in step 5.

[0014] Furthermore, the method of preprocessing the remote sensing image data in step 2 includes image stitching, geometric registration, field scale cropping and image channel fusion.

[0015] Furthermore, step 3 in data augmentation includes diagonal mirroring, horizontal flipping, vertical flipping and Gaussian blurring of the cropped image.

[0016] Furthermore, the crop growth monitoring model consists of two parts. The first part includes a Feature fusion module, which is used to extract the features of the input remote sensing image and thermal infrared image, and superimpose and fuse the features of the two images; the second part includes a Multi-Head Self-Attention module and an MLP module, which are used to further extract global image features from the fused image features and output a one-dimensional result, namely the BBCH value.

[0017] Furthermore, the method for extracting image features by the Feature fusion module includes:

[0018] The remote sensing image and thermal infrared image are input into the Feature fusion module. After a series of convolution transformations, the corresponding torch tensors are obtained respectively, and the meteorological information is converted into a torch tensor. The three are concatenated through the Concat function to obtain a torch tensor with multiple features.

[0019] Furthermore, the Multi-Head Self-Attention module includes a multi-head self-attention layer and a fully connected layer; the torch tensor extracted by the Feature Fusion module is input into the multi-head self-attention layer, in which it is expanded through the Patch Embedding layer and the input is mapped to q, k, v. After the attention calculation, the output is concatenated into a vector of the same size as the input through the linear layer;

[0020] The fully connected layer adjusts the image vector in the multi-head self-attention layer to a one-dimensional result, and outputs the BBCH value of the corresponding crop.

[0021] Furthermore, in step 6, whether the BBCH value of the crop at the key time node reaches the threshold is used as a basis for judging whether the crop is under water and nitrogen stress. If the crop is under water and nitrogen stress, the amount of irrigation and fertilization is set at the key time node; otherwise, no irrigation and fertilization is performed.

[0022] Furthermore, the key time node is the jointing period of crops.

[0023] Furthermore, when the BBCH value of the crop at the key time node is predicted to be less than 50, the crop is judged to be under water and nitrogen stress, otherwise, it is judged not to be under water and nitrogen stress.

[0024] The present invention also provides a system according to the above-mentioned crop growth monitoring method based on multi-source images, characterized in that it includes:

[0025] Data acquisition module, used to obtain meteorological information of the study area, remote sensing image data of crops in different zones, thermal infrared image data, and irrigation and fertilization data;

[0026] The data processing module is used to pre-process the remote sensing image data obtained by the data acquisition module, and to perform cropping and data augmentation processing on the thermal infrared image and the pre-processed remote sensing image;

[0027] A data set construction module is used to annotate the images processed by the data processing module with corresponding phenological information and irrigation and fertilization information to obtain a crop data set, and divide the data set into a training set, a validation set, and a test set;

[0028] The model building module is used to build a crop growth monitoring deep learning model for predicting crop BBCH values. The meteorological information of the study area is used as input. The model parameters of the crop growth deep learning model are trained using the training set in the data set building module and verified on the validation set. The accuracy is then tested using the test set. The crop growth deep learning model with the required accuracy is selected as the optimized crop growth deep learning model.

[0029] The water and fertilizer decision-making module is used to regress the relationship between the BBCH value and fertigation. The optimized crop growth monitoring deep learning model is used to predict the BBCH value of the crops to be fertilized, and the corresponding fertigation plan is formulated based on the predicted BBCH value and the relationship between the BBCH value and fertigation.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (1) The present invention uses multispectral and thermal infrared images acquired by drones as input images, and uses accumulated temperature information as auxiliary input data, which increases the data source and reasonably allocates the weight of each data. Different types of images of the same target can enhance the network's perception of image features, avoid falling into local optimal solutions, and improve the accuracy of monitoring;

[0032] (2) The present invention constructs a crop growth monitoring model, which uses multispectral and thermal infrared images as input, thereby improving the accuracy of the model in identifying the crop growth period, and regresses the relationship between the identified image features and the water and nitrogen stress, and finally outputs the BBCH value of the crop at the key time node. According to the BBCH value, it is judged whether the crop is under water and nitrogen stress, so as to formulate a corresponding irrigation and fertilization plan. The present invention can monitor the crop growth over a large area and quickly, and diagnose the water and nitrogen stress of the crop at an early stage based on the monitored crop growth, provide relevant information for water replenishment and fertilization, and prevent crop yield reduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of a crop growth monitoring method based on multi-source images according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of different irrigation and fertilization treatments in different zones according to an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of preprocessing a remote sensing image according to an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the structure of a deep learning model for crop growth monitoring according to an embodiment of the present invention;

[0037] Figure 5 BBCH curve variation diagram for different irrigation and fertilization treatments of crops according to an embodiment of the present invention;

[0038] Figure 6 This is a diagram showing changes in the BBCH curves for different irrigation and fertilization treatments of crops according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be described clearly and completely below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0041] The present invention will be further described below in conjunction with specific embodiments, but the present invention is not limited thereto.

[0042] like Figure 1 As shown, the embodiment of the present invention discloses a crop growth monitoring method based on multi-source images, comprising the following steps:

[0043] Step 1: Divide the crops in the study area into different zones and apply different water and fertilizer treatments to the crops in different zones;

[0044] In this step, the irrigation treatment includes moderate irrigation treatment W1 and low moisture treatment W2, the fertilization treatment includes mild nitrogen stress treatment N1, moderate nitrogen stress treatment N2, moderate nitrogen stress treatment N3 and severe nitrogen stress treatment N4, and any one of the water treatments (W1, W2) and fertilization treatments (N1, N2, N3, N4) are combined in pairs to obtain eight combinations, and one of the eight combinations is used for water and fertilizer treatment in different partitions.

[0045] In this embodiment, the growth of corn from the seedling stage to the mature stage in a certain irrigation and drainage test field is continuously monitored from early July to late September 2022. The irrigation and fertilization treatments are shown in Table 1. Different water and nitrogen stress treatments (one of W1N1, W1N2, W1N3, W1N4, W2N1, W2N2, W2N3, and W2N4) are adopted for different partitions. The specific pit water and nitrogen treatment is as follows: Figure 2 shown.

[0046] Table 1 Irrigation amount table for corn growth period

[0047]

[0048] The fertilization treatments (four nitrogen stress treatments) were:

[0049] N1: 45kg / hm 2 ;

[0050] N2: 120kg / hm 2 ;

[0051] N3: 240kg / hm 2 ;

[0052] N4: 360kg / hm 2 ;

[0053] Nitrogen fertilizer is applied in the form of 50% as basal fertilizer and the remaining 50% as topdressing during the jointing stage.

[0054] Step 2: Obtain meteorological information of the study area, remote sensing image data and thermal infrared image data of crops in different sub-areas, and pre-process the obtained remote sensing image data;

[0055] This example uses a test pit field in a certain irrigation and drainage test field as the research area, which is located between 113°41′ and 115°05′ east longitude and 29°58′ and 31°22′ north latitude. It is adjacent to East Lake in the north and has a subtropical monsoon climate, with an average annual precipitation of 1,300 mm and an average annual temperature of 15.8°C to 17.5°C.

[0056] The multispectral image in this embodiment comes from the Mica Sense RedEdge-M multispectral camera, which can simultaneously collect grayscale images of five channels: blue, green, red, red edge, and near infrared in one flight, with a resolution of 0.8m at an altitude of 120m. The carrier is the DJI M200 drone, which uses a flight altitude of 50m. The specific band parameter information is shown in Table 2.

[0057] Table 2 Mica Sense RedEdge-M optical sensor parameters

[0058]

[0059] The thermal infrared images in this study are from the DJI Mavic 2 Enterprise Edition, with a sensor resolution of 160×120 and a wavelength range of 8-14μm.

[0060] The multispectral camera photos were taken from July 16 to September 29, and the thermal infrared photos were taken from July 29 to September 26, covering most of the growing season.

[0061] Since the images obtained during the shooting process are discontinuous, it is difficult for the remote sensing images directly shot to accurately reflect the real characteristics of the objects. Therefore, it is necessary to correct and reconstruct the original image data to varying degrees according to the research purpose to meet the application requirements. In this step, Figure 3 As shown in the figure, the remote sensing image data was preprocessed by image stitching, geometric registration, field scale cropping, and image channel fusion. The image stitching used pix4Dmapper software to automatically generate point clouds and orthophoto mosaics; then ArcGIS Pro software was used for geometric registration; finally, ArcMap software was used to create field shapefiles, and the gdal library in Python was used to crop images and perform channel fusion.

[0062] Step 3: perform cropping and data augmentation on the thermal infrared image and the preprocessed remote sensing image, annotate the processed image with phenological information, and obtain the crop data set according to the corresponding irrigation and fertilization information annotated according to the partition in step 1, and divide the data set into a training set, a validation set, and a test set;

[0063] High-precision sample phenological period annotation and large-scale data set construction are the basis for phenological monitoring and identification. Based on the results of field investigation and photography, this study was able to determine the specific phenological labels for each period and manually annotated the photos, ultimately obtaining photos of the entire growth period of corn and phenological annotations.

[0064] Since the sizes of the cropped fields are not completely consistent, this embodiment fills all the images with 224×224 size pictures, generates 220 multispectral images and 220 thermal infrared images, and then performs data augmentation on the original data to further increase the quantity and diversity of the data. The data augmentation methods used in this article mainly include diagonal mirroring, horizontal flipping, vertical flipping and Gaussian blurring. Diagonal mirroring means flipping the image along the diagonal, horizontal flipping means flipping the image along the vertical axis, and vertical flipping means flipping the image along the horizontal axis. All three are ways to increase data diversity; Gaussian blurring is an image smoothing method that reduces noise and details in the image by applying a Gaussian filter, which helps to remove unnecessary subtle differences and abnormal noise, thereby focusing on the main features of the image. The above image augmentation method uses the numpy library in Python to complete the operation, and finally expands the number of samples to 990 images of each type, including 594 training sets, 198 validation sets, and 198 test sets.

[0065] Step 4: Construct a crop growth monitoring deep learning model for predicting crop BBCH values. Take the meteorological information of the study area as input, use the training set to train the model parameters of the crop growth deep learning model and verify it on the validation set, then use the test set to test the accuracy, and select the crop growth deep learning model with the required accuracy as the optimized crop growth deep learning model;

[0066] like Figure 4 As shown in the figure, the deep learning model for crop growth monitoring consists of two parts. The first part includes the Feature Fusion module, which is used to process the input remote sensing images and thermal infrared images, that is, the remote sensing images and thermal infrared images of the same field at the same time are gradually extracted and compressed using convolution and pooling layers, and the features are superimposed and fused; the second part includes the Multi-Head Self-Attention module and the MLP module. The second part performs segmentation and encoding processing after receiving the fused features, and uses the self-attention module to evaluate the influence of different parts on image feature extraction.

[0067] In the process of image feature extraction, a single remote sensing image input may result in insufficient feature extraction or fall into a local optimal solution. This usually requires the creation of a large-scale data set and the introduction of pre-trained weights. Convolution operations are performed on the two images separately, and the deep features are fused to enhance the network's feature extraction capabilities and global attention, thereby improving the accuracy and robustness of training. On this basis, this embodiment improves the input module of the existing network model, performs different convolution operations on remote sensing images and thermal infrared, extracts their deep features, and then fuses the features through steps such as band calculation, outlier processing, and superposition fusion, and passes them into the Multi-Head Self-Attention module for calculation. Specifically, the 224×224×5 remote sensing image and the 224×224×1 thermal infrared image are input into the Feature fusion module. After a series of convolution transformations, torch tensors with a size of 14×14 and 256 features are obtained respectively. The accumulated temperature information (obtained from meteorological information) is converted into a 14×14×1 torch tensor. The three are concatenated through the Concat function to obtain a torch tensor with a shape of 14×14×513 and multiple features.

[0068] The Multi-Head Self-Attention module includes a multi-head self-attention layer and a fully connected layer. The input of the multi-head self-attention layer (MHSA) is the torch tensor obtained by feature extraction, which is expanded into a vector of size 196×768 through the Patch Embedding layer. MHSA maps the input to q, k, and v. After attention calculation, the output is concatenated into a vector of the same size as the input through a linear layer. Each head of the multi-head attention module can extract different information independently, so that richer information can be extracted, which is conducive to better model results. The output dimension of the fully connected layer of the Multi-Head Self-Attention module is adjusted from the number of output categories (num_class) in the original model to the output of a one-dimensional digital result. The fully connected layer inherits the output matrix of the MHSA module and continuously learns the relationship between the model output value and the true value by adjusting the weights. Finally, the BBCH value of the crop corresponding to the one-dimensional result is obtained. BBCH is a set of standard methods for describing the growth status of crops. BBCH comes from the Biologische Bundesanstalt, Bundessortenamt and CHemical industry. The growth cycle of crops is divided into 10 main growth and development stages, and each growth and development stage is divided into 10 secondary growth stages. From 00-99, it is represented by numbers, 00 means that the seeds have not yet germinated, and 99 means after harvest or in storage. BBCH is currently a relatively common method for describing crop growth and development internationally. This fully connected layer is used to invert the regression problem of the BBCH value without changing the input data and the main network structure. Its loss function uses the mean square error (MSE), which is the average square difference between the predicted value and the true value. The reason for choosing MSE as the loss function is that the mean square error assumes that the prediction error of the model obeys the mean of 0 and the variance of σ 2 Normal distribution; and the second-order derivative of the MSE function is always not less than zero, that is, there is a global optimal solution, which ensures that the model can converge to the global optimal solution during training and is not affected by individual outliers, so that the model has good generalization ability and robustness. In this crop growth monitoring deep learning model, the Feature fusion module is used to extract local features, reduce the size of the feature matrix, and deepen the feature dimension to match the input of the MHSA module. The attention mechanism's focus on long-distance dependencies is used to process global features, so that the model can better adapt to information of different scales and improve the ability to understand the input feature matrix. When training more complex data, this model has higher accuracy than using a single model, while reducing the total number of parameters.

[0069] The crop growth monitoring model is trained. The hardware environment of this embodiment is as follows: the CPU is the 7th generation Intel (R) Core (TM) i7-7700, the GPU is NVIDIA GeForce RTX 3060 12G, and the memory is 24G. The software environment is as follows: the GPU computing architecture is CUDA11.9, the compilation language is Python3.9, the deep learning framework is Pytorch v13.0, the optimizer used in this study is Adam, the activation function is the ReLu function, the adaptive learning rate strategy is adopted, the initial learning rate is lr0=0.0001, the number of iterations is 200 cycles, and the loss function is the CrossEntropyLoss function. The model parameters of the crop growth monitoring deep learning model are trained using the training set and verified on the validation set. The hyperparameter settings in the crop growth monitoring deep learning model are shown in Table 3.

[0070] Table 3 Model hyperparameter value table

[0071]

[0072] In order to verify that the accuracy and feature acquisition ability of the crop growth monitoring deep model of this embodiment are higher than those of the traditional deep learning model, the test set is used for testing, and the test is carried out in the traditional deep learning model to compare the accuracy. The results are shown in Table 4.

[0073] Table 4 Recognition accuracy of different models for phenological monitoring

[0074]

[0075]

[0076] It can be seen from Table 4 that the crop growth monitoring deep model has improved accuracy compared with other models, and is more accurate than using only one image for recognition.

[0077] Step 5: Use the data in the data set to regress the relationship between BBCH value and fertigation;

[0078] In this example, the BBCH value predicted by the crop growth monitoring deep learning model was regressed with water-nitrogen stress (irrigation and fertilization), and the changes in the BBCH curve were observed to analyze the degree of water-nitrogen stress. The results are as follows: Figure 5 , Figure 6 As shown, from Figure 5 and 6It can be observed that low water treatment (W2) will significantly delay the time point when corn enters the jointing stage, while low water and high nitrogen (W2N3) treatment can alleviate this situation and prolong the flowering period. According to historical research, low irrigation combined with high nitrogen fertilizer is beneficial to improve the dry matter quality of corn. Taking the BBCH value on August 21st in the early growth period as the evaluation standard, water stress will delay the jointing stage, and high nitrogen (N3) will alleviate this situation. Under the condition of sufficient irrigation (W1), excessive nitrogen will also inhibit the normal growth of corn. Therefore, it can be diagnosed whether it is under water and nitrogen stress based on whether the growth of corn reaches the jointing stage (BBCH>50) at the key time point from the seedling stage to the jointing stage (August 21st). Under the condition of not excessive nitrogen application, the BBCH value on August 21st is less than 50, which means stress occurs. Then, the specific type of stress is judged according to the irrigation and fertilization amount in step 1 to determine the corresponding irrigation and fertilization amount.

[0079] Table 5 BBCH values ​​before and after jointing stage under different treatments

[0080]

[0081] In summary, using the jointing stage as a key period to diagnose corn stress can determine whether it is under water stress in the early stage of crop growth, when the LAI difference is not obvious. Corn under water stress can avoid yield reduction by supplementing nitrogen with topdressing during the jointing stage. Therefore, the model used in this section for regression calculation can, to a certain extent, diagnose stress in the early stage of crop growth and provide guidance information for subsequent water and fertilizer application and agricultural management measures.

[0082] Step 6: Use the optimized crop growth monitoring deep learning model to predict the BBCH value of the crops to be fertilized, and determine the crop irrigation and fertilization plan based on the BBCH value and the relationship obtained in step 5.

[0083] The embodiment of the present invention further provides a system according to the above-mentioned crop growth monitoring method based on multi-source images, comprising:

[0084] Data acquisition module, used to obtain meteorological information of the study area, remote sensing image data of crops in different zones, thermal infrared image data, and irrigation and fertilization data;

[0085] The data processing module is used to pre-process the remote sensing image data obtained by the data acquisition module, and to perform cropping and data augmentation processing on the thermal infrared image and the pre-processed remote sensing image;

[0086] A data set construction module is used to annotate the images processed by the data processing module with corresponding phenological information and irrigation and fertilization information to obtain a crop data set, and divide the data set into a training set, a validation set, and a test set;

[0087] The model building module is used to build a crop growth monitoring deep learning model for predicting crop BBCH values. The meteorological information of the study area is used as input. The model parameters of the crop growth deep learning model are trained using the training set in the data set building module and verified on the validation set. The accuracy is then tested using the test set. The crop growth deep learning model with the required accuracy is selected as the optimized crop growth deep learning model.

[0088] The water and fertilizer decision-making module is used to regress the relationship between the BBCH value and fertigation. The optimized crop growth monitoring deep learning model is used to predict the BBCH value of the crops to be fertilized, and the corresponding fertigation plan is formulated based on the predicted BBCH value and the relationship between the BBCH value and fertigation.

[0089] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the specification of the present invention should be included in the protection scope of the present invention.

Claims

1. A crop growth monitoring method based on multi-source images, characterized in that: The steps include: Step 1: Divide the crops in the study area into different zones and apply different irrigation and fertilization treatments to the crops in different zones; Step 2: Obtain meteorological information of the study area, remote sensing image data and thermal infrared image data of crops in different sub-areas, and pre-process the obtained remote sensing image data; Step 3: perform cropping and data augmentation on the thermal infrared images and preprocessed remote sensing images, annotate the processed images with phenological information and corresponding irrigation and fertilization treatment information to obtain a crop dataset, and divide the dataset into a training set, a validation set, and a test set; Step 4: Construct a crop growth monitoring deep learning model for predicting crop BBCH values. Take the meteorological information of the study area as input, use the training set to train the model parameters of the crop growth deep learning model and verify it on the validation set, then use the test set to test the accuracy, and select the crop growth deep learning model with the required accuracy as the optimized crop growth deep learning model; Step 5: Use the data in the data set to regress the relationship between BBCH value and fertigation; Step 6: Use the optimized crop growth monitoring deep learning model to predict the BBCH value of the crops to be fertilized, and formulate a corresponding irrigation and fertilization plan based on the predicted BBCH value and the relationship obtained in step 5.

2. The crop growth monitoring method based on multi-source images according to claim 1, characterized in that: The methods for preprocessing remote sensing image data in step 2 include image stitching, geometric registration, field scale cropping, and image channel fusion.

3. The crop growth monitoring method based on multi-source images according to claim 1, characterized in that: Step 3 in data augmentation includes diagonal mirroring, horizontal flipping, vertical flipping and Gaussian blurring of the cropped image.

4. The crop growth monitoring method based on multi-source images according to claim 1, characterized in that: The crop growth monitoring model consists of two parts. The first part includes the Feature fusion module, which is used to extract the features of the input remote sensing image and thermal infrared image, and superimpose and fuse the features of the two images; the second part includes the Multi-Head Self-Attention module and the MLP module, which are used to further extract the global image features from the fused image features and output the one-dimensional result, namely the BBCH value.

5. The crop growth monitoring method based on multi-source images according to claim 4, characterized in that: The methods for extracting image features by the Featurefusion module include: The remote sensing image and thermal infrared image are input into the Feature fusion module. After a series of convolution transformations, the corresponding torch tensors are obtained respectively, and the meteorological information is converted into a torch tensor. The three are concatenated through the Concat function to obtain a torch tensor with multiple features.

6. The crop growth monitoring method based on multi-source images according to claim 5, characterized in that: The Multi-HeadSelf-Attention module includes a multi-head self-attention layer and a fully connected layer; The torch tensor extracted by the Feature Fusion module is input into the multi-head self-attention layer, where it is expanded through the Patch Embedding layer and the input is mapped to q, k, and v. After attention calculation, the output is concatenated into a vector of the same size as the input through the linear layer. The fully connected layer adjusts the image vector in the multi-head self-attention layer to a one-dimensional result and outputs the BBCH value of the corresponding crop.

7. The crop growth monitoring method based on multi-source images according to claim 1, characterized in that: In step 6, whether the BBCH value of the crop at the key time node reaches the threshold is used to judge whether the crop is under water and nitrogen stress. If the crop is under water and nitrogen stress, an irrigation and fertilization plan is formulated at the key time node; otherwise, no irrigation and fertilization is performed.

8. The crop growth monitoring method based on multi-source images according to claim 7, characterized in that: The key time node is the jointing period of crops.

9. The crop growth monitoring method based on multi-source images according to claim 7, characterized in that: When the predicted BBCH value of the crop at the key time node is <50, the crop is judged to be under water and nitrogen stress, otherwise, it is judged not to be under water and nitrogen stress.

10. A system for implementing the crop growth monitoring method based on multi-source images according to any one of claims 1 to 9, characterized in that: include: Data acquisition module, used to obtain meteorological information of the study area, remote sensing image data of crops in different zones, thermal infrared image data, and irrigation and fertilization data; The data processing module is used to pre-process the remote sensing image data obtained by the data acquisition module, and to perform cropping and data augmentation processing on the thermal infrared image and the pre-processed remote sensing image; A data set construction module is used to annotate the images processed by the data processing module with corresponding phenological information and irrigation and fertilization information to obtain a crop data set, and divide the data set into a training set, a validation set, and a test set; The model building module is used to build a crop growth monitoring deep learning model for predicting crop BBCH values. The meteorological information of the study area is used as input. The model parameters of the crop growth deep learning model are trained using the training set in the data set building module and verified on the validation set. The accuracy is then tested using the test set. The crop growth deep learning model with the required accuracy is selected as the optimized crop growth deep learning model. The water and fertilizer decision-making module is used to regress the relationship between the BBCH value and fertigation. The optimized crop growth monitoring deep learning model is used to predict the BBCH value of the crops to be fertilized, and the corresponding fertigation plan is formulated based on the predicted BBCH value and the relationship between the BBCH value and fertigation.

Images