A method for remote sensing inversion of crop leaf nitrogen content based on a mixture model

Abstract

The application discloses a crop leaf nitrogen content remote sensing inversion method based on a hybrid model, and the method comprises the following steps: coupling a deep multi-task learning network model and an N-PROSAIL radiation transmission mechanism model to construct a hybrid model, inputting simulation data and measured data into the hybrid model to perform model training, and predicting the leaf nitrogen content of a target area based on the trained prediction model. Through the construction of two tasks (an auxiliary task trained based on simulation data and a main task trained based on measured data) with a partially shared network structure, the application extracts useful information in the simulation data for the training of the main task while training the two tasks, so as to improve the training effect of the main task, and the application combines the advantages of the empirical model method and the mechanism model method and overcomes the corresponding defects, and thus can realize accurate estimation of the leaf nitrogen content under the condition that there is less measured data for training the model.

Description

TECHNICAL FIELD

[0001] The application relates to a nitrogen content remote sensing inversion method, in particular to a crop leaf nitrogen content remote sensing inversion method based on a mixed model. BACKGROUND

[0002] Nitrogen is an important nutrient element for crop growth and development. Timely and accurate estimation of leaf nitrogen content is of great significance for carrying out precise nitrogen management of crops. Existing leaf nitrogen content inversion methods include empirical model method and mechanism model method. The empirical model method is based on the statistical relationship between crop canopy spectral reflectance and nitrogen, establishes a theoretical regression model, and realizes the inversion of crop nitrogen in the target area based on the model. This method has the advantages of easy understanding and easy operation, but also has the defects of lack of mechanism explanation and need for a large number of training samples. The mechanism model takes crop physiology and biochemical parameters as input, and describes the radiation transfer process of light in the vegetation canopy. Its reverse process can realize the inversion of crop nitrogen. The advantage of this method is that it has a mechanism explanation, but the existing mechanism model is a simplification of the real world, and it is also difficult to realize accurate simulation of the radiation transfer process, thereby causing errors in the inversion of leaf nitrogen content. The advantages of the above two methods are complementary, but so far there has been no related research on how to combine the two to establish a mixed model to estimate leaf nitrogen content.

[0003] At present, there are studies on estimating chlorophyll based on mixed models, but all have drawbacks; for example, based on multispectral unmanned aerial vehicle images, PROSAIL mechanism model is used to simulate rice canopy reflectance data, and Bayesian network (BN) model for inversion of chlorophyll content is trained to realize inversion of rice canopy chlorophyll content; again, a deep neural network model for inversion of chlorophyll is trained using wheat canopy spectral reflection data generated based on PROSAIL, and a small amount of field measured wheat samples are used to fine-tune the model parameters to realize inversion of wheat leaf chlorophyll content. These studies provide a reference for nitrogen inversion based on mixed models. But the existing modeling ideas have problems, limited by the difference between the simulation data and the measured data, this method directly uses the simulation data as part of the measured data, based on single task learning to train the network model, which will produce large errors in actual application.

[0004] In summary, in view of the defects of the prior art, in order to improve the estimation accuracy of leaf nitrogen content, a new leaf nitrogen content estimation method based on a hybrid model is provided to solve the following technical problems: 1) the empirical model method for inverting leaf nitrogen content lacks mechanism explanation and needs a large number of sample training; 2) the existing mechanism model is a simplification of the real world, and it is difficult to accurately simulate the radiation transmission process, so the existing mechanism model method for inverting leaf nitrogen content cannot support accurate inversion of leaf nitrogen content; 3) there is no method for inverting leaf nitrogen content based on a hybrid model, but there are hybrid models coupling deep learning methods and radiation transmission mechanism models to realize chlorophyll inversion. These methods can provide a reference for constructing a hybrid model of leaf nitrogen content, but they use a single-task learning algorithm, directly use model simulation data as part of the real data to train the network, and have a large error. SUMMARY

[0005] In order to solve the above technical problems, the technical scheme adopted by the present application is: a crop leaf nitrogen content remote sensing inversion method based on a hybrid model, comprising the following steps:

[0006] In order to solve the above technical problems, the technical scheme adopted by the present application is: a crop leaf nitrogen content remote sensing inversion method based on a hybrid model, comprising the following steps:

[0007] Step one, training data acquisition: including acquisition of simulation data and measured data;

[0008] Step two, multi-task learning network model training: input the simulation data and the measured data into the constructed multi-task learning network for model training;

[0009] The multi-task learning network includes three modules: a shared layer, a sub-task layer and a multi-task optimization layer; the shared layer is used for learning the common features of the simulation data and the measured data, the sub-task layer is used for learning the unique features of the simulation data and the measured data, and the multi-task optimization layer iteratively optimizes and trains the model parameters based on the error values of the two sub-tasks until an optimal trained model is obtained. The optimal model is used as a prediction model for leaf nitrogen content;

[0010] Step three, leaf nitrogen content prediction: based on the trained prediction model, input the target area remote sensing data to predict the leaf nitrogen content of the target area.

[0011] Further, in step one, the simulation data is generated by the N-PROSAIL model; the measured data is the real data obtained when measuring in the field.

[0012] Further, in step two, the shared layer is trained by simultaneously inputting the simulation data and the measured data into the deep neural network to obtain a general feature extraction model applicable to both models.

[0013] Furthermore, the shared layer consists of six fully connected neural networks with 20, 20, 20, 10, 10, and 10 nodes respectively, and the ReLU function is chosen as the activation function between each network layer.

[0014] Furthermore, the sub-task layers are trained by inputting the features extracted by the shared layer into their respective independent sub-task deep neural networks, so that the network can learn the common features of the two tasks while also paying attention to the unique features of each task.

[0015] Furthermore, the subtask layer contains two subtask deep neural networks with identical structures, each consisting of three fully connected neural networks with 10, 10, and 10 nodes respectively.

[0016] The network uses the ReLU function as the activation function in the first layer and the Tanh function in the last two layers.

[0017] Furthermore, the multi-task optimization layer employs a dynamic weighted average method to couple the errors based on simulated and measured data, using the overall mean square error (MSE) as the metric. all The objective function is minimized (t), and the model is trained iteratively.

[0018] Furthermore, the calculation process of the dynamic weighted average method is shown in formulas (1)-(4):

[0019] MSE all (t)=α k (t)×MSE k (t) Formula (1)

[0020]

[0021]

[0022]

[0023] In the formula, MSE all (t) represents the total mean square error of leaf nitrogen content during t training cycles; MSE k (t) represents the mean squared error of task k in t training iterations; a k (t) represents the standardized weights of the mean squared error of task k in training iterations t; W k (t) represents the weights of the mean squared error of task k in training iteration t; N represents the total number of tasks; r k (t-1) represents the training speed of task k in training iteration t-1; T is a constant; MSE k (t-1) and MSE k(t-2) represent the mean square error of k tasks in t-1 and t-2 training

[0024] The application discloses a leaf nitrogen content estimation method based on a hybrid model, which realizes accurate estimation of leaf nitrogen content based on the hybrid model by coupling a deep multi-task learning network model and an N-PROSAIL radiation transmission mechanism model. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A schematic diagram of a work flow of the application for inverting leaf nitrogen content.

[0026] Figure 2 A modeling result diagram of a leaf nitrogen content prediction model in the implementation of the application.

[0027] Figure 3 A verification result diagram of a leaf nitrogen content prediction model in the implementation of the application. DETAILED DESCRIPTION

[0028] The application will be further described in detail below in combination with the drawings and specific embodiments.

[0029] The application proposes a leaf nitrogen content estimation method based on a hybrid model, which forms a new method for estimating leaf nitrogen content by coupling a deep multi-task learning network model and an N-PROSAIL radiation transmission mechanism model.

[0030] As shown in Figure 1 The leaf nitrogen content estimation method based on the hybrid model comprises the following processing steps:

[0031] Step one, training data acquisition: including acquisition of simulation data and measured data; the simulation data are generated by the N-PROSAIL model; the measured data are real data obtained when the field is measured.

[0032] Step two, multi-task learning network model training: input the simulation data and the measured data into the constructed multi-task learning network for model training, and the obtained optimal model is the prediction model;

[0033] Step three, leaf nitrogen content prediction: based on the trained prediction model, the target area remote sensing data is input to predict the leaf nitrogen content of the target area.

[0034] Among them, the multi-task learning network includes three main modules: shared layer, sub-task layer and multi-task optimization layer;

[0035] 1) Shared layer: this module realizes the learning of common features of simulation data and measured data. By simultaneously inputting simulation data and measured data into a deep neural network for training, a general feature extraction model suitable for both models can be obtained. This design not only realizes information sharing between the two tasks, but also reduces the risk of overfitting of each task. The shared layer module of the present application is composed of six fully connected neural networks, and the node numbers are 20, 20, 20, 10, 10 and 10 respectively. The activation function between each layer of network is selected as ReLU function.

[0036] 2) Sub-task layer: this module realizes the learning of unique features of simulation data and measured data. By inputting the features extracted by the shared layer into independent sub-task deep neural networks respectively, the network can not only learn the common features of the two tasks, but also pay attention to the unique features of each task, effectively improving the fitting accuracy of each task. In the sub-task layer module of the present application, the two sub-task deep neural networks have consistent structure, each consisting of three fully connected neural networks with node numbers of 10, 10 and 10. Among them, the first layer of network selects ReLU function as the activation function, and the last two layers select Tanh function as the activation function.

[0037] 3) Multi-task optimization layer: this module iteratively optimizes the model parameters by combining the error values of the two sub-tasks until the optimal model is obtained. The whole process uses the method of dynamic weight average (DWA) to couple the errors based on simulation data and measured data, and the calculation formula is shown in (1)-(4). In the model training process, the total mean square error (MSE all (t)) in formula (1) is taken as the objective function to judge whether the training stops or not; before the total mean square error reaches the minimum value, the parameters are optimized in the shared layer and the sub-task layer, until the total mean square error reaches the minimum value, at which time the training is stopped, and the obtained trained model is the optimal model.

[0038] MSE all (t)=α k(t) x MSE k (t) Formula (1)

[0039]

[0040]

[0041]

[0042] MSE (t) = 1 / 2 (Y (t) - X (t) b)2 all (t) represents the overall mean square error of the nitrogen content of the leaf at the t training; MSE k (t) represents the mean square error of the k task at the t training; a k (t) represents the normalized weight of the mean square error of the k task at the t training; W k (t) represents the weight of the mean square error of the k task at the t training; N represents the total number of tasks; r k (t-1) represents the training speed of the k task at the t-1 training; T is a constant, which is set to 0.5 in the present application; MSE k (t-1) and MSE k (t-2) respectively represent the mean square error of the k task at the t-1 and t-2 training.

[0043]

EXAMPLE

[0044] A water and nitrogen coupling experiment of winter wheat was carried out at the Shandong Yucheng Agricultural Comprehensive Experiment Station of the Chinese Academy of Sciences, including 2 water treatments and 5 nitrogen fertilizer treatments, a total of 32 plots. Among them, the irrigation amount of the two water treatments was 80% and 60% of the field water holding capacity, respectively, using split plot experimental design; the nitrogen application amount of the five nitrogen fertilizer treatments was 0, 70, 140, 210, and 280 kg / ha, respectively, using randomized block experimental design, the number of repetitions of the first four nitrogen fertilizer treatments was 3, and the number of repetitions of the last one was 4. Except for nitrogen fertilizer, other field management measures were the same.

[0045] 1) In the pre-elongation stage, flowering stage and grain filling stage of wheat, DJI M600 multi-rotor unmanned aerial vehicle carrying S185 hyperspectral sensor was used to obtain field wheat hyperspectral image, and after the unmanned aerial vehicle flew, ground sample points were set in each plot, field sampling was carried out, and leaf nitrogen content data of each plot was obtained. Three quarters of these measured data were used as training data for training the main task of the hybrid model of the present application, and the remaining one quarter of data was used as test data for verifying the prediction performance of the trained model.

[0046] 2) According to the range of wheat leaf nitrogen content, leaf structure, leaf area index, solar zenith angle, hot spot, leaf inclination angle, leaf dry matter, equivalent water thickness, brown pigment and other parameters, different sampling methods are used, and 20000 pieces of related data are simulated based on the N-PROSAIL model to train the training data of the mixed model auxiliary task of the application. The parameter range and sampling method are shown in Table 1.

[0047] Table 1 Range of N-PROSAIL model input parameters and sampling method

[0048]

[0049] 3) The training data of the simulated data and the measured data are input into the mixed model of the multi-task learning network for model training. In the training process, the Adam algorithm is used as the parameter optimizer of the model, the learning rate of the model is set to 0.001, the weight decay coefficient is set to 0.0016, the learning rate decay strategy is to reduce to 0.2 times of the original every 20 cycles, and the maximum cycle number is set to 100. Each training extracts 20 and 8 samples from the simulated data and the measured data for joint training, and 1000 times of training is regarded as a cycle.

[0050] 4) The optimal model obtained by training is used as a prediction model of leaf nitrogen content, and the leaf nitrogen content of the test data of the measured data is predicted based on the prediction model. The training results and test results of the measured data are shown in Figure 2 and Figure 3 , respectively, which show the modeling results and verification results of the leaf nitrogen content prediction model. Based on the figures and the correlation coefficient (R 2 ), the root mean square error (RMSE), it is shown that the prediction model can accurately predict the leaf nitrogen content data.

[0051] Therefore, the leaf nitrogen content estimation method based on the mixed model proposed in the application is a method for estimating crop leaf nitrogen content based on the coupling of deep multi-task learning method and N-PROSAIL mechanism model, which fully considers the situation that the simulation data of the mechanism model contains useful information for model training, but it is difficult to fully reflect the real data. The simulation data and the real data are set as different training tasks by using the multi-task learning strategy, and the two tasks share part of the network. The prediction errors of the two tasks are considered by dynamic weighting method, so as to extract useful information from the simulation data to assist the training of the real data network model, and improve the leaf nitrogen content inversion effect of the model. This method combines the advantages of empirical model method and mechanism model method, and overcomes their corresponding defects. This method is a new method for estimating leaf content based on mixed model, which realizes accurate estimation of leaf nitrogen content under the condition of training model with less measured data.

[0052] The above embodiments are not intended to limit the present application, and the present application is not limited to the above examples. Changes, modifications, additions, or substitutions made by those skilled in the art within the technical scope of the present application are also within the scope of the present application.

Claims

1. A method for remote sensing inversion of crop leaf nitrogen content based on a mixture model, characterized in that: The method comprises the following steps: Step one, training data acquisition: including the acquisition of simulation data and measured data; Step two, multi-task learning network model training: input the simulation data and measured data into the constructed multi-task learning network for model training; The multi-task learning network comprises three modules: a shared layer, a sub-task layer and a multi-task optimization layer; the shared layer is used for learning the common features of the simulation data and the measured data, the sub-task layer is used for learning the unique features of the simulation data and the measured data, and the multi-task optimization layer iteratively optimizes and trains the model parameters by comprehensively considering the error values of the two sub-tasks until an optimal trained model is obtained, so as to use the optimal model as a prediction model of leaf nitrogen content; The shared layer is composed of six full connection neural networks; the subtask layer contains two subtask deep neural networks, the two subtask deep neural networks are consistent in structure and are each composed of three full connection neural networks; the multi-task optimization layer adopts a dynamic weight weighted average method to couple the errors based on simulation data and measured data, so as to minimize the overall mean square error MSE all (t) the minimum target function, model iterative training; Step three, leaf nitrogen content prediction: based on the trained prediction model, the target area remote sensing data is input to predict the leaf nitrogen content of the target area.

2. The method according to claim 1, wherein: In step one, the simulation data is generated by the N-PROSAIL model; the measured data is the real data obtained when the field is measured.

3. The method according to claim 1, wherein: In step two, the shared layer is trained by simultaneously inputting the simulation data and the measured data into the deep neural network to obtain a general feature extraction model suitable for both models.

4. The method according to claim 3, wherein: The node numbers of the shared layer are 20, 20, 20, 10, 10 and 10 respectively, and the activation function between each layer of network is selected as ReLU function.

5. The method according to claim 3, wherein: The sub-task layer is trained by inputting the features extracted by the shared layer into independent sub-task deep neural networks respectively, so that the network can learn the common features of the two tasks while paying attention to the unique features of each task.

6. The method according to claim 5, wherein: The node numbers of the three-layer fully connected neural network of the sub-task layer are 10, 10 and 10 respectively; The first layer of the network selects ReLU function as the activation function, and the last two layers select Tanh function as the activation function.

7. The method according to claim 6, wherein: The calculation process of the dynamic weight weighted average method is shown in formulas (1)-(4): Equation (1) Equation (2) Equation (3) Equation (4) where MSE all (t) represents the overall mean squared error of the leaf nitrogen content at the tth training; MSE k (t) represents the mean squared error of the kth task at the tth training; a k (t) represents the normalized weight of the mean squared error of the kth task at the tth training; W k (t) represents the weight of the k task at the t training mean square error; N represents the total number of tasks; r k (t-1) represents the training speed of the k task at the t-1 training; T is a constant; MSE k (t-1) and MSE k (t-2) respectively represent the mean square error of the k task in the t-1 and t-2 training.

Images