Soil organic matter deep learning inversion method with addition of error consideration mechanism

Abstract

The invention discloses a soil organic matter deep learning inversion method added with an error consideration mechanism. The method comprises the following steps: acquiring data, and constructing a feature data set; carrying out sampling point layout by utilizing conditional Latin hypercube sampling; detecting the soil sample; calculating a Pearson's correlation coefficient of spectral information of the satellite remote sensing image and organic matters, and selecting spectral features as input features based on correlation; performing model optimization by using a deep learning algorithm to obtain an optimal model; searching a most reasonable value in a true value error range by using a genetic algorithm, and storing a corresponding deep learning model at the moment; and performing soil organic matter inversion in a research area by using the model, and drawing a fertility distribution diagram by using drawing software. According to the invention, soil organic matter inversion is realized by using a machine learning algorithm, the sampling frequency is greatly reduced, the inversion precision of the model is improved, and thus the drawing of a soil organic matter distribution diagram in a large farm scale and a county scale is realized.

Description

Technical Field

[0001] This invention relates to the field of agricultural technology, and in particular to a deep learning inversion method for soil organic matter with an added error consideration mechanism. Background Technology

[0002] In modern agricultural production, scientific management of fertilization and increasing grain yield are crucial issues that urgently need to be addressed. Organic matter is an important component of soil, a key indicator of soil fertility, providing essential nutrients for crops and determining the soil's water and fertilizer retention capacity. Measuring soil organic matter content allows us to understand soil fertility status and replenish soil nutrients through precise fertilization, thereby improving soil fertility and ensuring both crop yield and quality.

[0003] There are two main methods for constructing conventional soil organic matter content distribution maps: geostatistical spatial interpolation methods (Kriging interpolation, inverse distance weighting, etc.) and geographic information system (GIS) machine learning algorithms (deep learning algorithms, Lasso regression algorithms, etc.). Spatial interpolation methods mainly rely on the geospatial correlation of nutrients, ignoring the influence of other factors. Their accuracy is primarily affected by the sample density, and accurate fertility distribution calculations require a large amount of measured data, which is very costly. Soil fertility distribution inversion algorithms based on remote sensing imagery, which have been developed in recent years, can also achieve good results with reduced sample density. However, due to the influence of environmental factors in different regions, these models often only have good fitting ability for specific areas and lack strong generalization. Moreover, they still require a large number of samples in high-dimensional feature spaces. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology. To achieve the above objective, a deep learning inversion method for soil organic matter with added error consideration mechanism is adopted to solve the problems mentioned in the background technology.

[0005] A deep learning inversion method for soil organic matter with added error consideration mechanism includes the following steps:

[0006] Step S1: Acquire satellite remote sensing images and ground positioning data, and construct a feature dataset;

[0007] Step S2: Use conditional Latin hypercube sampling to set up sampling points;

[0008] Step S3: Collect soil samples and test the soil samples;

[0009] Step S4: Calculate the Pearson correlation coefficient between the spectral information of the satellite remote sensing image and the organic matter, and select spectral features as input features based on the correlation.

[0010] Step S5: Optimize the model using deep learning algorithms to obtain the optimal model;

[0011] Step S6: Use a genetic algorithm to find the most reasonable value within the error range of the true value, and save the corresponding deep learning model at this time;

[0012] Step S7: Use the model to retrieve soil organic matter within the study area and use drawing software to draw a fertility distribution map.

[0013] As a further aspect of the present invention: the ground positioning data in step S1 specifically includes ground latitude and longitude, slope, aspect, and elevation data.

[0014] As a further aspect of the present invention: the specific steps of conditional Latin hypercube sampling in step S2 include:

[0015] Step S201: Set the total number of sample points that meet the conditions to N, and the feature dimension to k. Then the size of the sample data X is N×k.

[0016] Step S202: Divide X into n equally probable layers according to the distribution of each feature dimension, where n is the required number of samples, and extract sample x as a subsample of X.

[0017] Step S203: Define matrix η, where each element represents the number of x samples in each layer of the sample X.

[0018]

[0019] Step S204: Extract the quantiles of variable n Where j = 1, ..., k; i = 1, ..., n+1, calculate the correlation coefficient matrix C of X;

[0020] Step S205: Randomly select n samples, denoted as x, and the remaining samples are denoted as r. The correlation coefficient matrix corresponding to x is T.

[0021] Step S206: Define the objective function to obtain the quantity that needs to be maximized or minimized. The formula for the objective function is:

[0022]

[0023] in, It is x j Falling into the quantile and The number between;

[0024] Add another objective function to ensure the consistency of the correlation between the sampling and the original sample features:

[0025]

[0026] Among them, c is an element of the X quantile matrix C, and t is an element of the x quantile matrix T;

[0027] The overall objective function is:

[0028] O = w1O1 + w2O2

[0029] Among them, w is the weight of each element, and usually can be set to 1;

[0030] Step S207, execute the simulated annealing algorithm to select the optimal sampling:

[0031]

[0032] Among them, ΔO is the objective function variable, T is the cooling temperature, and it decreases as the number of iterations increases;

[0033] Step S208, generate a uniform random number rand. If rand < Metro, accept the new solution, otherwise discard the new solution;

[0034] Step S209, attempt to execute the change: generate a random number rand with a standard distribution;

[0035] If rand < p, randomly select a sample from the sample set x and swap it with the remaining sample set r; otherwise, replace the sample with the largest value in the sample set x with the sample in the remaining sample set;

[0036] Among them, the value of p is between 0 and 1, indicating that the search is a random search or the probability of systematically replacing the least suitable layer;

[0037] Step S210, repeat the process of steps S206 to S209 until an optimal solution that meets the conditions is found.

[0038] As a further solution of the present invention: the specific steps in step S4 include:

[0039] S401, calculate the Pearson correlation coefficient between the characteristics X L of the data set and the organic matter content. L is the feature dimension number. The calculation formula of the Pearson correlation coefficient is as follows:

[0040]

[0041] Among them, E represents the mathematical expectation;

[0042] S402, calculate the correlation coefficients between each feature and the organic matter content in turn, and select the features whose absolute value of the correlation coefficient is greater than the preset value.

[0043] As a further aspect of the present invention, step S6 specifically includes the following steps:

[0044] Step S601: Given the error range ε ​​of the output variable of the training set, and randomly select values ​​within the error range to generate several individuals and construct a population. The population size is determined according to the size of the training set samples.

[0045] Step S602: Calculate the fitness of each individual in the population. Fitness refers to the degree of advantage of an individual in the survival of the population. The fitness function is used to calculate the fitness.

[0046] Based on the problem itself, the fitness function is determined as follows:

[0047]

[0048] Among them, R 2 The coefficient of determination is represented by "Train" and "Test," which correspond to the training and test sets, respectively. The coefficient of determination is an important metric for evaluating the quality of a model and is defined as follows:

[0049]

[0050] in, For the predicted value, y i is the true value, This is the average value;

[0051] Step S603: Apply selection, crossover, and mutation genetic operators to the population to generate a new generation based on better individuals from the current generation;

[0052] Step S604: Recalculate the fitness of each individual in the new generation population. If the termination condition is not met, repeat the process of S603. If the condition is met, the program terminates and selects the individual with the highest fitness as the final optimization result.

[0053] Step S605: Adjust the value of ε multiple times, repeat the above operation, and select the optimal model.

[0054] As a further aspect of the present invention, step S7 specifically includes the following steps:

[0055] Step S701: Obtain a vector region map of the research objects within the region;

[0056] Step S702: Import the generated geographic coordinates and soil organic matter inversion results into the drawing software;

[0057] Step S703: For the points corresponding to outliers, select the Kriging interpolation algorithm for interpolation analysis;

[0058] Step S704: Generate a soil fertility distribution map within the study area.

[0059] Compared with the prior art, the present invention has the following technical advantages:

[0060] By adopting the above technical solution, and through the conditional Latin hypercube sampling method and the soil fertility distribution inversion algorithm that considers errors, compared with traditional spatial interpolation methods and machine learning algorithms, the conditional Latin hypercube sampling method can effectively avoid repeated sampling under the same features, greatly reducing the sampling frequency and saving detection costs. At the same time, the error information of agricultural data is considered in the training of the deep learning algorithm model, which improves the model inversion accuracy and ensures the overall generalization ability of the solution. This invention adds an error consideration mechanism to the deep learning model, so that the overall process of constructing the soil fertility distribution map requires fewer sampling points, has higher inversion accuracy, and stronger generalization ability. Attached Figure Description

[0061] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings:

[0062] Figure 1 This is a schematic diagram illustrating the steps of the deep learning inversion method for soil organic matter according to an embodiment of this application;

[0063] Figure 2 This is a flowchart illustrating the deep learning inversion method for soil organic matter according to an embodiment of this application. Detailed Implementation

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0065] Please refer to Figure 1 and Figure 2 In this embodiment of the invention, a deep learning inversion method for soil organic matter with added error consideration mechanism includes the following steps:

[0066] Step S1: Acquire satellite remote sensing images and ground positioning data, and construct a feature dataset;

[0067] In this embodiment, the ground positioning data specifically includes ground latitude and longitude, slope, aspect, and elevation data.

[0068] Step S2: Use conditional Latin hypercube sampling to set up sampling points;

[0069] In this embodiment, the conditions for soil organic matter testing specifically include the following points:

[0070] 1. Sampling points should be selected in areas where the soil type characteristics are obvious, the terrain is relatively flat and stable, and the vegetation is good;

[0071] 2. No sampling points will be set up at locations with subordinate landscape features, such as the foot of slopes and depressions;

[0072] 3. Sampling points should not be set up in areas with high human interference, such as towns, residential areas, roads, ditches, latrines, and graves, as these areas lose the representativeness of the soil. Sampling points should be at least 20 meters away from railways and highways.

[0073] 4. Do not set up sampling points in areas with severe soil erosion or damaged topsoil;

[0074] 5. Do not set up sampling points in small, peripheral areas.

[0075] All of the above conditions can be screened and identified using remote sensing images and DEM images from different periods.

[0076] The specific steps of conditional Latin hypercube sampling include:

[0077] Step S201: Set the total number of sample points that meet the conditions to N, and the feature dimension to k. Then the size of the sample data X is N×k.

[0078] Step S202: Divide X into n equally probable layers according to the distribution of each feature dimension, where n is the required number of samples, and extract sample x as a subsample of X.

[0079] Step S203: Define matrix η, where each element represents the number of x samples in each layer of the sample X.

[0080]

[0081] The columns of the matrix represent layers S1, S2, ..., S... n The rows represent variables X1, X2, ..., X. k In a standard Latin hypercube sampling matrix, each element is 1;

[0082] Step S204: Extract the quantiles of variable n Where j = 1, ..., k; i = 1, ..., n+1, calculate the correlation coefficient matrix C of X;

[0083] Step S205: Randomly select n samples, denoted as x, and the remaining samples are denoted as r. The correlation coefficient matrix corresponding to x is T.

[0084] Step S206: Define the objective function to obtain the quantity that needs to be maximized or minimized;

[0085] In this embodiment, the objective function, also known as the optimization criterion, is a core concept in optimization theory. It is a mathematical expression that represents the objective to be optimized, usually expressed in the form of a function. In an optimization problem, the objective function defines the quantity to be maximized or minimized;

[0086] The formula of the objective function is:

[0087]

[0088] where, is the quantity that x j falls within the quantile and ;

[0089] Add another objective function to ensure the consistency of the correlation between the sampling and the characteristics of the original samples:

[0090]

[0091] where c is an element of the X quantile matrix C, and t is an element of the x quantile matrix T;

[0092] The overall objective function is:

[0093] O = w1O1 + w2O2

[0094] where w is the weight of each element, and usually can be set to 1;

[0095] Step S207, execute the simulated annealing algorithm to select the optimal sampling:

[0096]

[0097] where ΔO is the objective function variable, and T is the cooling temperature, which decreases with the increase of the iteration number;

[0098] Step S208, generate a uniform random number rand between 0 and 1. If rand < Metro, accept the new solution, otherwise discard the new solution;

[0099] Step S209, attempt to execute the change: generate a random number rand with a standard distribution;

[0100] If rand < p, randomly select a sample from the sample set x and swap it with the remaining sample set r; otherwise, replace the sub-sample (s) with the largest value in the sample set x with the sample in the remaining sample set; ;

[0101] where the value of p is between 0 and 1, indicating the probability that the search is a random search or a systematic replacement of the least suitable layer;

[0102] Step S210: Repeat steps S206 to S209 until the optimal solution that satisfies the conditions is found.

[0103] Step S3: Collect soil samples and test the soil samples;

[0104] Step S4: Calculate the Pearson correlation coefficient between the spectral information of the satellite remote sensing image and the organic matter, and select spectral features as input features based on the correlation. The specific steps include:

[0105] S401, Calculate the dataset features X L The Pearson correlation coefficient between organic matter content and feature dimension number is calculated as follows:

[0106]

[0107] Where E represents the expected value;

[0108] S402. Calculate the correlation coefficient between each feature and the organic matter content in sequence, and select features whose absolute value of the correlation coefficient is greater than the preset value.

[0109] Step S5: Optimize the model using deep learning algorithms to obtain the optimal model;

[0110] The specific steps for building a deep learning algorithm model are as follows:

[0111] Step S501, Data Preparation: The collected data samples are randomly divided into a training set and a test set in a 7:3 ratio. Simultaneously, the input features and output parameters are standardized on the training set, as expressed by the following formula:

[0112]

[0113] Where x i f is the standardized parameter value. i Let μ be the initial value of each parameter, μ be the mean of the distribution of each parameter, and σ be the standard deviation.

[0114] The test set data is standardized according to the standardization parameters set for the training set.

[0115] Step S502, Network Initialization: Using the remote sensing band reflectance of the training set as the input variable and soil organic matter content as the output layer, the model includes one hidden layer. The number of neurons is determined based on the input feature dimension. The input layer and hidden layer, as well as the hidden layer and output layer, are activated using the Sigmoid activation function. The mathematical expression of the Sigmoid function is:

[0116]

[0117] Where x is the input feature.

[0118] In the stochastic gradient descent optimization algorithm, the weight decay is set to 0.0001, the base learning rate is set to 0.001, and the training is conducted for 300 epochs. The regression function is the mean squared error loss function, and its mathematical expression is as follows:

[0119]

[0120] where y i The original value of the output result, y i ′ represents the neural network prediction value.

[0121] Step S503, Model Training and Hyperparameter Tuning: First, perform forward propagation on the input data to calculate the error between the output and the true value. Then, backpropagate the error to update the network's weights and biases. Iterate this process repeatedly until the network performance meets the expected requirements or reaches the maximum number of iterations. Simultaneously, optimize the model's hyperparameters based on experience or using the Python AutoML module to obtain the optimal model and save it.

[0122] Step S6: Use a genetic algorithm to find the most reasonable value within the error range of the true value, and save the corresponding deep learning model. The specific steps include:

[0123] Step S601: Given the error range ε ​​of the output variable of the training set, and randomly select values ​​within the error range to generate several individuals and construct a population. The population size is determined according to the size of the training set samples.

[0124] Step S602: Calculate the fitness of each individual in the population. Fitness refers to the degree of advantage of an individual in the survival of the population. The fitness function is used to calculate the fitness.

[0125] Based on the problem itself, the fitness function is determined as follows:

[0126]

[0127] Among them, R 2 The coefficient of determination is represented by "Train" and "Test," which correspond to the training and test sets, respectively. The coefficient of determination is an important metric for evaluating the quality of a model and is defined as follows:

[0128]

[0129] in, For the predicted value, y i is the true value, This is the average value;

[0130] Step S603: Apply selection, crossover, and mutation genetic operators to the population to generate a new generation based on better individuals from the current generation;

[0131] Among these methods, selection, crossover, and mutation operations are applied to the current population to generate a new generation of population based on individuals with better performance (high fitness).

[0132] Step S604: Recalculate the fitness of each individual in the new generation population. If the termination condition is not met, repeat the process of S603. If the condition is met, the program terminates and selects the individual with the highest fitness as the final optimization result.

[0133] Specifically, the fitness of all individuals in the new generation is calculated. If the termination condition (such as the maximum number of iterations) is not met, the process jumps back to S603 to continue evolution; if the termination condition is met, the individual with the highest fitness in the current population is output as the final result.

[0134] Step S605: Adjust the value of ε multiple times, repeat the above operation, and select the optimal model.

[0135] Specifically, the key parameter value ε is adjusted multiple times, and the optimization process of S603 and S604 is completely repeated after each adjustment. Finally, the model with the best performance is selected from all the running results.

[0136] Step S7: Use the model to retrieve soil organic matter within the study area and use drawing software to create a fertility distribution map. The specific steps include:

[0137] Step S701: Obtain a vector region map of the research objects within the region;

[0138] Step S702: Import the geographic coordinates and soil organic matter inversion results generated in step S206 into the drawing software;

[0139] Step S703: For the points corresponding to outliers, select the Kriging interpolation algorithm for interpolation analysis;

[0140] Step S704: Generate a soil fertility distribution map of the study area and select an appropriate color scheme.

[0141] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is defined by the appended claims and their equivalents, all of which should be included within the scope of protection of the invention.

Claims

1. A deep learning inversion method for soil organic matter with added error consideration mechanism, characterized in that, It includes the following steps: Step S1: Obtain satellite remote sensing images and ground positioning data, and construct a feature dataset; Step S2: Use conditional Latin hypercube sampling to arrange sampling points; Step S3: Collect soil samples and detect the soil samples; Step S4: Calculate the Pearson correlation coefficient between the spectral information of the satellite remote sensing image and the organic matter, and select spectral features as input features based on the correlation; Step S5: Use a deep learning algorithm to optimize the model to obtain an optimal model; Step S6: Use a genetic algorithm to find the most reasonable value within the range of the true value error, and save the corresponding deep learning model at this time; Step S7: Use the model to invert the soil organic matter in the study area, and use drawing software to complete the drawing of the fertility distribution map.

2. The soil organic matter deep learning inversion method with added error consideration mechanism according to claim 1, characterized in that, In step S1, the ground positioning data specifically includes ground longitude and latitude, slope, aspect, and elevation data.

3. The soil organic matter deep learning inversion method with added error consideration mechanism according to claim 1, characterized in that, The specific steps of conditional Latin hypercube sampling in step S2 include: Step S201: Set the total number of sample points that meet the conditions to N, and the feature dimension to k, then the size of the sample data X is N×k; Step S202: Divide X into n equally probable layers according to the distribution of each feature dimension, where n is the number of samples to be drawn, and draw the sample x as a sub-sample of X; Step S203: Define the matrix η, and the matrix elements are the number of x included in each layer of the sample X: Step S204: Extract the quantiles of variable n Where j = 1, ..., k; i = 1, ..., n+1, calculate the correlation coefficient matrix C of X; Step S205: Randomly select n samples, denoted as x, and the remaining samples are denoted as r. The correlation coefficient matrix corresponding to x is T; Step S206: Define the objective function to obtain the quantity to be maximized or minimized. The formula of the objective function is: in, It is x j Falling into the quantile and The number between; Add another objective function to ensure the consistency of the correlation between the sampling and the original sample features: Among them, c is an element of the quantile matrix C of X, and t is an element of the quantile matrix T of x; The overall objective function is: O = w1O1 + w2O2 Among them, w is the weight of each element, and usually can be set to 1; Step S207: Execute the simulated annealing algorithm to select the optimal sampling: Among them, ΔO is the variable of the objective function, and T is the cooling temperature, which decreases with the increase of the iteration number; Step S208: Generate a uniform random number rand. If rand < Metro, accept the new solution, otherwise discard the new solution; Step S209: Try to execute the change: generate a random number rand with a standard distribution; If rand < p, randomly select a sample from the sample set x and swap it with the remaining sample set r; otherwise, replace the sub-sample(s) with the largest value in the sample set x with the samples in the remaining sample set; ​ Among them, the value of p is between 0 and 1, indicating that the search is a random search or the probability of systematically replacing the least suitable layer; Step S210: Repeat the process of steps S206 to S209 until an optimal solution that meets the conditions is found.

4. The soil organic matter deep learning inversion method with added error consideration mechanism according to claim 1, characterized in that, The specific steps in step S4 include: S401, Calculate the dataset features X L The Pearson correlation coefficient between organic matter content and feature dimension number is calculated as follows: Among them, E represents the mathematical expectation; S402: Calculate the correlation coefficient between each feature and the organic matter content in turn, and select the features whose absolute value of the correlation coefficient is greater than the preset value.

5. The soil organic matter deep learning inversion method with added error consideration mechanism according to claim 1, characterized in that, The specific steps in step S6 include: Step S601: Given the error range ε of the output variable of the training set, and randomly take values within the error range to generate several individuals, construct a population, and the population size is determined according to the sample size of the training set; Step S602: Calculate the fitness of each individual in the population. Fitness refers to the degree of advantage of an individual in the survival of the population. The fitness function is used to calculate the fitness. Based on the problem itself, the fitness function is determined as follows: Among them, R 2 The coefficient of determination is represented by "Train" and "Test," which correspond to the training and test sets, respectively. The coefficient of determination is an important metric for evaluating the quality of a model and is defined as follows: in, For the predicted value, y i For the true value, This is the average value; Step S603: Apply selection, crossover, and mutation genetic operators to the population to generate a new generation based on better individuals from the current generation; Step S604: Recalculate the fitness of each individual in the new generation population. If the termination condition is not met, repeat the process of S603. If the condition is met, the program terminates and selects the individual with the highest fitness as the final optimization result. Step S605: Adjust the value of ε multiple times, repeat the above operation, and select the optimal model.

6. The soil organic matter deep learning inversion method with added error consideration mechanism according to claim 1, characterized in that, The specific steps in step S7 include: Step S701: Obtain a vector region map of the research objects within the region; Step S702: Import the generated geographic coordinates and soil organic matter inversion results into the drawing software; Step S703: For the points corresponding to outliers, select the Kriging interpolation algorithm for interpolation analysis; Step S704: Generate a soil fertility distribution map within the study area.

Images