Deep neural network soil organic carbon content prediction method based on attention mechanism

Abstract

The invention relates to the technical field of deep learning, and discloses a deep neural network soil organic carbon content prediction method based on an attention mechanism, and the method comprises the following steps: obtaining diffuse reflection spectrum data of a soil sample by using a spectrometer; to-be-measured soil diffuse reflection spectrum data are input into the model, the organic carbon content of the soil sample is predicted, and the model is a pre-established deep neural network soil organic carbon content prediction model based on an attention mechanism; the invention provides a deep neural network soil organic carbon content prediction method based on an attention mechanism, and solves the technical problems that a soil sample shows higher diversity and a simple linear model cannot be suitable for prediction of soil organic carbon content in all regions in the related technology.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and more specifically, to a method for predicting soil organic carbon content based on a deep neural network with an attention mechanism. Background Art

[0002] Soil organic carbon is one of the key indicators of ecosystem health, affecting soil fertility and vegetation growth. Therefore, it is necessary to measure the content of soil organic carbon quickly and accurately. At present, the measurement of soil organic carbon mainly relies on laboratory chemical detection. Although chemical detection methods can provide accurate test results, they are expensive and time-consuming, and have high requirements for equipment and operators. Infrared spectroscopy technology is a qualitative and quantitative measurement method that has developed rapidly in recent years. By establishing a correlation model between the spectrum and sample characteristics, the chemical and physical properties of the sample can be quantitatively and qualitatively analyzed. Infrared spectroscopy technology plays an important role in analyzing the specific components of the soil and is widely used to evaluate various components of the soil. Different soil components absorb light of specific wavelengths according to their molecular structure and composition, especially in the carbon, nitrogen and oxygen bond regions bound to hydrogen atoms. By establishing the relationship between these soil components and their characteristic absorption bands, the measurement of soil components can be achieved.

[0003] Previous studies have often been limited to a narrow range of a single region. Due to the small differences in soil, the modeling sample size used is usually small (hundreds of samples), which limits the applicability of the model. In addition, although small-scale models can provide accurate prediction results, their applicability is limited, they lack generalization ability, and are not suitable for other regions. However, with the establishment of a global soil spectral database, a rich data foundation has been provided for the study of large-scale soil composition prediction models. Over a large area, soil samples show a higher diversity, which requires the use of more complex nonlinear mathematical models to describe this diversity. Deep learning methods, especially convolutional neural networks (CNNs), provide new possibilities in this regard. In traditional CNN architectures, the relative insensitivity to the spatial layout of input features may lead to a decline in model performance. This structural limitation needs to be considered when designing and applying CNN models. Summary of the invention

[0004] The present invention provides a method for predicting soil organic carbon content using a deep neural network based on an attention mechanism, which solves the technical problem in the related art that soil samples show higher diversity and simple linear models cannot be applied to predict soil organic carbon content in all regions.

[0005] The present invention provides a method for predicting soil organic carbon content using a deep neural network based on an attention mechanism, comprising the following steps:

[0006] The diffuse reflectance spectrum data of soil samples are obtained using a spectrometer;

[0007] The soil diffuse reflectance spectrum data to be tested is input into the model to predict the organic carbon content of the soil sample. The model refers to a pre-built deep neural network soil organic carbon content prediction model based on the attention mechanism.

[0008] Furthermore, the model structure includes multiple parallel convolutional layers, multi-scale attention mechanism modules, pooling layers and fully connected layers, and a multi-scale attention mechanism module is added after the multiple convolutional layers.

[0009] Furthermore, the parallel convolution layer includes three convolution branches. The input tensor x is processed by three convolution branches. Each convolution branch uses a different convolution kernel to process. The outputs of these branches are concatenated in the channel dimension to form a new input tensor y.

[0010] y=Concat(y 1×1 ,y 3×3 ,y 5×5 )

[0011] Among them, y 1×1 ,y 3×3 ,y 5×5 They are different convolution kernel sizes respectively. The Concat operation merges the outputs of each convolution branch along the channel dimension.

[0012] Furthermore, a multi-scale attention mechanism module is connected after the parallel convolution layer to capture the importance of different channels in the input features. Let x be the input tensor with the shape of . For each channel, the vector f is obtained by the adaptive average pooling and adaptive maximum pooling of the pooling layer. avg and vector f max .

[0013] f avg =AvgPool(x)

[0014] f max =MaxPool(x).

[0015] Furthermore, the vector f avg and vector f max After being further processed by the fully connected network, the output vectors of the two pooling strategies are passed through the fully connected layer to obtain FC(f avg ) and FC(f max );

[0016] FC(f)=W2·ReLU(W1·f)

[0017] Where W1 and W2 are the weight matrices of the fully connected layer, FC(f avg ) and FC(f max) are added together and the Sigmoid function is used to convert these weights into values ​​between 0 and 1.

[0018] Finally, these weights are multiplied by the original input vector x to obtain x';

[0019] x'=x×(sigmod((FC(f avg )+FC(f max )))).

[0020] Furthermore, the model training method includes:

[0021] Remove the organic carbon content data below the instrument detection limit, remove the spectral edge bands, and perform filtering, and divide the model training set and test set into a 3:1 ratio;

[0022] Preprocess the diffuse reflectance spectral data and organic carbon content data of the training set and test set to construct features;

[0023] Determine the organic carbon content data of soil samples to construct a label matrix;

[0024] The model is supervised and trained using the training set, and the model performance is tested using the test set.

[0025] Furthermore, using R 2 , RMSE, MSE, RPIQ and RPD indicators are used to evaluate the model performance. The specific formula is as follows:

[0026]

[0027]

[0028] in, is the predicted value, y i is the true value, is the sample mean, SD is the sample standard deviation, n is the number of samples, Q3 and Q1 are the 1st and 3rd quartiles of the samples respectively.

[0029] Furthermore, in Step 1, diffuse reflectance spectra of soil samples at different angles are collected to form a multi-channel characteristic spectrum.

[0030] Furthermore, Step 1 also includes a data preprocessing stage, in which the spectral data in the wavelength range of 400 to 500 nm are removed and the first-order derivative processing is performed on each spectrum.

[0031] The present invention provides a deep neural network model based on an attention mechanism, comprising multiple parallel convolutional layers, a multi-scale attention mechanism module, a pooling layer and a fully connected layer, wherein a multi-scale attention mechanism module is added after the multiple convolutional layers.

[0032] The beneficial effects of the present invention are:

[0033] Compared with traditional chemical analysis methods, the method of the present invention has the advantages of low cost, high speed and strong scalability, and is particularly suitable for large-scale soil monitoring and management. Through accurate soil organic carbon prediction, it can provide a scientific basis for agricultural production and promote the precision and sustainability of soil management.

[0034] Through the implementation of the above method, the present invention not only improves the accuracy of soil organic carbon content prediction, but also has the characteristics of simple operation and high cost-effectiveness, providing a new technical means for soil science research and related agricultural environment practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the method for predicting soil organic carbon content based on a deep neural network with an attention mechanism of the present invention;

[0036] Figure 2 It is a structural schematic diagram of the model of the present invention;

[0037] Figure 3 It is a graph of the model performance evaluation results on the test set of the present invention. DETAILED DESCRIPTION

[0038] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that the discussion of these implementations is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the present specification. Various examples may omit, replace, or add various processes or components as needed. In addition, the features described in some examples may also be combined in other examples.

[0039] At least one embodiment of the present invention discloses a method for predicting soil organic carbon content using a deep neural network based on an attention mechanism, such as Figure 1 As shown, the following steps are included:

[0040] Step 1: Use a spectrometer to obtain diffuse reflectance spectrum data of soil samples;

[0041] Step 2: Input the soil diffuse reflectance spectrum data to be tested into the model to predict the organic carbon content of the soil sample. The model refers to a pre-built deep neural network soil organic carbon content prediction model based on the attention mechanism.

[0042] In one embodiment of the present invention, the training method of the model includes:

[0043] Remove the organic carbon content data below the instrument detection limit, remove the spectral edge bands, and perform filtering, and divide the model training set and test set into a 3:1 ratio;

[0044] Preprocess the diffuse reflectance spectral data and organic carbon content data of the training set and test set to construct features;

[0045] Determine the organic carbon content data of soil samples to construct a label matrix;

[0046] The model is supervised and trained using the training set, and the model performance is tested using the test set.

[0047] Furthermore, the model hyperparameter space is defined, the Bayesian optimization technique is used to search for the optimal parameters, and the optimal hyperparameter space is determined based on the RMSE value of the validation set.

[0048] In one embodiment of the present invention, in Step 1, diffuse reflectance spectra of soil samples at different angles are collected to form a multi-channel characteristic spectrum.

[0049] In one embodiment of the present invention, the model structure includes multiple parallel convolutional layers, multi-scale attention mechanism modules, pooling layers and fully connected layers, and a multi-scale attention mechanism module is added after the multiple convolutional layers;

[0050] In one embodiment of the present invention, the model also includes a batch normalization layer to convert the inter-layer outputs to a standard format for neural networks, called normalization. This effectively "resets" the distribution of the outputs of the previous layer so that they can be more efficiently processed by subsequent layers.

[0051] In one embodiment of the present invention, the model further includes an activation layer for introducing nonlinearity.

[0052] In one embodiment of the present invention, the parallel convolution layer includes three convolution branches, the input tensor x is processed by the three convolution branches, each convolution branch is processed by a different convolution kernel, and the outputs of these branches are spliced ​​in the channel dimension to form a new input tensor y;

[0053] y=Concat(y 1×1 ,y 3×3 ,y 5×5 )

[0054] Among them, y 1×1 ,y 3×3 ,y 5×5 They are different convolution kernel sizes respectively. The Concat operation merges the outputs of each convolution branch along the channel dimension.

[0055] In one embodiment of the present invention, a multi-scale attention mechanism module is connected after the parallel convolution layer to capture the importance of different channels in the input features. Let x be the input tensor with a shape of (batch size, number of channels, sequence length). For each channel, the vector f is obtained by adaptive average pooling and adaptive maximum pooling of the pooling layer. avg and vector f max .

[0056] f avg =AvgPool(x)

[0057] f max =MaxPool(x)

[0058] Furthermore, each value is further processed by the fully connected network, and the output vectors of the two pooling strategies are respectively obtained after passing through the fully connected layer. avg ) and FC(f max );

[0059] FC(f)=W2·ReLU(W1·f)

[0060] Where W1 and W2 are the weight matrices of the fully connected layer, FC(f avg ) and FC(f max ) are added together and the Sigmoid function is used to convert these weights into values ​​between 0 and 1.

[0061] Finally, these weights are multiplied by the original input vector x to obtain x';

[0062] x'=x×(sigmod((FC(f avg )+FC(f max ))))

[0063] As a preferred step, in Step 3, R 2 , RMSE, MSE, RPIQ and RPD indicators are used to evaluate the model performance. The specific formula is as follows:

[0064]

[0065]

[0066] in, is the predicted value, y i is the true value, is the sample mean, SD is the sample standard deviation, n is the number of samples, Q3 and Q1 are the 1st and 3rd quartiles of the samples respectively.

[0067] In this embodiment, in order to facilitate the establishment of the model, the existing LUCAS soil database is used, the Spanish country in the database is selected, and a deep neural network soil organic carbon prediction model based on the attention mechanism is established. The organic carbon content of the soil sample is determined by the ISO10694-1995 dry burning method. The soil spectrum was measured using a FOSS fast content analyzer, and the diffuse reflectance spectra of all soil samples in the visible light (400-780nm) and near infrared (780-2499.5nm) regions were collected, with a spectral resolution of 0.5nm.

[0068] In the data preprocessing stage, various measures were taken to improve the performance of the model. First, in order to eliminate the influence of instrument artifacts, the spectral data in the wavelength range of 400 to 500 nm were removed. At the same time, the first-order derivative of each spectrum was processed.

[0069] The dataset is divided into training set and test set in a ratio of 3:1 to ensure the effectiveness of model training and evaluation.

[0070] A multi-scale processing strategy is adopted to improve the network's ability to capture features at different levels.

[0071] In the present invention, convolution kernels of different scales (1×1, 3×3, 5×5) are used to capture features of different scales. By parallelizing multiple convolution kernels and pooling operations, the feature learning ability of the network is enriched, ensuring that even if the network depth and width increase, the model can effectively capture and integrate multi-scale information. An attention mechanism module is introduced after the parallel convolution layer to form a multi-scale attention block. By inserting multiple multi-scale attention blocks in the deep neural network, the overall performance of the model is improved.

[0072] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are protected by this embodiment.

Claims

1. A method for predicting soil organic carbon content based on a deep neural network with an attention mechanism, characterized in that: The following steps are involved: The diffuse reflectance spectrum data of soil samples are obtained using a spectrometer; The soil diffuse reflectance spectrum data to be tested is input into the model to predict the organic carbon content of the soil sample. The model refers to a pre-built deep neural network soil organic carbon content prediction model based on the attention mechanism.

2. According to claim 1, a method for predicting soil organic carbon content based on a deep neural network with an attention mechanism is characterized in that: The model structure includes multiple parallel convolutional layers, multi-scale attention mechanism modules, pooling layers and fully connected layers. A multi-scale attention mechanism module is added after multiple convolutional layers.

3. The method for predicting soil organic carbon content using a deep neural network based on an attention mechanism according to claim 2, characterized in that: The parallel convolution layer includes three convolution branches. The input tensor x is processed by three convolution branches. Each convolution branch uses a different convolution kernel to process the outputs of these branches in the channel dimension to form a new input tensor y. and=Concat(and 1×1 ,and 3×3 ,and 5×5 ) Among them, y 1×1 ,y 3×3 ,y 5×5 They are different convolution kernel sizes respectively. The Concat operation merges the outputs of each convolution branch along the channel dimension.

4. The method for predicting soil organic carbon content using a deep neural network based on an attention mechanism according to claim 3 is characterized in that: A multi-scale attention mechanism module is connected after the parallel convolution layer to capture the importance of different channels in the input features. Let x be the input tensor with the shape of . For each channel, the vector f is obtained by the adaptive average pooling and adaptive maximum pooling of the pooling layer. avg and vector f max ; f avg =AvgPool(x) f max =MaxPool(x)。 5. The method for predicting soil organic carbon content using a deep neural network based on an attention mechanism according to claim 4, characterized in that: Vector f avg and vector f max After being further processed by the fully connected network, the output vectors of the two pooling strategies are passed through the fully connected layer to obtain FC(f avg ) and FC(f max ); FC(f)=W2·ReLU(W1·f) Where W1 and W2 are the weight matrices of the fully connected layer, FC(f avg ) and FC(f max ) are added, and the Sigmoid function is used to convert these weights into values ​​between 0 and 1; Finally, these weights are multiplied by the original input vector x to obtain x'; x'=x×(sigmod((FC(f avg )+FC(f max ))))。 6. A method for predicting soil organic carbon content based on a deep neural network with an attention mechanism according to any one of claims 1 to 5, characterized in that: The model training methods include: Remove the organic carbon content data below the instrument detection limit, remove the spectral edge bands, and perform filtering, and divide the model training set and test set into a 3:1 ratio; Preprocess the diffuse reflectance spectral data and organic carbon content data of the training set and test set to construct features; Determine the organic carbon content data of soil samples to construct a label matrix; The model is supervised and trained using the training set, and the model performance is tested using the test set.

7. The method for predicting soil organic carbon content based on a deep neural network with an attention mechanism according to claim 6, characterized in that: Using R 2 , RMSE, MSE, RPIQ and RPD indicators are used to evaluate the model performance. The specific formula is as follows: in, is the predicted value, y i is the true value, is the sample mean, SD is the sample standard deviation, n is the number of samples, Q3 and Q1 are the 1st and 3rd quartiles of the samples respectively.

8. The method for predicting soil organic carbon content based on a deep neural network with an attention mechanism according to claim 1, characterized in that: In Step 1, diffuse reflectance spectra of soil samples at different angles are collected to form a multi-channel characteristic spectrum.

9. The method for predicting soil organic carbon content using a deep neural network based on an attention mechanism according to claim 1, characterized in that: Step 1 also includes a data preprocessing stage, in which the spectral data in the wavelength range of 400 to 500 nm are removed and the first-order derivative processing is performed on each spectrum.

10. A deep neural network model based on attention mechanism, characterized in that: It includes multiple parallel convolutional layers, multi-scale attention mechanism modules, pooling layers and fully connected layers. A multi-scale attention mechanism module is added after multiple convolutional layers.

Images