A soil data enhancement method based on spectral difference and content difference

Abstract

The present invention discloses a soil data enhancement method based on spectral difference and content difference, and relates to the field of quantitative detection of chemometrics. An original sample set is obtained; a spectral difference subtraction method based on the maximum value is used to process the original sample to obtain a number of enhanced samples, and then an enhanced sample set is obtained; different forms of data enhancement are performed on the original samples to obtain spectral difference and content difference; a soil quantitative detection model is trained and tested; an optimal soil quantitative detection model is selected from a number of soil quantitative detection models; a new sample containing a spectrum and content is obtained, and a content prediction value of the new sample is obtained based on the selected optimal soil quantitative detection model. The present invention obtains spectral difference and content difference based on the maximum or minimum value of the content through the spectrum and the content, which not only realizes spectral data enhancement, but also realizes content data enhancement; reduces the number limit of the original samples, so that the model is not limited to machine learning due to limited samples and introduces deep learning.

Description

Technical Field

[0001] The invention belongs to the field of quantitative detection of chemometrics, and in particular relates to a soil data enhancement method based on spectral difference and content difference. Background Art

[0002] Quantitative analysis of soil is conducive to precision farming and land quality monitoring. Unlike the time-consuming and labor-intensive wet chemical method, spectral-based chemometrics can meet the needs of large-scale rapid detection of soil content. Due to the differences in the spectra and content determination methods of large soil spectral libraries such as LUCAS in the European Union, Kellogg Soil Survey Laboratory (KSSL) in the United States, Chinese Soil Spectral Library (CSSL) and Chinese forestsoil spectral library (CFSSL), the Global Soil Laboratory Network (GLOSOLAN) under the Food and Agriculture Organization of the United Nations aims to unify and coordinate the spectra and content determination methods of different soil databases. Soil spectral libraries provide data support for soil analysis at the global, continental, national and regional levels, and provide conditions for large-scale quantitative detection of soil. However, small-area data sets are still limited by the number of samples, which are mainly collected by researchers themselves, ranging from dozens to hundreds, and are characterized by small scale, small sample size and simple model. At the same time, the number of samples in a small area in the soil database is also limited. At present, the mainstream method is to use simple machine learning models, while deep learning models cannot be widely used due to the limitation of sample number. Therefore, it is necessary to use data enhancement to expand the number of samples to meet the sample number requirements of deep learning models.

[0003] Spectral difference is the difference between two spectra. It is used to remove background interference and water vapor interference, spectral separation of a mixture containing two components, and monitoring of reaction processes at different times. It does not involve data enhancement. Spectral data enhancement, which draws on the idea of ​​image enhancement, simulates the noise changes of the spectrum, namely baseline shift, slope difference and intensity difference, but it does not really enhance the content but only copies it. Data enhancement based on generative adversarial networks (GAN) is currently only applicable to the classification field. A separate GAN model is built for each category to achieve spectral data enhancement for the corresponding category. At the same time, the quality of sample generation is affected by the data distribution and data quality of the original sample set, and requires a sufficient number of original samples instead of no limit on the number. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a soil data enhancement method based on spectral difference and content difference, which realizes spectral data enhancement and content data enhancement, reduces the number limit of original samples, and introduces deep learning instead of being limited to machine learning due to limited samples.

[0005] A soil data enhancement method based on spectral difference and content difference comprises the following steps:

[0006] Step 1: obtaining a number of soil samples through sample collection, measuring the spectrum and content of the soil samples to obtain a number of original samples, and then obtaining an original sample set; the original sample includes the spectrum and content of a soil sample;

[0007] Step 1.1: Collect several soil samples on site, and air-dry, grind, and screen the collected soil samples to obtain processed soil samples;

[0008] Step 1.2: Divide each soil sample in the processed soil samples into two parts, use a portable spectrometer to obtain a spectrum from one part of the soil samples, and obtain the content from the other part of the soil samples, so as to obtain the spectrum and content of each processed soil sample, that is, to obtain the original sample, and then obtain the original sample set;

[0009] Step 2: using the spectral difference subtraction method based on the maximum value to process the original sample to obtain a number of enhanced samples, and then obtain an enhanced sample set; the enhanced samples include spectral differences and content differences;

[0010] Step 2.1: Based on the content, find the original sample with the maximum content and the original sample with the minimum content, and obtain the content Y of the original sample with the maximum content max and Spectrum X max , the content Y of the original sample with the minimum content min and Spectrum X min ;

[0011] Step 2.2: The spectral difference subtraction method based on the maximum value is used to obtain the spectral difference and content difference under two different benchmarks with the maximum value and the minimum value as the benchmark, and the enhanced samples composed of the spectral difference and the content difference are obtained. In addition, under each benchmark, an enhanced sample set is obtained by eliminating the enhanced samples whose spectral difference and content difference are all 0 and an enhanced sample set is obtained by not eliminating the above enhanced samples, and finally the enhanced sample sets under four conditions are obtained;

[0012] The spectrum subtraction method based on the maximum value includes the spectrum subtraction method based on the maximum value and the spectrum subtraction method based on the minimum value;

[0013] The maximum value-based spectral difference subtraction method is a method for obtaining spectral difference and content difference based on the maximum value;

[0014] The minimum value-based spectral difference subtraction method is a method for obtaining spectral difference and content difference based on the minimum value;

[0015] The maximum value is the content Y of the original sample with the maximum content max With Spectrum X max ;

[0016] The minimum value is the content Y of the original sample with the minimum content min With Spectrum X min ;

[0017] The enhanced sample sets in the four cases are:

[0018] (1) An enhanced sample set based on the maximum value spectral difference subtraction method and retaining enhanced samples whose spectral difference and content difference are all zero;

[0019] (2) an enhanced sample set based on the maximum value spectral difference subtraction method and excluding enhanced samples whose spectral difference and content difference are all zero;

[0020] (3) an enhanced sample set based on the minimum value spectral difference subtraction method and retaining enhanced samples whose spectral difference and content difference are all zero;

[0021] (4) an enhanced sample set based on the minimum value spectral difference subtraction method and not retaining enhanced samples whose spectral difference and content difference are all zero;

[0022] The maximum value-based spectral subtraction method is:

[0023]

[0024]

[0025] Where i is the serial number of the original sample, and i = 1, 2, 3, ...., N, N is the number of original samples; is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample; Y max is the content of the original sample with the maximum content; X max is the spectrum of the original sample with the maximum content; when the enhanced samples whose spectrum difference and content difference are both 0 are not eliminated, N enhanced samples are obtained; when the enhanced samples whose spectrum difference and content difference are both 0 are eliminated, N-1 enhanced samples are obtained;

[0026] The minimum value-based spectral subtraction method is:

[0027]

[0028] In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample; Y min is the content of the original sample with the minimum content; X min is the spectrum of the original sample with the minimum content; when the enhanced samples whose spectrum difference and content difference are both 0 are not eliminated, N enhanced samples are obtained; when the enhanced samples whose spectrum difference and content difference are both 0 are eliminated, N-1 enhanced samples are obtained;

[0029] Step 3: Perform different forms of data enhancement on the original samples to obtain spectral differences and content differences, and obtain a number of enhanced samples composed of spectral differences and content differences after different forms of data enhancement. In addition, under each data enhancement method, an enhanced sample set is obtained by removing enhanced samples whose spectral differences and content differences are all 0 and an enhanced sample set is obtained by not removing the above enhanced samples, thereby obtaining multiple enhanced sample sets obtained by using different forms of data enhancement;

[0030] The data enhancement method includes all data enhancement based on the spectral difference of the maximum value and training set data enhancement based on the spectral difference of the maximum value;

[0031] The total data enhancement based on the spectrum subtraction of the maximum value includes the total data enhancement based on the spectrum subtraction of the maximum value and the total data enhancement based on the spectrum subtraction of the minimum value;

[0032] The training set data enhancement based on the spectral subtraction of the maximum value includes the training set data enhancement based on the spectral subtraction of the maximum value and the training set data enhancement based on the spectral subtraction of the minimum value;

[0033] The multiple enhanced sample sets include:

[0034] (1) An enhanced sample set obtained by a full data enhancement method based on maximum value spectral subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0035] (2) an enhanced sample set obtained by a full data enhancement method based on maximum value spectral subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero;

[0036] (3) an enhanced sample set obtained by a full data enhancement method based on minimum spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0037] (4) an enhanced sample set obtained by a full data enhancement method based on minimum value spectral difference subtraction, excluding enhanced samples whose spectral difference and content difference are all zero;

[0038] (5) an enhanced sample set obtained by a training set data enhancement method based on maximum value spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0039] (6) an enhanced sample set obtained by a training set data enhancement method based on maximum value-based spectral difference subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero;

[0040] (7) an enhanced sample set obtained by a training set data enhancement method based on minimum value spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0041] (8) an enhanced sample set obtained by a training set data enhancement method based on minimum value spectral difference subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero;

[0042] The overall data enhancement method based on maximum value spectral subtraction is specifically as follows:

[0043] A1: Sort the original samples from large to small according to their content, and take the content of each original sample as the maximum content in turn. At the same time, the content of the original sample with the smallest content is not taken as the maximum content. The jth maximum content is The spectrum of the original sample with the jth maximum content is And N is the number of original samples;

[0044] A2: Based on the jth maximum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced sample corresponding to the original sample, and obtain an enhanced sample set of all data enhancement methods based on spectral difference subtraction of the maximum value, which excludes enhanced samples whose spectral difference and content difference are all 0, and an enhanced sample set of all data enhancement methods based on spectral difference subtraction of the maximum value, which does not exclude the above enhanced samples;

[0045] The j-th maximum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample is:

[0046]

[0047] In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y iWith X i are the content and spectrum of the i-th original sample respectively and is the content of the original sample with the jth maximum content; is the spectrum of the original sample with the jth maximum content;

[0048] When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, since the content is not greater than The number of original samples is N+1-j, so N+1-j enhanced samples are generated;

[0049] For the original data containing N original samples, the number of enhanced samples generated is

[0050] When N is an even number,

[0051] When N is an odd number,

[0052] When the enhanced samples whose spectral difference and content difference are all 0 are eliminated, since the content is not greater than The number of original samples is Nj, so Nj enhanced samples are generated;

[0053] For the original data containing N original samples, the number of enhanced samples generated is

[0054] When N is an even number,

[0055] When N is an odd number,

[0056] The whole data enhancement method based on minimum value spectral subtraction is specifically as follows:

[0057] B1: Sort the original samples from small to large content, and take the content of each original sample as the minimum content in turn. At the same time, the content of the original sample with the largest content is not taken as the minimum content. The jth minimum content is The spectrum of the original sample with the jth minimum content is And N is the number of original samples;

[0058] B2: Based on the jth minimum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced sample corresponding to the original sample, and obtain an enhanced sample set that eliminates the enhanced samples whose spectral difference and content difference are all 0 by all data enhancement methods based on the minimum spectral difference subtraction and an enhanced sample set that does not eliminate the above enhanced samples by all data enhancement methods based on the minimum spectral difference subtraction;

[0059] The jth minimum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample is:

[0060]

[0061] In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample respectively and is the content of the original sample with the jth minimum content; is the spectrum of the original sample with the jth minimum content;

[0062] The calculation formula for the number of enhanced samples that can be generated by the full data enhancement based on the spectral subtraction of the minimum value is the same as that of the full data enhancement based on the spectral subtraction of the maximum value.

[0063] The training set data enhancement method based on maximum value spectral subtraction is specifically as follows:

[0064] C1: The original sample set is randomly divided into a training set and a test set, and the content range of the training set is not less than the content range of the test set;

[0065] C2: Sort the original samples in the training set obtained by dividing the original sample set by content from large to small, and take the content of each original sample in the training set as the maximum content in turn. At the same time, the content of the original sample with the smallest content in the training set is not taken as the maximum content. train The maximum content is With the jth train The spectrum of the original sample with the maximum content is And N train is the original number of samples in the training set;

[0066] C3: Based on the jth train Maximum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced samples corresponding to the original samples in the training set, and obtain the enhanced sample set of the training set data enhancement method based on the maximum value spectral difference subtraction, which eliminates the enhanced samples whose spectral difference and content difference are all 0, and the enhanced sample set of the training set data enhancement method based on the maximum value spectral difference subtraction, which does not eliminate the above enhanced samples;

[0067] Based on the j train Maximum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample in the training set is:

[0068]

[0069] Where m is the serial number of the original sample in the training set, and m = 1, 2, 3, ...., N train ; is the content difference in the enhanced sample corresponding to the mth original sample in the training set; is the spectral difference in the enhanced sample corresponding to the mth original sample in the training set; Y m With X m are the content and spectrum of the mth original sample in the training set, respectively. is the jth train The content of the original sample with the largest content; is the jth train The spectrum of the original sample with the maximum content;

[0070] When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, since the content is not greater than The original sample size is N train +1-j train , so N is generated train +1-j train Enhanced samples;

[0071] For N train The training set data of original samples, the number of enhanced samples finally generated is

[0072] When N train When is an even number,

[0073] When N train When is an odd number,

[0074] When the enhanced samples whose spectral difference and content difference are all 0 are eliminated, since the content is not greater than The original sample size is N train -j train , so N is generated train -j train Enhanced samples;

[0075] For N trainThe training set data of original samples, the number of enhanced samples finally generated is

[0076] When N train When is an even number,

[0077] When N train When is an odd number,

[0078] The training set data enhancement method based on minimum value spectral subtraction is specifically as follows:

[0079] D1: The original sample set is randomly divided into a training set and a test set, and the content range of the training set is not less than the content range of the test set;

[0080] D2: Sort the original samples in the training set obtained by dividing the original sample set in ascending order according to the content, and take the content of each original sample in the training set as the minimum content in turn. Meanwhile, the content of the original sample with the largest content in the training set is not taken as the minimum content. train The minimum content is With the jth train The spectrum of the original sample with the minimum content is And N train is the original number of samples in the training set;

[0081] D3: Based on the jth train Minimum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced samples corresponding to the original samples in the training set, and obtain the enhanced sample set of the training set data enhancement method based on the minimum spectral difference subtraction, which eliminates the enhanced samples whose spectral difference and content difference are all 0, and the enhanced sample set of the training set data enhancement method based on the minimum spectral difference subtraction, which does not eliminate the above enhanced samples;

[0082] The jth train Minimum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample in the training set is:

[0083]

[0084] In the formula, is the content difference in the enhanced sample corresponding to the mth original sample in the training set; is the spectral difference in the enhanced sample corresponding to the mth original sample in the training set; Y m With X mare the content and spectrum of the mth original sample in the training set, respectively. is the jth train The content of the original sample with the minimum content; is the jth train The spectrum of the original sample with the smallest content;

[0085] The calculation formula for the number of enhanced samples that can be generated by the training set data enhancement based on the minimum value spectral subtraction is the same as that of the training set data enhancement based on the maximum value spectral subtraction.

[0086] Step 4: Use the random vector function chain neural network and the extreme learning machine as soil quantitative detection models respectively, and use the four enhanced sample sets obtained in step 2 to train and test the random vector function chain neural network and the extreme learning machine respectively, and calculate the evaluation index during the training and testing process to obtain several soil quantitative detection models;

[0087] The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ;

[0088] The random vector function chain neural network RVFL and the extreme learning machine ELM, the input weights and biases between the input layer and the hidden layer are randomly taken and kept fixed, the output weights from the hidden layer to the output layer are solved by pseudo-inverse, and the random vector function chain neural network has a direct link between the input layer and the output layer;

[0089] Step 4.1: Set the operating environment and parameters of the soil quantitative detection model;

[0090] Step 4.2: Use principal component analysis to reduce the spectral differences of the enhanced samples in the four enhanced sample sets obtained in step 2, and obtain four enhanced sample sets after dimensionality reduction;

[0091] Step 4.3: Use the random partitioning method to divide the four enhanced sample sets after dimensionality reduction into training sets and test sets respectively;

[0092] Step 4.4: Perform maximum and minimum normalization on the spectral differences and content differences in the training set and the test set obtained by randomly dividing the four enhanced sample sets after dimensionality reduction;

[0093] Step 4.5: Use the four normalized training sets to train and test the random vector function chain neural network and extreme learning machine for a set number of cycles, and calculate the mean of the training set RMSE and the training set R of the soil quantitative detection model after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated.2 The mean of , a total of eight trained soil quantitative detection models were obtained;

[0094] Step 5: Use random vector function chain neural network and extreme learning machine as soil quantitative detection models respectively, and use the four enhanced sample sets obtained by the full data enhancement method of spectral subtraction based on the maximum value in step 3 to train and test the random vector function chain neural network and extreme learning machine respectively, and calculate the evaluation index in the process of training and testing to obtain several soil quantitative detection models;

[0095] The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ;

[0096] Step 5.1: Set the operating environment and parameters of the soil quantitative detection model;

[0097] Step 5.2: Use principal component analysis to reduce the spectral difference of enhanced samples in the enhanced sample sets obtained by all data enhancement methods based on the four maximum value spectral difference subtractions, and obtain the enhanced sample sets obtained by all data enhancement methods based on the four maximum value spectral difference subtractions after dimensionality reduction; the number of principal components of the principal component analysis, i.e., the number of model input variables, is selected through experience;

[0098] Step 5.3: Use the random partitioning method to randomly divide the enhanced sample sets obtained by all data enhancement methods based on the maximum value spectral subtraction after four dimension reduction into training sets and test sets;

[0099] Step 5.4: Perform maximum and minimum normalization on the spectral differences and content differences in the training set and the test set obtained by randomly dividing the enhanced sample set obtained by all data enhancement methods based on the maximum value of the spectral difference after four dimension reductions;

[0100] Step 5.5: The training set obtained by randomly dividing the enhanced sample set obtained by all data enhancement methods based on the four normalized maximum spectral subtraction methods is used to train and test the random vector function chain neural network and the extreme learning machine for a set number of cycles, and the mean RMSE of the training set and the training set R of the soil quantitative detection model are calculated after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2 The mean of , a total of eight trained soil quantitative detection models were obtained;

[0101] Step 6: Use random vector function chain neural network, extreme learning machine and one-dimensional convolutional neural network CNN as soil quantitative detection models respectively, use the enhanced sample set obtained by the training set data enhancement method based on the maximum value spectral subtraction in step 3 to train the soil quantitative detection model, and use the test set of the original samples obtained in step 3 to test the soil quantitative detection model, and calculate the evaluation index of the soil quantitative detection model in the process of training and testing to obtain several soil quantitative detection models;

[0102] The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ;

[0103] Step 6.1: Set the operating environment and parameters of the soil quantitative detection model;

[0104] Step 6.2: For the enhanced sample sets obtained by the four training set data enhancement methods based on the maximum value spectral difference, the spectral difference dimension reduction is performed using principal component analysis to obtain the enhanced sample sets obtained by the four training set data enhancement methods based on the maximum value spectral difference after dimension reduction; the number of principal components of the principal component analysis, i.e., the number of model input variables, is selected by experience;

[0105] Step 6.3: When a one-dimensional convolutional neural network is used as a soil quantitative detection model, the enhanced sample sets obtained by the four dimensionally reduced spectral subtraction-based training set data enhancement methods are divided into new training sets and validation sets using a random partitioning method;

[0106] Step 6.4: When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, the spectral difference and content difference of the enhanced sample set obtained by the four dimensionality reduction-based spectral difference subtraction training set data enhancement methods are respectively normalized to the maximum and minimum; when using one-dimensional convolutional neural network as soil quantitative detection model, the spectral difference and content difference of the new training set and validation set obtained in step 6.3 are respectively normalized by StandardScaler;

[0107] Step 6.5: When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, use the normalized enhanced sample set obtained in step 6.4 to train the random vector function chain neural network and extreme learning machine for a set number of cycles, and use the test set of the original sample obtained in step 3 to test the soil quantitative detection model for a set number of times to obtain the predicted value of the content, and calculate the mean value of the training set RMSE and the training set R of the soil quantitative detection model after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2The mean of eight trained soil quantitative detection models was obtained;

[0108] When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, for each test process, when the maximum value is taken as the benchmark, the soil quantitative detection model using the trained training set data enhancement method based on the maximum value of the spectrum subtraction is used, and the calculation process of the predicted value of the content is specifically as follows:

[0109] E1: Find the test sample with a content greater than the kth one from the training set of the original sample obtained in step 3 The training sample of the jth test The content of qualified training samples is And the spectrum is The content of the kth test sample is Content greater than The number of training samples is N test ;

[0110] E2: Based on the jth test The content is greater than The content of training samples and its corresponding spectrum Calculate the spectral difference and content difference of the kth test sample and get N test Spectral difference and content difference of the kth test sample:

[0111]

[0112] In the formula, Based on the jth test The content is greater than The content difference of the kth test sample of the training sample; Based on the jth test The content is greater than The spectral difference of the kth test sample of the training sample; and are the content and spectrum of the kth test sample respectively; For the jth test The content is greater than The content of training samples; For the jth test The content is greater than The spectra of the training samples;

[0113] E3: To N test The spectral difference and content difference of the kth test sample are subjected to principal component analysis for dimensionality reduction, and then maximum and minimum normalization is performed;

[0114] E4: Combine the dimension reduction and normalization N test The spectral difference and content difference of the k-th test sample are input into the trained soil quantitative detection model model, and the output is N test The predicted value of the content difference of the k-th test sample is obtained, and then the corresponding N test Predicted value of content;

[0115] The calculation method of the content prediction value is:

[0116]

[0117] in, is the j-th test The content is greater than The spectral difference of the kth test sample of the training sample; Based on the jth test The content is greater than The predicted value of the content difference of the kth test sample of the training sample;

[0118] E5: According to N test The final content prediction value Y of the kth test sample is obtained by ensemble averaging. test ;

[0119]

[0120] E6: Repeat E1 to E5 for each test set sample of the original sample in turn until the final content prediction value of all test set samples is obtained;

[0121] Step 6.6: When using a one-dimensional convolutional neural network as the soil quantitative detection model, use the new training set and validation set standardized in step 6.4 to train the soil quantitative detection model, obtain four trained soil quantitative detection models, and calculate the training set RMSE and training set R 2 , validation set RMSE, validation set R 2 ; Use the test set of the original sample obtained from step 3 to test the soil quantitative detection model, obtain the predicted value of the content, and calculate the test set RMSE and test set R of the soil quantitative detection model after the test process 2 ;

[0122] When using a one-dimensional convolutional neural network as a soil quantitative detection model, for the test process, when the maximum value is used as the benchmark, the soil quantitative detection model trained using the training set data enhancement method based on the maximum value of the spectrum subtraction is used. The calculation process of the predicted value of the content is as follows:

[0123] F1: Obtain N in the same way as E1 to E2test The spectral difference and content difference of the kth test sample;

[0124] F2: To N test The spectral difference and content difference of the kth test sample are subjected to principal component analysis for dimensionality reduction, and then standardized by StandardScaler;

[0125] F3: The standardized N test The spectral difference and content difference of the k-th test sample are processed according to steps E4 to E6 to obtain the final content prediction value of the k-th test sample;

[0126] Step 7: Select the optimal soil quantitative detection model from the several soil quantitative detection models obtained in Step 4, Step 5 and Step 6; the optimal soil quantitative detection model is the one with the smallest RMSE of the test set and R 2 maximum;

[0127] Step 8: Obtain a new sample containing spectrum and content according to step 1, and obtain the content prediction value of the new sample based on the selected optimal soil quantitative detection model;

[0128] Compared with the prior art, the beneficial effects of the present invention are:

[0129] 1. The present invention obtains the spectrum difference and content difference through the spectrum and content based on the maximum or minimum value of the content, which not only realizes the spectrum data enhancement, but also realizes the content data enhancement;

[0130] 2. The present invention constructs a quantitative relationship between the number of samples of original data and the number of samples of enhanced data, thereby reducing the number limit of original samples;

[0131] 3. The present invention adopts the random vector function chain neural network and extreme learning machine belonging to machine learning and the one-dimensional convolutional neural network belonging to deep learning as models, which improves the accuracy of the soil quantitative detection model;

[0132] 4. The spectral difference and content difference of the present invention provide a new data form for model construction and also lay a foundation for the use of subsequent spectral preprocessing methods;

[0133] 5. The present invention constructs a model based on spectral difference and content difference, obtains the content prediction value through the relationship between content difference and content, and can also predict new samples containing spectrum and content in the form of integrated mean. BRIEF DESCRIPTION OF THE DRAWINGS

[0134] Figure 1 It is a flow chart of a soil data enhancement method based on spectral difference and content difference in an embodiment of the present invention;

[0135] Figure 2 It is a pseudo code for all data enhancement based on the maximum value spectral subtraction in an embodiment of the present invention;

[0136] Wherein, (a) is the pseudo code of all data enhancement based on the maximum value spectrum subtraction in the embodiment of the present invention;

[0137] (b) is a pseudo code for all data enhancement based on minimum value spectrum subtraction in an embodiment of the present invention;

[0138] Figure 3 Flow chart of all data enhancement based on spectral subtraction of maximum or minimum values ​​in an embodiment of the present invention;

[0139] Figure 4 is a pseudo code for predicting the content of a new sample based on the maximum value in an embodiment of the present invention;

[0140] Figure 5 This is a result diagram of content prediction of a new sample based on the maximum value in an embodiment of the present invention;

[0141] Wherein, (a) is the prediction result of a new sample in the embodiment of the present invention;

[0142] (b) is the prediction result of all new samples in the embodiment of the present invention. DETAILED DESCRIPTION

[0143] In order to better explain the present invention and facilitate understanding, the specific implementation methods of the present invention are further described in detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0144] A soil data enhancement method based on spectral difference and content difference, such as Figure 1 As shown, the following steps are included:

[0145] Step 1: obtaining a number of soil samples through sample collection, measuring the spectrum and content of the soil samples to obtain a number of original samples, and then obtaining an original sample set; the original sample includes the spectrum and content of a soil sample;

[0146] Step 1.1: Collect several soil samples on site, and air-dry, grind, and screen the collected soil samples to obtain processed soil samples;

[0147] This example uses 72 organic soil samples collected from open-pit coal mine dumps and 62 salinity soil samples collected from saline-alkali land, all of which have been air-dried, ground, and screened;

[0148] Step 1.2: Divide each soil sample in the processed soil samples into two parts, use a portable spectrometer to obtain a spectrum from one part of the soil samples, and obtain the content from the other part of the soil samples, so as to obtain the spectrum and content of each processed soil sample, that is, to obtain the original sample, and then obtain the original sample set;

[0149] This embodiment uses the same portable spectrometer to obtain reflectance data, i.e., spectrum, with a wavelength range of 343.1 nm to 2502.1 nm, a total of 973 wavelengths; and obtains organic matter data and salt content data based on national standards;

[0150] The data sets used in this embodiment include an organic matter data set and a salt content data set; the organic matter data set is the spectra and organic matter contents of 72 processed organic soil samples; the salt content data set is the spectra and salt contents of 62 processed salt soil samples.

[0151] Step 2: using the spectral difference subtraction method based on the maximum value to process the original sample to obtain a number of enhanced samples, and then obtain an enhanced sample set; the enhanced samples include spectral differences and content differences;

[0152] Step 2.1: Based on the content, find the original sample with the maximum content and the original sample with the minimum content, and obtain the content Y of the original sample with the maximum content max and Spectrum X max , the content Y of the original sample with the minimum content min and Spectrum X min ;

[0153] Step 2.2: The spectral difference subtraction method based on the maximum value is used to obtain the spectral difference and content difference under two different benchmarks with the maximum value and the minimum value as the benchmark, and the enhanced samples composed of the spectral difference and the content difference are obtained. In addition, under each benchmark, an enhanced sample set is obtained by eliminating the enhanced samples whose spectral difference and content difference are all 0 and an enhanced sample set is obtained by not eliminating the above enhanced samples, and finally the enhanced sample sets under four conditions are obtained;

[0154] The spectrum subtraction method based on the maximum value includes the spectrum subtraction method based on the maximum value and the spectrum subtraction method based on the minimum value;

[0155] The maximum value-based spectral difference subtraction method is a method for obtaining spectral difference and content difference based on the maximum value;

[0156] The minimum value-based spectral difference subtraction method is a method for obtaining spectral difference and content difference based on the minimum value;

[0157] The maximum value is the content Y of the original sample with the maximum content max With Spectrum X max ;

[0158] The minimum value is the content Y of the original sample with the minimum content min With Spectrum X min ;

[0159] The enhanced sample sets in the four cases are:

[0160] (1) An enhanced sample set based on the maximum value spectral difference subtraction method and retaining enhanced samples whose spectral difference and content difference are all zero;

[0161] (2) an enhanced sample set based on the maximum value spectral difference subtraction method and excluding enhanced samples whose spectral difference and content difference are all zero;

[0162] (3) an enhanced sample set based on the minimum value spectral difference subtraction method and retaining enhanced samples whose spectral difference and content difference are all zero;

[0163] (4) an enhanced sample set based on the minimum value spectral difference subtraction method and not retaining enhanced samples whose spectral difference and content difference are all zero;

[0164] Based on the correspondence between spectrum and content, there is also a correspondence between spectrum difference and content difference;

[0165] Since the content difference is non-negative, the spectral difference and content difference between any two original samples cannot be obtained. The original sample with the maximum or minimum content is selected as the benchmark, and the spectral difference and content difference are obtained by subtracting it from other original samples.

[0166] The maximum value-based spectral subtraction method is:

[0167]

[0168] Where i is the serial number of the original sample, and i = 1, 2, 3, ...., N, N is the number of original samples; is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample; Y max is the content of the original sample with the maximum content; X max is the spectrum of the original sample with the maximum content; when the enhanced samples whose spectrum difference and content difference are both 0 are not eliminated, N enhanced samples can be obtained; when the enhanced samples whose spectrum difference and content difference are both 0 are eliminated, N-1 enhanced samples can be obtained;

[0169] The minimum value-based spectral subtraction method is:

[0170]

[0171] In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample; Y min is the content of the original sample with the minimum content; X min is the spectrum of the original sample with the minimum content; when the enhanced samples whose spectrum difference and content difference are both 0 are not eliminated, N enhanced samples are obtained; when the enhanced samples whose spectrum difference and content difference are both 0 are eliminated, N-1 enhanced samples are obtained;

[0172] This embodiment finally obtains enhanced sample sets in four cases, namely, the enhanced sample set when the maximum value is used as the benchmark and the enhanced samples with spectral difference and content difference of all 0 are not eliminated, the enhanced sample set when the maximum value is used as the benchmark and the enhanced samples with spectral difference and content difference of all 0 are eliminated, the enhanced sample set when the minimum value is used as the benchmark and the enhanced samples with spectral difference and content difference of all 0 are not eliminated, and the enhanced sample set when the minimum value is used as the benchmark and the enhanced samples with spectral difference and content difference of all 0 are eliminated.

[0173] Step 3: Perform different forms of data enhancement on the original samples to obtain spectral differences and content differences, and obtain a number of enhanced samples composed of spectral differences and content differences after different forms of data enhancement. In addition, under each data enhancement method, an enhanced sample set is obtained by removing enhanced samples whose spectral differences and content differences are all 0 and an enhanced sample set is obtained by not removing the above enhanced samples, thereby obtaining multiple enhanced sample sets obtained by using different forms of data enhancement;

[0174] The data enhancement method includes all data enhancement based on the spectral difference of the maximum value and training set data enhancement based on the spectral difference of the maximum value;

[0175] The total data enhancement based on the spectrum subtraction of the maximum value includes the total data enhancement based on the spectrum subtraction of the maximum value and the total data enhancement based on the spectrum subtraction of the minimum value;

[0176] The training set data enhancement based on the spectral subtraction of the maximum value includes the training set data enhancement based on the spectral subtraction of the maximum value and the training set data enhancement based on the spectral subtraction of the minimum value;

[0177] The multiple enhanced sample sets include:

[0178] (1) An enhanced sample set obtained by a full data enhancement method based on maximum value spectral subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0179] (2) an enhanced sample set obtained by a full data enhancement method based on maximum value spectral subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero;

[0180] (3) an enhanced sample set obtained by a full data enhancement method based on minimum spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0181] (4) an enhanced sample set obtained by a full data enhancement method based on minimum value spectral difference subtraction, excluding enhanced samples whose spectral difference and content difference are all zero;

[0182] (5) an enhanced sample set obtained by a training set data enhancement method based on maximum value spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0183] (6) an enhanced sample set obtained by a training set data enhancement method based on maximum value-based spectral difference subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero;

[0184] (7) an enhanced sample set obtained by a training set data enhancement method based on minimum value spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero;

[0185] (8) an enhanced sample set obtained by a training set data enhancement method based on minimum value spectral difference subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero;

[0186] The overall data enhancement method based on maximum value spectral subtraction is specifically as follows:

[0187] A1: Sort the original samples from large to small according to their content, and take the content of each original sample as the maximum content in turn. At the same time, the content of the original sample with the smallest content is not taken as the maximum content. The jth maximum content is The spectrum of the original sample with the jth maximum content is And N is the number of original samples;

[0188] A2: Based on the jth maximum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced sample corresponding to the original sample, and obtain an enhanced sample set of all data enhancement methods based on spectral difference subtraction of the maximum value, which excludes enhanced samples whose spectral difference and content difference are all 0, and an enhanced sample set of all data enhancement methods based on spectral difference subtraction of the maximum value, which does not exclude the above enhanced samples;

[0189] The j-th maximum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample is:

[0190]

[0191] In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample respectively and is the content of the original sample with the jth maximum content; is the spectrum of the original sample with the jth maximum content;

[0192] When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, since the content is not greater than The number of original samples is N+1-j, so N+1-j enhanced samples are generated;

[0193] For the original data containing N original samples, the number of enhanced samples generated is

[0194] When N is an even number,

[0195] When N is an odd number,

[0196] When the enhanced samples whose spectral difference and content difference are all 0 are eliminated, since the content is not greater than The number of original samples is Nj, so Nj enhanced samples are generated;

[0197] For the original data containing N original samples, the number of enhanced samples generated is

[0198] When N is an even number,

[0199] When N is an odd number,

[0200] The whole data enhancement method based on minimum value spectral subtraction is specifically as follows:

[0201] B1: Sort the original samples from small to large content, and take the content of each original sample as the minimum content in turn. At the same time, the content of the original sample with the largest content is not taken as the minimum content. The jth minimum content is The spectrum of the original sample with the jth minimum content is And N is the number of original samples;

[0202] B2: Based on the jth minimum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced sample corresponding to the original sample, and obtain an enhanced sample set that eliminates the enhanced samples whose spectral difference and content difference are all 0 by all data enhancement methods based on the minimum spectral difference subtraction and an enhanced sample set that does not eliminate the above enhanced samples by all data enhancement methods based on the minimum spectral difference subtraction;

[0203] The jth minimum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample is:

[0204]

[0205] In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample respectively and is the content of the original sample with the jth minimum content; is the spectrum of the original sample with the jth minimum content;

[0206] The calculation formula for the number of enhanced samples that can be generated by the full data enhancement based on the spectral subtraction of the minimum value is the same as that of the full data enhancement based on the spectral subtraction of the maximum value.

[0207] The flowchart and pseudo code of the full data enhancement based on the maximum spectral subtraction are as follows: Figure 2 and Figure 3 The enhanced sample statistics of the organic matter dataset and the salt content dataset are shown in Table 1.

[0208] Table 1 Statistics of enhanced samples for all data augmentation

[0209]

[0210] Different from performing data augmentation on all the data of the original sample, data augmentation is only performed on the training set of the original sample; all the data of the original sample is divided into a training set and a test set by a data partitioning method;

[0211] The training set data enhancement method based on maximum value spectral subtraction is specifically as follows:

[0212] C1: The original sample set is randomly divided into a training set and a test set, and the content range of the training set is not less than the content range of the test set;

[0213] C2: Sort the original samples in the training set obtained by dividing the original sample set by content from large to small, and take the content of each original sample in the training set as the maximum content in turn. At the same time, the content of the original sample with the smallest content in the training set is not taken as the maximum content. train The maximum content is With the jth train The spectrum of the original sample with the maximum content is And N train is the original number of samples in the training set;

[0214] C3: Based on the jth train Maximum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced samples corresponding to the original samples in the training set, and obtain the enhanced sample set of the training set data enhancement method based on the maximum value spectral difference subtraction, which eliminates the enhanced samples whose spectral difference and content difference are all 0, and the enhanced sample set of the training set data enhancement method based on the maximum value spectral difference subtraction, which does not eliminate the above enhanced samples;

[0215] Based on the j train Maximum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample in the training set is:

[0216]

[0217] Where m is the serial number of the original sample in the training set, and m = 1, 2, 3, ...., N train ; is the content difference in the enhanced sample corresponding to the mth original sample in the training set; is the spectral difference in the enhanced sample corresponding to the mth original sample in the training set; Y m With X m are the content and spectrum of the mth original sample in the training set, respectively. is the jth train The content of the original sample with the largest content; is the jth train The spectrum of the original sample with the maximum content;

[0218] When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, since the content is not greater than The original sample size is N train +1-j train , so N is generated train +1-j train Enhanced samples;

[0219] For N train The training set data of original samples, the number of enhanced samples finally generated is

[0220] When N train When is an even number,

[0221] When N train When is an odd number,

[0222] When the enhanced samples whose spectral difference and content difference are all 0 are eliminated, since the content is not greater than The original sample size is N train -j train , so N is generated train -j train Enhanced samples;

[0223] For N train The training set data of original samples, the number of enhanced samples finally generated is

[0224] When N train When is an even number,

[0225] When N train When is an odd number,

[0226] The training set data enhancement method based on minimum value spectral subtraction is specifically as follows:

[0227] D1: The original sample set is randomly divided into a training set and a test set, and the content range of the training set is not less than the content range of the test set;

[0228] D2: Sort the original samples in the training set obtained by dividing the original sample set in ascending order according to the content, and take the content of each original sample in the training set as the minimum content in turn. Meanwhile, the content of the original sample with the largest content in the training set is not taken as the minimum content. train The minimum content is With the jth train The spectrum of the original sample with the minimum content is And N train is the original number of samples in the training set;

[0229] D3: Based on the jthtrain Minimum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced samples corresponding to the original samples in the training set, and obtain the enhanced sample set of the training set data enhancement method based on the minimum spectral difference subtraction, which eliminates the enhanced samples whose spectral difference and content difference are all 0, and the enhanced sample set of the training set data enhancement method based on the minimum spectral difference subtraction, which does not eliminate the above enhanced samples;

[0230] The jth train Minimum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample in the training set is:

[0231]

[0232] In the formula, is the content difference in the enhanced sample corresponding to the mth original sample in the training set; is the spectral difference in the enhanced sample corresponding to the mth original sample in the training set; Y m With X m are the content and spectrum of the mth original sample in the training set, respectively. is the jth train The content of the original sample with the minimum content; is the jth train The spectrum of the original sample with the smallest content;

[0233] The calculation formula for the number of enhanced samples that can be generated by the training set data enhancement based on the minimum value spectral subtraction is the same as that of the training set data enhancement based on the maximum value spectral subtraction.

[0234] When the training set data augmentation based on maximum value spectral subtraction includes enhanced samples whose spectral difference and content difference are all 0, taking this as an example, the number of enhanced samples used for model training of the organic matter dataset is 1484, while that of the salt content dataset is 1080.

[0235] Step 4: Use the random vector function chain neural network and the extreme learning machine as soil quantitative detection models respectively, and use the four enhanced sample sets obtained in step 2 to train and test the random vector function chain neural network and the extreme learning machine respectively, and calculate the evaluation index during the training and testing process to obtain several soil quantitative detection models;

[0236] The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ;

[0237] The random vector function chain neural network RVFL extreme learning machine ELM, the input weight and bias between the input layer and the hidden layer are randomly valued and kept fixed, the output weight from the hidden layer to the output layer is solved by pseudo-inverse, and the random vector function chain neural network has a direct link between the input layer and the output layer;

[0238] Step 4.1: Set the operating environment and parameters of the soil quantitative detection model;

[0239] For the operating environment, the software of this embodiment uses Python 3.10.9 and PyCharm 2021.2.2, the operating system uses Windows 10, and the running processor uses Intel(R) Core(TM) i5-8400 CPU@2.80GHz 2.81GHz

[0240] The parameters of the random vector function chain neural network and extreme learning machine are set as follows: the input weights are uniformly distributed in the interval [-1, 1], the deviations are uniformly distributed in the interval [0, 1], the activation function is sigmoid, the model adds ridge regression to prevent overfitting, and the number of hidden layer nodes is selected based on experience;

[0241] Step 4.2: Use principal component analysis to reduce the spectral differences of the enhanced samples in the four enhanced sample sets obtained in step 2, respectively, to obtain four enhanced sample sets after dimensionality reduction; the number of principal components of the principal component analysis is the number of model input variables, which is selected based on experience;

[0242] Step 4.3: Use the random partitioning method to divide the four enhanced sample sets after dimensionality reduction into training sets and test sets respectively;

[0243] In this embodiment, the ratio of the training set to the test set is 3:1;

[0244] Step 4.4: Perform maximum and minimum normalization on the spectral differences and content differences in the training set and the test set obtained by randomly dividing the four enhanced sample sets after dimensionality reduction;

[0245] In this embodiment, after maximum and minimum normalization, the spectral difference and the content difference are both within the range [-1,1];

[0246] Step 4.5: Use the four normalized training sets to train and test the random vector function chain neural network and extreme learning machine for a set number of cycles, and calculate the mean of the training set RMSE and the training set R of the soil quantitative detection model after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2The mean of , a total of eight trained soil quantitative detection models were obtained;

[0247] The training process and testing process of the soil quantitative detection model are repeated several times. Each training process is accompanied by a corresponding testing process. During each training process, the training set RMSE and training set R of the soil quantitative detection model are calculated. 2 Each training process is accompanied by a corresponding test process. During each test process, the test set RMSE and test set R of the soil quantitative detection model are calculated. 2 ;

[0248] Evaluation indicators can be selected from root mean square error RMSE, determination coefficient R 2 , relative analysis error RPD, the ratio of the interquartile range to the predicted standard deviation RPIQ, Huber loss and Lin's consistency correlation coefficient LCCC; due to the limited capacity of the table, RMSE and R 2 As a representative indicator;

[0249] The training process and the testing process of the soil quantitative detection model in this embodiment are repeated 10 times simultaneously;

[0250] The results of the soil quantitative detection model based on the maximum value spectral subtraction are shown in Table 2;

[0251] The accuracy achieved by the model based on spectral difference and content difference (including enhanced samples with spectral difference and content difference all equal to 0) is similar to that of the model based on spectrum and content, but it provides a new data perspective for model construction;

[0252] Except for the model of the salt content dataset based on the maximum value, the prediction ability of the models that do not contain the enhanced samples with both spectral difference and content difference of 0 has decreased, indicating that the samples are retentive;

[0253] Table 2 Evaluation of soil quantitative detection model based on spectral subtraction of maximum values

[0254]

[0255]

[0256] * indicates that the enhanced samples whose spectral difference and content difference are all 0 are not included; the original data refers to the model based on spectrum and content;

[0257] Step 5: Use random vector function chain neural network and extreme learning machine as soil quantitative detection models respectively, and use the four enhanced sample sets obtained by the full data enhancement method of spectral subtraction based on the maximum value in step 3 to train and test the random vector function chain neural network and extreme learning machine respectively, and calculate the evaluation index in the process of training and testing to obtain several soil quantitative detection models;

[0258] The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ;

[0259] Step 5.1: Set the operating environment and parameters of the soil quantitative detection model;

[0260] For the operating environment, the software of this embodiment uses Python 3.10.9 and PyCharm 2021.2.2, the operating system uses Windows 10, and the running processor uses Intel(R) Core(TM) i5-8400 CPU@2.80GHz 2.81GHz

[0261] The parameters of the random vector function chain neural network and extreme learning machine are set as follows: the input weights are uniformly distributed in the interval [-1, 1], the deviations are uniformly distributed in the interval [0, 1], the activation function is sigmoid, the model adds ridge regression to prevent overfitting, and the number of hidden layer nodes is selected based on experience;

[0262] Step 5.2: Use principal component analysis to reduce the spectral difference of enhanced samples in the enhanced sample sets obtained by all data enhancement methods based on the four maximum value spectral difference subtractions, and obtain the enhanced sample sets obtained by all data enhancement methods based on the four maximum value spectral difference subtractions after dimensionality reduction; the number of principal components of the principal component analysis, i.e., the number of model input variables, is selected through experience;

[0263] Step 5.3: Use the random partitioning method to randomly divide the enhanced sample sets obtained by all data enhancement methods based on the maximum value spectral subtraction after four dimension reduction into training sets and test sets;

[0264] In this embodiment, the ratio of the training set to the test set is 3:1;

[0265] Step 5.4: Perform maximum and minimum normalization on the spectral differences and content differences in the training set and the test set obtained by randomly dividing the enhanced sample set obtained by all data enhancement methods based on the maximum value of the spectral difference after four dimension reductions;

[0266] In this embodiment, after maximum and minimum normalization, the spectral difference and the content difference are both within the range [-1,1];

[0267] Step 5.5: The training set obtained by randomly dividing the enhanced sample set obtained by all data enhancement methods based on the four normalized maximum spectral subtraction methods is used to train and test the random vector function chain neural network and the extreme learning machine for a set number of cycles, and the mean RMSE of the training set and the training set R of the soil quantitative detection model are calculated after several training processes. 2After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2 The mean of , a total of eight trained soil quantitative detection models were obtained;

[0268] The training process and testing process of the soil quantitative detection model are repeated several times. Each training process is accompanied by a corresponding testing process. During each training process, the training set RMSE and training set R of the soil quantitative detection model are calculated. 2 During each test, the test set RMSE and test set R of the soil quantitative detection model were calculated. 2 ;

[0269] The training process and the testing process of the soil quantitative detection model in this embodiment are repeated 10 times simultaneously;

[0270] The model results based on all data enhancement are shown in Table 3;

[0271] The accuracy achieved by ELM is far inferior to that of RVFL due to the existence of direct links. In general, the accuracy of RVFL is worse than that of ELM.

[0272] For the RVFL-based model, the model including the enhanced sample with all spectral differences and content differences of 0 achieved better accuracy than the model without it, which again shows that the sample is retentive;

[0273] The benchmark uses the maximum or minimum value, which has little impact on the performance of the model;

[0274] Since all data augmentation models based on RVFL achieve superior performance, it makes little sense to use convolutional neural networks;

[0275] The soil quantitative detection model based on full data enhancement of content difference and spectral difference provides a new idea for spectral data enhancement;

[0276] Table 3 Evaluation of soil quantitative detection model based on full data enhancement of spectral difference and content difference

[0277]

[0278] * indicates that the enhanced samples whose spectral difference and content difference are all 0 are not included;

[0279] Step 6: Use random vector function chain neural network, extreme learning machine and one-dimensional convolutional neural network CNN as soil quantitative detection models respectively, use the enhanced sample set obtained by the training set data enhancement method based on the maximum value spectral subtraction in step 3 to train the soil quantitative detection model, and use the test set of the original samples obtained in step 3 to test the soil quantitative detection model, and calculate the evaluation index of the soil quantitative detection model in the process of training and testing to obtain several soil quantitative detection models;

[0280] The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ;

[0281] Step 6.1: Set the operating environment and parameters of the soil quantitative detection model;

[0282] For the operating environment, the software of this embodiment uses Python 3.10.9 and PyCharm 2021.2.2, the operating system uses Windows 10, and the running processor uses Intel(R) Core(TM) i5-8400 CPU@2.80GHz 2.81GHz

[0283] The parameters of the random vector function chain neural network and extreme learning machine are set as follows: the input weights are uniformly distributed in the interval [-1, 1], the deviations are uniformly distributed in the interval [0, 1], the activation function is sigmoid, the model adds ridge regression to prevent overfitting, and the number of hidden layer nodes is selected based on experience;

[0284] The one-dimensional convolutional neural network CNN is built using Keras with Tensorflow; in this embodiment, the Adam optimizer is used, the loss function uses the mean square error, the number of iterations is 200, each batch of data is 10 samples, and the structure and hyperparameters are shown in Table 4;

[0285] Table 4 Structure and hyperparameters of one-dimensional convolutional neural network

[0286]

[0287] Step 6.2: For the enhanced sample sets obtained by the four training set data enhancement methods based on the maximum value spectral difference, the spectral difference dimension reduction is performed using principal component analysis to obtain the enhanced sample sets obtained by the four training set data enhancement methods based on the maximum value spectral difference after dimension reduction; the number of principal components of the principal component analysis, i.e., the number of model input variables, is selected by experience;

[0288] Step 6.3: When a one-dimensional convolutional neural network is used as a soil quantitative detection model, the enhanced sample sets obtained by the four dimensionally reduced spectral subtraction-based training set data enhancement methods are divided into new training sets and validation sets using a random partitioning method;

[0289] The new training set is used for training the soil quantitative detection model, the validation set is used for adjusting the hyperparameters of the soil quantitative detection model, and the test set of the original samples obtained from step 3 is used for testing and evaluation of the soil quantitative detection model;

[0290] In this embodiment, the ratio of the new training set to the validation set is 3:1;

[0291] Step 6.4: When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, the spectral difference and content difference of the enhanced sample set obtained by the four dimensionality reduction-based spectral difference subtraction training set data enhancement methods are respectively normalized to the maximum and minimum; when using one-dimensional convolutional neural network as soil quantitative detection model, the spectral difference and content difference of the new training set and validation set obtained in step 6.3 are respectively normalized by StandardScaler;

[0292] In this embodiment, after maximum and minimum normalization, the spectral difference and the content difference are both within the range [-1,1];

[0293] Step 6.5: When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, use the normalized enhanced sample set obtained in step 6.4 to train the random vector function chain neural network and extreme learning machine for a set number of cycles, and use the test set of the original sample obtained in step 3 to test the soil quantitative detection model for a set number of times to obtain the predicted value of the content, and calculate the mean value of the training set RMSE and the training set R of the soil quantitative detection model after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2 The mean of eight trained soil quantitative detection models was obtained;

[0294] The training process and testing process of the soil quantitative detection model are repeated several times. Each training process is accompanied by a corresponding testing process. During each training process, the training set RMSE and training set R of the soil quantitative detection model are calculated. 2 Each training process is accompanied by a corresponding test process. During each test process, the test set RMSE and test set R of the soil quantitative detection model are calculated. 2 ;

[0295] The training process and the testing process of the soil quantitative detection model in this embodiment are repeated 10 times simultaneously;

[0296] When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, for each test process, when the maximum value is taken as the benchmark, the soil quantitative detection model using the trained training set data enhancement method based on the maximum value of the spectrum subtraction is used, and the calculation process of the predicted value of the content is specifically as follows:

[0297] E1: Find the test sample with a content greater than the kth one from the training set of the original sample obtained in step 3 The training sample of the jth test The content of qualified training samples is And the spectrum is The content of the kth test sample is Content greater than The number of training samples is N test ;

[0298] E2: Based on the jth test The content is greater than The content of training samples and its corresponding spectrum Calculate the spectral difference and content difference of the kth test sample and get N test Spectral difference and content difference of the kth test sample:

[0299]

[0300] In the formula, Based on the jth test The content is greater than The content difference of the kth test sample of the training sample; Based on the jth test The content is greater than The spectral difference of the kth test sample of the training sample; and are the content and spectrum of the kth test sample respectively; For the jth test The content is greater than The content of training samples; For the jth test The content is greater than The spectra of the training samples;

[0301] E3: To N test The spectral difference and content difference of the kth test sample are subjected to principal component analysis for dimensionality reduction, and then maximum and minimum normalization is performed;

[0302] E4: Combine the dimension reduction and normalization N testThe spectral difference and content difference of the k-th test sample are input into the trained soil quantitative detection model model, and the output is N test The predicted value of the content difference of the k-th test sample is obtained, and then the corresponding N test Predicted value of content;

[0303] The calculation method of the content prediction value is:

[0304]

[0305] in, is the j-th test The content is greater than The spectral difference of the kth test sample of the training sample; Based on the jth test The content is greater than The predicted value of the content difference of the kth test sample of the training sample;

[0306] E5: According to N test The final content prediction value Y of the kth test sample is obtained by ensemble averaging. test ;

[0307]

[0308] E6: Repeat E1 to E5 for each test set sample of the original sample in turn until the final content prediction value of all test set samples is obtained;

[0309] Step 6.6: When using a one-dimensional convolutional neural network as the soil quantitative detection model, use the new training set and validation set standardized in step 6.4 to train the soil quantitative detection model, obtain four trained soil quantitative detection models, and calculate the training set RMSE and training set R 2 , validation set RMSE, validation set R 2 ; Use the test set of the original sample obtained from step 3 to test the soil quantitative detection model, obtain the predicted value of the content, and calculate the test set RMSE and test set R of the soil quantitative detection model after the test process 2 ;

[0310] In this example, only the training set RMSE and training set R 2 , and does not list the validation set RMSE and validation set R 2 ;

[0311] When using a one-dimensional convolutional neural network as a soil quantitative detection model, for the test process, when the maximum value is used as the benchmark, the soil quantitative detection model trained using the training set data enhancement method based on the maximum value of the spectrum subtraction is used. The calculation process of the predicted value of the content is as follows:

[0312] F1: Obtain N in the same way as E1 to E2 test The spectral difference and content difference of the kth test sample;

[0313] F2: To N test The spectral difference and content difference of the kth test sample are subjected to principal component analysis for dimensionality reduction, and then standardized by StandardScaler;

[0314] F3: The standardized N test The spectral difference and content difference of the k-th test sample are processed according to steps E4 to E6 to obtain the final content prediction value of the k-th test sample;

[0315] The case meets the following conditions: taking the maximum value as the benchmark, using the soil quantitative detection model that has been trained using the training set data enhancement method based on the maximum value of the spectral difference and including the enhanced samples whose spectral difference and content difference are all 0; the results are shown in Table 5;

[0316] The soil quantitative detection model outputs the predicted value of content difference. The training set is based on content difference evaluation, and the test set is based on content evaluation.

[0317] The performance of the training data augmentation model is not as good as the full data augmentation model;

[0318] The enhanced samples with both spectral difference and content difference of 0 are retainable and the accuracy of the model using the maximum or minimum value is the same;

[0319] The training set data augmentation model based on one-dimensional convolutional neural network can effectively improve the model accuracy, which verifies that one-dimensional convolutional neural network is suitable for big data;

[0320] Table 5 Evaluation of soil quantitative detection model based on training data enhancement based on spectral difference and content difference

[0321]

[0322] The original data refers to the spectral and content-based models; except for CNN, the other models use the average of 10 iterations.

[0323] Step 7: Select the optimal soil quantitative detection model from the several soil quantitative detection models obtained in Step 4, Step 5 and Step 6; the optimal soil quantitative detection model is the one with the smallest RMSE of the test set and R 2 maximum;

[0324] It can be seen from Tables 2, 3 and 5 that the optimal models of the organic matter data set and the salt content data set all meet the following conditions: all data enhancement based on the minimum value of spectral difference subtraction, including enhanced samples with spectral difference and content difference of all 0, and the model adopts random vector function chain neural network;

[0325] The optimal model inputs the spectral difference of the enhanced sample and outputs the predicted value of the content difference of the enhanced sample; the accuracy of the optimal model of the organic matter dataset is 0.80 for the test set RMSE and R 2 The accuracy of the optimal model for the salt content dataset is 0.99; the accuracy of the optimal model for the test set is 0.16 for the test set RMSE and 0.16 for the test set R 2 is 0.99;

[0326] Step 8: Obtain a new sample containing spectrum and content according to step 1, and obtain the content prediction value of the new sample based on the selected optimal soil quantitative detection model;

[0327] All models of the present invention can realize this function; the case model, i.e., the content prediction of new samples based on the maximum value, uses all data enhancement based on the spectral difference subtraction based on the maximum value, including enhanced samples whose spectral difference and content difference are all 0, and the model adopts a random vector function chain neural network, which is applied to the organic matter data set; the evaluation index is based on content calculation;

[0328] The new samples are samples whose contents are within the content range of the original samples, and the new samples are selected as all the remaining original samples in the organic matter data set excluding the ones with the maximum content;

[0329] The calculation process of the predicted value of the content of the new sample is the same as E1 to E6 of step 6.5, where the new sample is equivalent to the test sample in step 6.5; the prediction of the new sample, i.e. the test, is only performed once;

[0330] The pseudo code is Figure 4 As shown;

[0331] Since the random value of the random vector function chain neural network leads to unstable model results, we select a result such as Figure 5 As shown, it includes the prediction results of a new sample and the prediction results of all new samples;

[0332] For new samples, the prediction accuracy is good, that is, RMSE is 0.35, R 2 is 0.99.

[0333] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope defined by the claims of the present invention.

Claims

1. A soil data enhancement method based on spectral difference and content difference, characterized in that: The steps include: Step 1: obtain a number of soil samples through sample collection, measure the spectrum and content of the soil samples to obtain a number of original samples, and then obtain an original sample set; each original sample includes the spectrum and content of a soil sample; Step 2: Processing the original sample using the maximum value-based spectral difference subtraction method to obtain a number of enhanced samples, and then obtaining an enhanced sample set; the enhanced samples include spectral differences and content differences; Step 2.1: Based on the content, find the original sample with the maximum content and the original sample with the minimum content, and obtain the content Y of the original sample with the maximum content max and Spectrum X max , the content Y of the original sample with the minimum content min and Spectrum X min ; Step 2.2: The spectral difference subtraction method based on the maximum value is used to obtain the spectral difference and content difference under two different benchmarks with the maximum value and the minimum value as the benchmark, and the enhanced samples composed of the spectral difference and the content difference are obtained. In addition, under each benchmark, an enhanced sample set is obtained by eliminating the enhanced samples whose spectral difference and content difference are all 0 and an enhanced sample set is obtained by not eliminating the above enhanced samples, and finally the enhanced sample sets under four conditions are obtained; The spectrum subtraction method based on the maximum value includes the spectrum subtraction method based on the maximum value and the spectrum subtraction method based on the minimum value; The maximum value-based spectral difference subtraction method is a method for obtaining spectral difference and content difference based on the maximum value; The minimum value-based spectral difference subtraction method is a method for obtaining spectral difference and content difference based on the minimum value; The maximum value is the content Y of the original sample with the maximum content max With Spectrum X max ; The minimum value is the content Y of the original sample with the minimum content min With Spectrum X min ; The enhanced sample sets in the four cases are: (1) An enhanced sample set based on the maximum value spectral difference subtraction method and retaining enhanced samples whose spectral difference and content difference are all zero; (2) an enhanced sample set based on the maximum value spectral difference subtraction method and excluding enhanced samples whose spectral difference and content difference are all zero; (3) an enhanced sample set based on the minimum value spectral difference subtraction method and retaining enhanced samples whose spectral difference and content difference are all zero; (4) an enhanced sample set based on the minimum value spectral difference subtraction method and not retaining enhanced samples whose spectral difference and content difference are all zero; The maximum value-based spectral subtraction method is: Where i is the serial number of the original sample, and i = 1, 2, 3, ...., N, N is the number of original samples; is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample; Y max is the content of the original sample with the maximum content; X max is the spectrum of the original sample with the maximum content; When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, N enhanced samples are obtained; when the enhanced samples whose spectral difference and content difference are all 0 are eliminated, N-1 enhanced samples are obtained; The minimum value-based spectral subtraction method is: In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample; Y min is the content of the original sample with the minimum content; X min is the spectrum of the original sample with the minimum content; When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, N enhanced samples are obtained; when the enhanced samples whose spectral difference and content difference are all 0 are eliminated, N-1 enhanced samples are obtained; Step 3: Perform different forms of data enhancement on the original samples to obtain spectral differences and content differences, and obtain a number of enhanced samples composed of spectral differences and content differences after different forms of data enhancement. In addition, under each data enhancement method, an enhanced sample set is obtained by removing enhanced samples whose spectral differences and content differences are all 0 and an enhanced sample set is obtained by not removing the above enhanced samples, thereby obtaining multiple enhanced sample sets obtained by using different forms of data enhancement; The data enhancement method includes all data enhancement based on the spectral difference of the maximum value and training set data enhancement based on the spectral difference of the maximum value; The total data enhancement based on the spectrum subtraction of the maximum value includes the total data enhancement based on the spectrum subtraction of the maximum value and the total data enhancement based on the spectrum subtraction of the minimum value; The training set data enhancement based on the spectral subtraction of the maximum value includes the training set data enhancement based on the spectral subtraction of the maximum value and the training set data enhancement based on the spectral subtraction of the minimum value; The multiple enhanced sample sets include: (1) An enhanced sample set obtained by a full data enhancement method based on maximum value spectral subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero; (2) an enhanced sample set obtained by a full data enhancement method based on maximum value spectral subtraction, which eliminates enhanced samples whose spectral difference and content difference are all zero; (3) an enhanced sample set obtained by a full data enhancement method based on minimum spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero; (4) an enhanced sample set obtained by a full data enhancement method based on minimum value spectral difference subtraction, excluding enhanced samples whose spectral difference and content difference are all zero; (5) an enhanced sample set obtained by a training set data enhancement method based on maximum value spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero; (6) an enhanced sample set obtained by using a training set data enhancement method based on maximum value-based spectral difference subtraction to eliminate enhanced samples whose spectral difference and content difference are all zero; (7) an enhanced sample set obtained by a training set data enhancement method based on minimum value spectral difference subtraction without eliminating enhanced samples whose spectral difference and content difference are all zero; (8) an enhanced sample set obtained by using a training set data enhancement method based on minimum value spectral difference subtraction to eliminate enhanced samples whose spectral difference and content difference are all zero; The overall data enhancement method based on maximum value spectral subtraction is specifically as follows: A1: Sort the original samples from large to small according to their content, and take the content of each original sample as the maximum content in turn. At the same time, the content of the original sample with the smallest content is not taken as the maximum content. The jth maximum content is The spectrum of the original sample with the jth maximum content is j=1,2,3,...,N-1 and N is the number of original samples; A2: Based on the jth maximum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced sample corresponding to the original sample, and obtain an enhanced sample set of all data enhancement methods based on spectral difference subtraction of the maximum value, which excludes enhanced samples whose spectral difference and content difference are all 0, and an enhanced sample set of all data enhancement methods based on spectral difference subtraction of the maximum value, which does not exclude the above enhanced samples; The j-th maximum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample is: In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample respectively and is the content of the original sample with the jth maximum content; is the spectrum of the original sample with the jth maximum content; When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, since the content is not greater than The number of original samples is N+1-j, so N+1-j enhanced samples are generated; For the original data containing N original samples, the number of enhanced samples generated is When N is an even number, When N is an odd number, When the enhanced samples whose spectral difference and content difference are all 0 are eliminated, since the content is not greater than The number of original samples is Nj, so Nj enhanced samples are generated; For the original data containing N original samples, the number of enhanced samples generated is When N is an even number, When N is an odd number, The whole data enhancement method based on minimum value spectral subtraction is specifically as follows: B1: Sort the original samples from small to large content, and take the content of each original sample as the minimum content in turn. At the same time, the content of the original sample with the largest content is not taken as the minimum content. The jth minimum content is The spectrum of the original sample with the jth minimum content is j=1,2,3,...,N-1 and N is the number of original samples; B2: Based on the jth minimum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced sample corresponding to the original sample, and obtain an enhanced sample set that eliminates the enhanced samples whose spectral difference and content difference are all 0 by all data enhancement methods based on the minimum spectral difference subtraction and an enhanced sample set that does not eliminate the above enhanced samples by all data enhancement methods based on the minimum spectral difference subtraction; The jth minimum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample is: In the formula, is the content difference in the enhanced sample corresponding to the i-th original sample; is the spectral difference in the enhanced sample corresponding to the i-th original sample; Y i With X i are the content and spectrum of the i-th original sample respectively and is the content of the original sample with the jth minimum content; is the spectrum of the original sample with the jth minimum content; The calculation formula for the number of enhanced samples that can be generated by the total data enhancement based on the minimum value spectral subtraction is the same as that of the total data enhancement based on the maximum value spectral subtraction; The training set data enhancement method based on maximum value spectral subtraction is specifically as follows: C1: The original sample set is randomly divided into a training set and a test set, and the content range of the training set is not less than the content range of the test set; C2: Sort the original samples in the training set obtained by dividing the original sample set by content from large to small, and take the content of each original sample in the training set as the maximum content in turn. At the same time, the content of the original sample with the smallest content in the training set is not taken as the maximum content. train The maximum content is With the jth train The spectrum of the original sample with the maximum content is j train =1,2,3,...,N train -1 and N train is the original number of samples in the training set; C3: Based on the jth train Maximum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced samples corresponding to the original samples in the training set, and obtain the enhanced sample set of the training set data enhancement method based on the maximum value spectral difference subtraction, which eliminates the enhanced samples whose spectral difference and content difference are all 0, and the enhanced sample set of the training set data enhancement method based on the maximum value spectral difference subtraction, which does not eliminate the above enhanced samples; Based on the j train Maximum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample in the training set is: Where m is the serial number of the original sample in the training set, and m = 1, 2, 3, ...., N train ; is the content difference in the enhanced sample corresponding to the mth original sample in the training set; is the spectral difference in the enhanced sample corresponding to the mth original sample in the training set; Y m With X m are the content and spectrum of the mth original sample in the training set, respectively. is the jth train The content of the original sample with the largest content; is the jth train The spectrum of the original sample with the maximum content; When the enhanced samples whose spectral difference and content difference are all 0 are not eliminated, since the content is not greater than The original sample size is N train +1-j train , so N is generated train +1-j train Enhanced samples; For N train The training set data of original samples, the number of enhanced samples finally generated is When N train When is an even number, When N train When is an odd number, When the enhanced samples whose spectral difference and content difference are all 0 are eliminated, since the content is not greater than The original sample size is N train -j train , so N is generated train -j train Enhanced samples; For N train The training set data of original samples, the number of enhanced samples finally generated is When N train When is an even number, When N train When is an odd number, The training set data enhancement method based on minimum value spectral subtraction is specifically as follows: D1: The original sample set is randomly divided into a training set and a test set, and the content range of the training set is not less than the content range of the test set; D2: Sort the original samples in the training set obtained by dividing the original sample set in ascending order according to the content, and take the content of each original sample in the training set as the minimum content in turn. Meanwhile, the content of the original sample with the largest content in the training set is not taken as the minimum content. train The minimum content is With the jth train The spectrum of the original sample with the minimum content is j train =1,2,3,...,N train -1 and N train is the original number of samples in the training set; D3: Based on the jth train Minimum content and its corresponding spectrum Calculate the spectral difference and content difference in the enhanced samples corresponding to the original samples in the training set, and obtain the enhanced sample set of the training set data enhancement method based on the minimum spectral difference subtraction, which eliminates the enhanced samples whose spectral difference and content difference are all 0, and the enhanced sample set of the training set data enhancement method based on the minimum spectral difference subtraction, which does not eliminate the above enhanced samples; The jth train Minimum content and its corresponding spectrum The method for calculating the spectral difference and content difference in the enhanced sample corresponding to the original sample in the training set is: In the formula, is the content difference in the enhanced sample corresponding to the mth original sample in the training set; is the spectral difference in the enhanced sample corresponding to the mth original sample in the training set; Y m With X m are the content and spectrum of the mth original sample in the training set, respectively. is the jth train The content of the original sample with the minimum content; is the jth train The spectrum of the original sample with the smallest content; The calculation formula for the number of enhanced samples that can be generated by the training set data enhancement based on the minimum spectral difference is the same as that of the training set data enhancement based on the maximum spectral difference; Step 4: Use the random vector function chain neural network and the extreme learning machine as soil quantitative detection models respectively, and use the four enhanced sample sets obtained in step 2 to train and test the random vector function chain neural network and the extreme learning machine respectively, and calculate the evaluation index during the training and testing process to obtain several soil quantitative detection models; The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ; The random vector function chain neural network RVFL and the extreme learning machine ELM, the input weights and biases between the input layer and the hidden layer are randomly taken and kept fixed, the output weights from the hidden layer to the output layer are solved by pseudo-inverse, and the random vector function chain neural network has a direct link between the input layer and the output layer; Step 5: Use random vector function chain neural network and extreme learning machine as soil quantitative detection models respectively, and use the four enhanced sample sets obtained by the full data enhancement method of spectral subtraction based on the maximum value in step 3 to train and test the random vector function chain neural network and extreme learning machine respectively, and calculate the evaluation index in the process of training and testing to obtain several soil quantitative detection models; The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ; Step 6: Use random vector function chain neural network, extreme learning machine and one-dimensional convolutional neural network CNN as soil quantitative detection models respectively, use the enhanced sample set obtained by the training set data enhancement method based on the maximum value spectral subtraction in step 3 to train the soil quantitative detection model, and use the test set of the original samples obtained in step 3 to test the soil quantitative detection model, and calculate the evaluation index of the soil quantitative detection model in the process of training and testing to obtain several soil quantitative detection models; The evaluation indicators include root mean square error RMSE, determination coefficient R 2 ; Step 7: Select the optimal soil quantitative detection model from the several soil quantitative detection models obtained in Step 4, Step 5 and Step 6; the optimal soil quantitative detection model is the one with the smallest RMSE of the test set and R 2 maximum; Step 8: Obtain a new sample containing spectrum and content according to step 1, and obtain the content prediction value of the new sample based on the selected optimal soil quantitative detection model.

2. The soil data enhancement method based on spectral difference and content difference according to claim 1 is characterized in that: The step 1 specifically includes: Step 1.1: Collect several soil samples on site, and air-dry, grind, and screen the collected soil samples to obtain processed soil samples; Step 1.2: Divide each soil sample in the processed soil samples into two parts, use a portable spectrometer to obtain the spectrum from one part of the soil samples, and obtain the content from the other part of the soil samples, so as to obtain the spectrum and content of each processed soil sample, that is, to obtain the original sample, and then obtain the original sample set.

3. The soil data enhancement method based on spectral difference and content difference according to claim 1 is characterized in that: The step 4 specifically includes: Step 4.1: Set the operating environment and parameters of the soil quantitative detection model; Step 4.2: Use principal component analysis to reduce the spectral differences of the enhanced samples in the four enhanced sample sets obtained in step 2, and obtain four enhanced sample sets after dimensionality reduction; Step 4.3: Use the random partitioning method to divide the four enhanced sample sets after dimensionality reduction into training sets and test sets respectively; Step 4.4: Perform maximum and minimum normalization on the spectral differences and content differences in the training set and the test set obtained by randomly dividing the four enhanced sample sets after dimensionality reduction; Step 4.5: Use the four normalized training sets to train and test the random vector function chain neural network and extreme learning machine for a set number of cycles, and calculate the mean of the training set RMSE and the training set R of the soil quantitative detection model after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2 The mean of , a total of eight trained soil quantitative detection models were obtained.

4. The soil data enhancement method based on spectral difference and content difference according to claim 1 is characterized in that: The step 5 specifically includes: Step 5.1: Set the operating environment and parameters of the soil quantitative detection model; Step 5.2: Use principal component analysis to reduce the spectral difference of enhanced samples in the enhanced sample sets obtained by all data enhancement methods based on the four maximum value spectral difference subtractions, and obtain the enhanced sample sets obtained by all data enhancement methods based on the four maximum value spectral difference subtractions after dimensionality reduction; the number of principal components of the principal component analysis, i.e., the number of model input variables, is selected through experience; Step 5.3: Use the random partitioning method to randomly divide the enhanced sample sets obtained by all data enhancement methods based on the maximum value spectral subtraction after four dimension reduction into training sets and test sets; Step 5.4: Perform maximum and minimum normalization on the spectral differences and content differences in the training set and the test set obtained by randomly dividing the enhanced sample set obtained by all data enhancement methods based on the maximum value of the spectral difference after four dimension reductions; Step 5.5: The training set obtained by randomly dividing the enhanced sample set obtained by all data enhancement methods based on the four normalized maximum spectral subtraction methods is used to train and test the random vector function chain neural network and the extreme learning machine for a set number of cycles, and the mean RMSE of the training set and the training set R of the soil quantitative detection model are calculated after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2 The mean of , a total of eight trained soil quantitative detection models were obtained.

5. The soil data enhancement method based on spectral difference and content difference according to claim 1 is characterized in that: The step 6 specifically includes: Step 6.1: Set the operating environment and parameters of the soil quantitative detection model; Step 6.2: For the enhanced sample sets obtained by the four training set data enhancement methods based on the maximum value spectral difference, the spectral difference dimension reduction is performed using principal component analysis to obtain the enhanced sample sets obtained by the four training set data enhancement methods based on the maximum value spectral difference after dimension reduction; the number of principal components of the principal component analysis, i.e., the number of model input variables, is selected by experience; Step 6.3: When a one-dimensional convolutional neural network is used as a soil quantitative detection model, the enhanced sample sets obtained by the four dimensionally reduced spectral subtraction-based training set data enhancement methods are divided into new training sets and validation sets using a random partitioning method; Step 6.4: When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, the spectral difference and content difference of the enhanced sample set obtained by the four dimensionality reduction-based spectral difference subtraction training set data enhancement methods are respectively normalized to the maximum and minimum; when using one-dimensional convolutional neural network as soil quantitative detection model, the spectral difference and content difference of the new training set and validation set obtained in step 6.3 are respectively normalized by StandardScaler; Step 6.5: When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, use the normalized enhanced sample set obtained in step 6.4 to train the random vector function chain neural network and extreme learning machine for a set number of cycles, and use the test set of the original sample obtained in step 3 to test the soil quantitative detection model for a set number of times to obtain the predicted value of the content, and calculate the mean value of the training set RMSE and the training set R of the soil quantitative detection model after several training processes. 2 After several test processes, the mean of the test set RMSE and the test set R of the soil quantitative detection model are calculated. 2 The mean of eight trained soil quantitative detection models was obtained; When using random vector function chain neural network and extreme learning machine as soil quantitative detection model, for each test process, when the maximum value is taken as the benchmark, the soil quantitative detection model using the trained training set data enhancement method based on the maximum value of the spectrum subtraction is used, and the calculation process of the predicted value of the content is specifically as follows: E1: Find the test sample with a content greater than the kth one from the training set of the original sample obtained in step 3 The training sample of the jth test The content of qualified training samples is Y jtest And the spectrum is X jtest , where the content of the kth test sample is j test =1,2,...,N test , the content is greater than The number of training samples is N test ; E2: Based on the jth test The content is greater than The content of the training sample Y jtest and its corresponding spectrum X jtest , calculate the spectral difference and content difference of the kth test sample, and get N test Spectral difference and content difference of the kth test sample: In the formula, Based on the jth test The content is greater than The content difference of the kth test sample of the training sample; Based on the jth test The content is greater than The spectral difference of the kth test sample of the training sample; and are the content and spectrum of the kth test sample; Y jtest For the jth test The content is greater than The content of training samples; X jtest For the jth test The content is greater than The spectra of the training samples; E3: To N test The spectral difference and content difference of the kth test sample are subjected to principal component analysis for dimensionality reduction, and then maximum and minimum normalization is performed; E4: Combine the dimension reduction and normalization N test The spectral difference and content difference of the k-th test sample are input into the trained soil quantitative detection model model, and the output is N test The predicted value of the content difference of the k-th test sample is obtained, and then the corresponding N test Predicted value of content; The calculation method of the content prediction value is: in, is the j-th test The content is greater than The spectral difference of the kth test sample of the training sample; Based on the jth test The content is greater than The predicted value of the content difference of the kth test sample of the training sample; E5: According to N test The final content prediction value Y of the kth test sample is obtained by ensemble averaging. test ; E6: Repeat E1 to E5 for each test set sample of the original sample in turn until the final content prediction value of all test set samples is obtained; Step 6.6: When using a one-dimensional convolutional neural network as the soil quantitative detection model, use the new training set and validation set standardized in step 6.4 to train the soil quantitative detection model, obtain four trained soil quantitative detection models, and calculate the training set RMSE and training set R 2 , validation set RMSE, validation set R 2 ; Use the test set of the original sample obtained from step 3 to test the soil quantitative detection model, obtain the predicted value of the content, and calculate the test set RMSE and test set R of the soil quantitative detection model after the test process 2 ; When using a one-dimensional convolutional neural network as a soil quantitative detection model, for the test process, when the maximum value is used as the benchmark, the soil quantitative detection model trained using the training set data enhancement method based on the maximum value of the spectrum subtraction is used. The calculation process of the predicted value of the content is as follows: F1: Obtain N in the same way as E1 to E2 test The spectral difference and content difference of the kth test sample; F2: To N test The spectral difference and content difference of the kth test sample are subjected to principal component analysis for dimensionality reduction, and then standardized by StandardScaler; F3: The standardized N test The spectral difference and content difference of the k-th test sample are processed according to steps E4 to E6 to obtain the final content prediction value of the k-th test sample.

Images