Sample transfer learning method and program product for field scale soil organic matter spectrum inversion modeling

Abstract

The invention discloses a field scale soil organic matter spectrum inversion modeling-oriented sample transfer learning method and program product, and the method comprises the steps: carrying out the first-order differential processing of spectrum data in a source domain data set and a target domain data set, environment variables with Spearman correlation coefficients and variance expansion factors higher than threshold values in the source domain data set are eliminated, and target domain samples are divided into a training set and a test set; calculating environment similarity and spectrum similarity between the source domain and the training set samples, further obtaining comprehensive similarity between each source domain sample and each training set sample, and obtaining an intersection of the source domain samples with the comprehensive similarity larger than a threshold value and the source domain samples with the organic matter content within a preset range to serve as a source domain migration sample set; and obtaining a union set of the source domain migration sample set and the training set as an amplification training data set of soil organic matter spectrum inversion learning. The test precision obtained by performing field scale organic matter spectrum inversion based on the amplification training data set formed by the method is higher.

Description

TECHNICAL FIELD

[0001] The present application relates to geographic information technology, and in particular to a sample transfer learning method for soil organic matter spectral inversion modeling at field plot scale and a program product. BACKGROUND

[0002] At the field plot scale, it is difficult to take a large number of samples, and the organic matter content of the same depth soil layer in the same plot has little difference, which leads to low accuracy of high-spectral remote sensing inversion of soil organic matter at the field plot scale. In view of this problem, the sample transfer learning method is used to select samples from a large soil spectrum library to expand the sample set for spectral modeling at the field plot scale, so as to improve the spectral inversion accuracy of farmland soil properties.

[0003] At present, there are two main ideas for soil property spectral inversion modeling based on sample transfer learning. The first idea is to use only source domain samples for training and modeling, and there are two specific implementation methods: one is to select source domain samples based on spectral similarity, which requires that the soil spectrum library samples have good representativeness and coverage; the other is to select source domain samples based on modeling effect, which is less efficient and needs to try and verify multiple times to find the best source domain sample combination. The second idea is to combine source domain and target domain samples for training and modeling, and there are also two specific implementation methods: one is to merge all or part of the representative samples of the target domain and the soil spectrum library samples of the source domain to form a training set to establish a prediction model, and the performance of the model constructed in this way is affected by the difference in sample quantity between the source domain and the target domain; the other way is to select samples similar to the target domain samples from the source domain spectrum library based on spectral similarity and geographical distance, and merge them with the target domain samples to form a training set to establish a prediction model, which is not suitable for application scenarios where the geographical distribution of source domain samples and target domain samples is quite different, and the inversion accuracy is not high. SUMMARY

[0004] In view of the problems existing in the prior art, the purpose of the present application is to provide a sample transfer learning method for soil organic matter spectral inversion modeling at field plot scale with higher inversion accuracy and a program product.

[0005] In order to achieve the above-mentioned purpose of the application, the present application provides the following technical scheme:

[0006] A sample transfer learning method for soil organic matter spectral inversion modeling at field plot scale, comprising:

[0007] (1) reading the organic matter content, spectral reflectance and environmental variable data of each soil sample in the source domain soil spectrum library and each soil sample in the target domain study area, and constructing a data set ;

[0008] (2) selecting source domain samples from the data set spectrum data of each sample in the spectrum data set is subjected to first-order differential processing to construct a data set 1 1 is divided into a source domain sample data set and a target domain sample data set ;

[0009] is divided into a spectrum data set , an environmental variable data set and an organic matter content data set ;

[0010] (4) Based on the environmental variable data set , the Spearman correlation coefficient and the variance inflation factor between the environmental variables are calculated, and the environmental variables with the Spearman correlation coefficient and the variance inflation factor higher than a threshold value are removed to form a screened environmental variable data set 1

[0011] (5) The target domain sample data set is divided into a training set and a test set , and the training set is divided into a spectrum data set , an environmental variable data set and an organic matter content data set ;

[0012] (6) The similarity between any element in the environmental variable data set 1 of the source domain and any element in the environmental variable data set of the target domain is calculated to form an environmental similarity data set ;

[0013] (7) The similarity between any element in the spectrum data set of the source domain and any element in the spectrum data set of the target domain is calculated to form a spectrum similarity data set ;

[0014] (8) The comprehensive similarity between each source domain sample and each target domain training sample is calculated according to and , and the source domain samples with the comprehensive similarity greater than a threshold value are screened to form a first transfer sample set;

[0015] (9) For , the source domain samples with the organic matter within a preset range are screened to form a second transfer sample set, and the preset range is a range around the mean value;

[0016] (10) calculating the intersection of the first migration sample set and the second migration sample set as a source domain migration sample set;

[0017] (11) obtaining the union of the source domain migration sample set and the target domain training set as an augmented training data set for soil organic matter spectral inversion learning.

[0018] Further, the determination method of the comprehensive similarity and the comprehensive similarity threshold value in step (8) is:

[0019] According to and , a plurality of comprehensive similarity calculation methods are used, so that for each source domain sample, a plurality of comprehensive similarities are calculated;

[0020] A plurality of comprehensive similarity threshold values are set, and the screening of the source domain sample with the comprehensive similarity greater than the comprehensive similarity threshold value is performed, so as to obtain a plurality of first migration sample sets;

[0021] For , the source domain sample with the organic matter within the preset range is screened to form a second migration sample set;

[0022] The intersection of the first migration sample set and the second migration sample set is calculated as a third migration sample set;

[0023] Obtaining the union of the source domain migration sample set and the training set as an augmented training data set for soil organic matter spectral inversion learning ;

[0024] Using the random forest algorithm to perform organic matter inversion model training on , verifying the constructed organic matter inversion model using the test set , calculating the test precision index, selecting the similarity calculation method and the comprehensive similarity threshold value with the highest test precision index as the final values of the comprehensive similarity calculation method and the comprehensive similarity threshold value.

[0025] Further, step (3) specifically includes the following steps:

[0026] Dividing the source domain sample data set into a spectral data set , an environmental variable data set and an organic matter data set , wherein represents the spectral reflectance data of the i-th source domain sample, represents the spectral reflectance of the k1-th spectral band of , represents the environmental variable of the i-th source domain sample, express The value of the k2th environment variable, Indicates the first Organic matter content of each source domain sample The total number of spectral bands, This represents the total number of environment variables. This represents the number of samples in the source domain.

[0027] Furthermore, step (4) specifically includes the following steps:

[0028] (4-1) Based on environmental variable dataset ,calculate Spearman correlation coefficient matrix :

[0029]

[0030] In the formula, This represents the function used to calculate the Spearman correlation coefficient. This represents the Spearman correlation coefficient of the k2th environmental variable. This represents the k2-th environmental variable value of the i-th source domain sample. The number of samples in the source domain. This represents the total number of environment variables.

[0031] (4-2) Based on environmental variable dataset ,calculate variance expansion factor matrix ;

[0032]

[0033] in, This represents the function for calculating the difference expansion factor. This represents the variance inflation factor of the k2th environmental variable;

[0034] (4-3) For the environmental variable dataset For each source domain sample, environmental variables were removed, and samples with Spearman correlation coefficients greater than a threshold were excluded. Environmental variables and variance inflation factor greater than the threshold Environment variables, forming a dataset ;

[0035]

[0036] In the formula, This represents the environmental variables of the i-th source domain sample after filtering. This represents the total number of environment variables after filtering.

[0037] Furthermore, step (5) specifically includes the following steps:

[0038] (5-1) Target domain sample dataset The data is divided into training sets according to a predetermined ratio. and test set , This represents the j1-th target domain training sample. The total number of training samples for the target domain. This represents the j2th target domain test sample. The total number of training and testing sessions for the target domain;

[0039] (5-2) The training set Divided into spectral datasets Environment variable dataset And organic matter content dataset In the formula, This shows the spectral reflectance data of the j1th target domain training sample. express Spectral reflectance of the k1th spectral band, This represents the environment variable for the j1-th target domain training sample. express The value of the k2th environment variable, This represents the organic matter content of the j1th target domain training sample. The total number of spectral bands, This represents the total number of environment variables.

[0040] Furthermore, step (6) specifically includes the following steps:

[0041] (6-1) Convert the environmental variable dataset from the source domain 1. Environmental variable dataset of the target domain Each element in the array is normalized so that its value is in the range of 0-1.

[0042] (6-2) Calculation any element in 1 The average Euclidean distance between any elements in the set;

[0043]

[0044] In the formula, express The i-th element in 1 and The j1th element The average Euclidean distance, express The value of the k2th environment variable, representing the k2th environmental variable value of the kth environment, is the total number of environmental variables in step 1;

[0045] (6-3) Based on the average Euclidean distance, the similarity between any element in 1 and any element in is calculated and stored in the environmental similarity dataset

[0046]

[0047] wherein, is the similarity of and is the number of source domain samples, is the total number of target domain training samples.

[0048] Further, step (7) specifically includes the following steps:

[0049] (7-1) The spectral dataset of the source domain is normalized with each element in the spectral dataset of the target domain respectively, so that the value is in the range of 0-1;

[0050] (7-2) The average Euclidean distance between any element in and any element in is calculated;

[0051]

[0052] wherein, represents the average Euclidean distance between the ith element in and the j1th element in , represents the spectral reflectance of the k1th spectral band of , represents the spectral reflectance of the k1th spectral band of is the number of spectral bands;

[0053] (7-3) Based on the average Euclidean distance, the similarity between any element in and any element in is calculated and stored in the spectral similarity dataset

[0054]

[0055] wherein, is the similarity of and , is the number of source domain samples, is the total number of target domain training samples.

[0056] Further, step (8) specifically includes the following steps:

[0057] (8-1) reading an arbitrary element from , reading an element corresponding to from , calculating the comprehensive similarity based on and as the comprehensive similarity of the ith source domain sample and the j1th target domain training sample , and storing it in the comprehensive similarity dataset , , is the number of source domain samples, is the number of target domain training set samples;

[0058] (8-2) screening all source domain samples with a comprehensive similarity greater than a threshold to the target domain training sample, as the first migration sample set .

[0059] Further, step (9) specifically includes the following steps:

[0060] (9-1) calculating the mean and standard deviation of the organic matter content dataset of the target domain training set;

[0061] (9-2) from the source domain organic matter dataset , screening out the source domain samples corresponding to the data with organic matter content in the interval[ , ] as the second migration sample set , is a multiple of the standard deviation.

[0062] A computer program product comprising a computer program which, when executed by a processor, implements the above method.

[0063] Compared with the prior art, the present application has the beneficial effects that: the present application is a kind of field scale soil organic matter spectral inversion modeling sample transfer learning method based on comprehensive spectrum and environmental similarity. This method not only considers the similarity of spectral variables, but also comprehensively considers the similarity of environmental variables, so as to more comprehensively evaluate the representativeness and applicability of source domain samples. Through this method of comprehensively considering spectrum and landscape factors, the source domain soil spectrum library migration samples suitable for the target domain can be more effectively screened out, and the spectral inversion precision of farmland soil properties can be improved by using the samples obtained by the method for inversion. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 is a flowchart of the sample transfer learning method for field scale soil organic matter spectral inversion modeling provided by the embodiments of the present application;

[0065] Figure 2 is a histogram of the organic matter content of the soil samples of a certain dry land field in Jiangxi Province;

[0066] Figure 3 is the visible-near infrared spectral reflectance curve of the soil samples of a certain dry land field in Jiangxi Province. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0068] Embodiment one

[0069] The embodiments of the present application provide a kind of sample transfer learning method for field scale soil organic matter spectral inversion modeling, as shown in Figure 1 , comprising:

[0070] (1) read the organic matter content, spectral reflectance and environmental variable data of each soil sample in source domain soil spectrum library and each soil sample in target domain research area, and construct data set .

[0071] The source domain dataset is as follows: the source domain soil spectrum library is from the soil system survey project of some provinces in China, including 5645 pieces of soil samples from 1395 soil profiles. The spectrum library contains soil visible-near infrared spectrum reflectivity data, soil physicochemical property data and environmental variable data. The visible-near infrared band (350 nm-2500 nm) reflectance spectrum of the soil spectrum library sample is measured by Cary 5000 spectrophotometer, and the spectral sampling interval is 1 nm. Before measurement, the soil sample is dried, ground, sieved (0.25 mm) and dried (45 ℃). The soil organic matter data is obtained from the soil system survey report of each province. The environmental variables include longitude and latitude coordinates, altitude, slope, aspect, plan curvature, profile curvature, multi-resolution valley bottom flatness index (MRVBF), terrain ruggedness index (TRI), terrain surface texture index (TST), terrain surface convexity index (TSC), terrain humidity index (TWI), normalized vegetation index maximum value (NDVImax), annual average precipitation, precipitation in the wettest season, precipitation in the warmest season, precipitation in the coldest season, annual average temperature, average temperature in the coldest season, average temperature in the warmest season, average temperature in the wettest season.

[0072] The target domain dataset is as follows: the target domain research area of the embodiment is a dry land plot in Jiangxi Province. The soil type is red soil, and the parent material is quaternary red clay. In the fall bare soil period, 50 surface soil samples (0-10 cm) were collected by grid sampling design. Organic matter (SOM) was determined by Walkley-Black K2Cr2O7-H2SO4 oxidation method. The soil organic matter content of the research area is between 12.22 and 22.62 g / kg (as shown in Table 1). Figure 2The moisture content was between 5.19 and 19.54 g / kg. Visible-near infrared reflectance spectra were measured in a dark room using an ASD FieldSpec4 spectrometer. After removing the residual roots, gravel and lime particles, the moist soil samples were broken into 5 mm pieces and poured into a sample dish (black square tray, 25 cm long, 2 cm deep), flattened, and slightly compacted with a ruler to make the surface as flat as possible. The sample dish was placed on a black rubber plate with nearly zero reflectivity. A halogen light source with a power of 50 W and a probe with a 8° field of view were used. The probe was perpendicular to the soil sample surface, and the distance from the probe to the soil sample surface was 18 cm. The light source was 30 cm away from the sample surface, and the angle between the light source and the vertical direction was 15°. Considering that the surface of the soil sample was not uniform, which would cause anisotropy of the soil sample spectrum, the sample was moved for multiple tests when measuring each soil sample. Four directions (the sample dish was rotated by 90° each time) and five different points (the center point was the vertical drop point of the probe on the test table, and the soil sample dish was moved to five different positions) were measured for each sample. Five sample lines were taken at each direction and each point, and 100 scans (4 directions x 5 points x 5 scans) were accumulated to obtain the reflectance spectrum data of the soil sample. The spectrometer was optimized and whiteboard corrected before measuring the soil sample spectrum, and then optimized and corrected every 1 hour. The measurement band range was 350-2500 nm, and the high-noise band at both ends was removed. The 400-2450 nm band data were used, as shown in FIG. 2. Figure 3 The environmental variables are the same as the source domain.

[0073] Therefore, in this embodiment, , represents the first sample data, represents the total number of samples. In this embodiment, .

[0074] (2) The spectral data of each sample in the data set is subjected to first-order differential processing to construct a data set 1, and the data set 1 is divided into a source domain sample data set and a target domain sample data set .

[0075] The first-order differential is a Savitzky-Golay (SG) second-order polynomial fitting first-order differential processing with a processing window of 11. The data set is divided into a source domain sample data set and a target domain sample data set , where is the number of source domain samples, The number of target domain samples. In this embodiment, , .

[0076] (3) dividing the source domain sample data set into a spectral data set , an environmental variable data set and an organic matter content data set .

[0077] Specifically, the source domain sample data set is divided into a spectral data set , an environmental variable data set and an organic matter data set , wherein denotes the spectral reflectance data of the ith source domain sample, denotes the spectral reflectance of the k1th spectral band of , denotes the environmental variable of the ith source domain sample, denotes the k2th environmental variable value of , denotes the organic matter content of the ith source domain sample, is the total number of spectral bands, is the total number of environmental variables, is the number of source domain samples. In this embodiment, , , .

[0078] (4) calculating the Spearman correlation coefficients and the variance inflation factors between the environmental variables based on the environmental variable data set , and removing the environmental variables with the Spearman correlation coefficients and the variance inflation factors higher than a threshold value to form a screened environmental variable data set 1.

[0079] This step specifically includes the following steps:

[0080] (4-1) calculating the Spearman correlation coefficient matrix of based on the environmental variable data set :

[0081]

[0082] wherein denotes a function for calculating the Spearman correlation coefficient, denotes the Spearman correlation coefficient of the k2th environmental variable, denotes the k2th environmental variable value of the ith source domain sample, is the number of source domain samples.

[0083] (4-2) Based on environmental variable dataset ,calculate variance expansion factor matrix ;

[0084]

[0085] in, This represents the function for calculating the difference expansion factor. This represents the variance inflation factor of the k2th environmental variable;

[0086] (4-3) For the environmental variable dataset For each source domain sample, environmental variables were removed, and samples with Spearman correlation coefficients greater than a threshold were excluded. Environmental variables and variance inflation factor greater than the threshold Environment variables, forming a dataset ;

[0087]

[0088] In the formula, This represents the environmental variables of the i-th source domain sample after filtering. This represents the total number of environment variables after filtering.

[0089] In this embodiment , Ultimately, 15 environmental variables were selected: latitude and longitude coordinates, altitude, slope, aspect, plane curvature, profile curvature, multi-resolution valley floor flatness index (MRVBF), topographic surface convexity index (TSC), topographic surface texture index (TST), topographic humidity index (TWI), annual maximum normalized vegetation index (NDVImax), annual average precipitation, annual average temperature, and average temperature of the warmest season. .

[0090] (5) Target domain sample dataset Divided into training set and test set and the training set Divided into spectral datasets Environment variable dataset And organic matter content dataset .

[0091] This step specifically includes the following steps:

[0092] (5-1) Target domain sample dataset The data is divided into training sets according to a predetermined ratio. and test set , denotes the j1th target domain training sample, is the total number of target domain training samples, denotes the j2th target domain test sample, is the total number of target domain training and test samples; in the embodiment, the preset ratio is specifically 7:3 after stratified sampling according to the organic matter content, to ensure balanced distribution of samples in each organic matter content interval; in the embodiment , ;

[0093] (5-2) The training set is divided into a spectral data set , an environmental variable data set and an organic matter content data set , wherein, denotes the spectral reflectance data of the j1th target domain training sample, denotes the spectral reflectance of the k1th spectral band of , denotes the environmental variable of the j1th target domain training sample, denotes the k2th environmental variable value of , denotes the organic matter content of the j1th target domain training sample, is the total number of spectral bands, is the total number of environmental variables;

[0094] (5-3) The test set is divided into a spectral data set , an environmental variable data set and an organic matter content data set , wherein, denotes the spectral reflectance data of the j2th target domain test sample, denotes the spectral reflectance of the k1th spectral band of , denotes the environmental variable of the j2th target domain test sample, denotes the k2th environmental variable value of , denotes the organic matter content of the j2th target domain test sample.

[0095] (6) The similarity between any element in the environmental variable data set 1 of the source domain and any element in the environmental variable data set of the target domain is calculated to form an environmental similarity data set .

[0096] This step specifically includes:

[0097] (6-1) Normalize each element in the source domain's environmental variable dataset 1 to the range of 0-1 respectively;

[0098] (6-2) Calculate the average Euclidean distance between any element in 1 and any element in .

[0099]

[0100] wherein, denotes the average Euclidean distance between the i-th element in 1 and the j1-th element in , denotes the k2-th environmental variable value of , denotes the k2-th environmental variable value of , denotes the k2-th environmental variable value of , is the total number of environmental variables in 1.

[0101] (6-3) Based on the average Euclidean distance, calculate the similarity between any element in 1 and any element in , and store it in the environmental similarity dataset

[0102]

[0103] wherein, is the similarity between and , is the number of source domain samples, is the total number of target domain training samples.

[0104] (7) Calculate the similarity between any element in the source domain's spectral dataset and any element in the target domain's spectral dataset , and form the spectral similarity dataset .

[0105] This step specifically includes the following steps:

[0106] (7-1) Normalize each element in the source domain's spectral dataset and the target domain's spectral dataset to the range of 0-1 respectively; ​

[0107] (7-2) Calculation any element and The average Euclidean distance between any elements in the set;

[0108]

[0109] In the formula, express The i-th element and The j1th element The average Euclidean distance, express Spectral reflectance of the k1th spectral band, express Spectral reflectance of the k1th spectral band, This refers to the number of spectral bands.

[0110] (7-3) Calculate based on the mean Euclidean distance. any element and The similarity between any elements in the dataset is stored in the spectral similarity dataset.

[0111]

[0112] In the formula, for and similarity, The number of samples in the source domain. The total number of training samples for the target domain.

[0113] (8) According to and The comprehensive similarity between each source domain sample and each target domain training sample is calculated, and source domain samples with comprehensive similarity greater than a threshold are selected to form the first transfer sample set.

[0114] This step specifically includes the following steps:

[0115] (8-1) From Read any element ,from Reading and Corresponding elements ,based on and Calculate the overall similarity, which is the combined similarity between the i-th source domain sample and the j1-th target domain training sample. Store in the comprehensive similarity dataset , The number of samples in the source domain. The number of target domain training set samples; for example, , or ;

[0116] (8-2) All source domain samples with a comprehensive similarity greater than a threshold value to the target domain training samples are screened out as the first migration sample set . .

[0117] The screening process is specifically: for each target domain training sample, all source domain samples with a comprehensive similarity greater than a threshold value to it are screened out, and then the source domain samples screened out for all target domain training samples are removed from duplicates to obtain the first migration sample set . Specifically, it can be expressed as: , represents the sample index number with a comprehensive similarity greater than a threshold value , represents the comprehensive similarity column between the target domain training sample j1 and all source domain samples, represents the sample index number after removing duplicates, retaining unique values, represents the source domain sample with the sample index number in the brackets.

[0118] (9) For , source domain samples with organic matter within a preset range are screened out to form a second migration sample set.

[0119] The preset range is a range around the mean value.

[0120] Specifically, this step specifically includes:

[0121] (9-1) Calculate the mean value and the standard deviation of the organic matter content data set of the target domain training set;

[0122] (9-2) From the organic matter data set of the source domain, the source domain samples corresponding to the data with organic matter content within the interval , are screened out as the second migration sample set , is a multiple of the standard deviation.

[0123]

[0124] In this embodiment, .

[0125] (10) Calculate the intersection of the first migration sample set and the second migration sample set, and use it as the source domain migration sample set.

[0126] (11) Obtain the source domain transfer sample set and training set The union of these sets is used as the augmented training dataset for soil organic matter spectral inversion learning.

[0127] Furthermore, the method for determining the comprehensive similarity and the comprehensive similarity threshold in step (8) is as follows:

[0128] S1, according to and Several comprehensive similarity calculation methods are employed to calculate several comprehensive similarities for each source domain sample; for example, in this embodiment, a total of [number] comprehensive similarities are set. Six different combinations of methods were tested to generate comprehensive similarity scores, and the results are as follows: As a similarity calculation method ; As a similarity calculation method ;based on As a similarity calculation method ; As a similarity calculation method ; As a similarity calculation method ; As a similarity calculation method ;

[0129] S2. Set several comprehensive similarity thresholds and perform filtering on source domain samples with comprehensive similarity greater than the comprehensive similarity thresholds to obtain several first migration sample sets; in this embodiment, for The comprehensive similarity thresholds are set as follows: =0.75, 0.80, 0.85, and 0.9, several first-transfer sample sets were selected, denoted as . ;right The thresholds are set as follows: =0.75, 0.80, 0.85, and 0.9, several first-transfer sample sets were selected, denoted as . ;right The thresholds are set as follows: =0.65, 0.70, 0.72, 0.75, and 0.77, several first-transfer sample sets were selected, denoted as . ;right The thresholds are set as follows: =0.80, 0.82, 0.85, and 0.87, several first-transfer sample sets were selected, denoted as . ; the comprehensive similarity The threshold values are set to 0.80, 0.82, 0.85 and 0.87 respectively, and a plurality of first migration sample sets are screened, denoted as ; the comprehensive similarity The threshold values are set to 0.80, 0.82, 0.85 and 0.87 respectively, and a plurality of first migration sample sets are screened, denoted as ; ;

[0130] S3, for , the source domain sample with the organic matter in the preset range is screened to form a second migration sample set;

[0131] S4, the intersection of the first migration sample set and the second migration sample set is calculated as a third migration sample set; in the embodiment, and The intersection of and constitutes the migration sample set ; and The intersection of and constitutes the migration sample set ; and The intersection of and constitutes the migration sample set ; and The intersection of and constitutes the migration sample set ; and The intersection of and constitutes the migration sample set ; and The intersection of and constitutes the migration sample set ;

[0132] S5, the union set of the third migration sample set and the training set is obtained as the data set for soil organic matter spectrum inversion learning ; in the embodiment, and the target domain training set constitute the augmented training set ; and the target domain training set constitute the augmented training set ; and the target domain training set constitute the augmented training set ; and the target domain training set constitute the augmented training set ; and the target domain training set constitute the augmented training set​ ; and target domain training set constitute augmented training set ;

[0133] S6, using a random forest algorithm to organic matter inversion model training, using the test set verify the constructed organic matter inversion model, calculate the test precision index, select the highest test precision index of the similarity calculation method and the comprehensive similarity threshold value as the final value of the comprehensive similarity calculation method and the comprehensive similarity threshold value.

[0134] S6 specifically includes:

[0135] S6.1, using a random forest algorithm to train the training set model training. Specifically, the training set model training to obtain ; the training set model training to obtain ; the training set model training to obtain ; the training set model training to obtain ; the training set model training to obtain ; the training set model training to obtain ;

[0136] S6.2, using to obtain the prediction result . In this embodiment, the prediction result of ; the prediction result of ; the prediction result of ; the prediction result of ; the prediction result of ; the prediction result of ;

[0137] S6.3, calculate the and the measured value of the prediction result and , to verify the performance of the organic matter inversion model constructed based on the augmented training set .

[0138]

[0139] wherein, is , indicates to what extent the independent variable of the model can explain the change of the dependent variable; is an index measuring the difference between the predicted value of the model and the true value, the smaller the smaller the difference between the predicted value and the true value; is used to measure the average difference between the predicted value and the true value.

[0140] In this embodiment, the The test results of the constructed organic matter inversion model are shown in Table 1. It can be seen that for the transfer learning modeling of field scale organic matter spectrum inversion, considering the environmental similarity to screen the source domain samples can obviously improve the performance of the inversion model. In the principle of “prefer to select a small number of high similarity source domain transfer samples for modeling under the condition of close model performance”, the best model is constructed based on the augmented training set formed by screening the source domain samples under the threshold of 0.87 based on the comprehensive similarity of “0.5*environmental similarity+0.5*spectral similarity”, which screens 26 source domain samples for modeling, and the model validation is 0.68, RMSE is 1.01 g kg-1, and MAE is 0.85 g kg-1.

[0141] Table 1

[0142] Example two

[0143] The embodiment of the application also provides a computer program product, such as an app on a mobile phone, a tablet computer, an installation program on a computer, etc. The product includes computer programs / instructions, which are executed by a processor to implement the method of the first embodiment. The computer executable program code for executing the operation of the application can be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as “C” language or similar programming languages. The program code can be completely executed on a user computer, partially executed on a user computer, executed as a separate software package, partially executed on a user computer and partially executed on a remote computer, or completely executed on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0144] It should be noted that the above-mentioned embodiments and description of the application are only intended to illustrate the principles of the application, the main features and the advantages of the application, and various changes and modifications can be made to the application without departing from the spirit and scope of the application, and these changes and modifications all fall within the scope of the application.

Claims

1. A sample transfer learning method for field-scale soil organic matter spectral inversion modeling, characterized in that, Comprise: (1) Read the soil sample in the source domain soil spectrum library and the soil sample in the target domain study area, and construct a dataset of organic matter content, spectral reflectance and environmental variable data ; (2) first-order differential processing is performed on the spectral data of each sample in the data set , a data set A1 is constructed, and the data set A1 is divided into a source domain sample data set and a target domain sample data set ; (3) dividing the spectral data set into a spectral data set , an environmental variable data set and an organic matter content data set ; (4) Based on the environmental variable dataset Spearman correlation coefficients and variance inflation factors between the environmental variables are calculated, and environmental variables with Spearman correlation coefficients and variance inflation factors higher than a threshold value are removed to form a screened environmental variable dataset 1; (5) Target domain sample dataset Divided into training set and test set and the training set Divided into spectral datasets Environment variable dataset And organic matter content dataset ; (6) Compute the environmental variable dataset of the source domain 1 Any element of the target domain's environmental variable dataset Similarity between any elements in the source domain, forming an environmental similarity dataset ; (7) computing a spectral dataset for the source domain arbitrary elements to the spectral dataset of the target domain similarity between any of the elements, forming a spectral similarity dataset ; (8) according to and The comprehensive similarity between each source domain sample and each target domain training sample is calculated, and the source domain samples with a comprehensive similarity greater than a threshold value are screened to form a first migration sample set; (9) For , the source domain samples within a preset range are screened to form a second migration sample set, and the preset range is a range around the mean value. (10) calculating the intersection of the first migration sample set and the second migration sample set as the source domain migration sample set; (11) obtaining a union of the source domain migration sample set and the target domain training set as an augmented training dataset for soil organic matter spectral inversion learning. of the source domain migration sample set and the target domain training set as an augmented training dataset for soil organic matter spectral inversion learning.

2. The field-plot-scale soil organic matter spectroscopy inversion modeling sample transfer learning method according to claim 1, characterized in that, The calculation method of the comprehensive similarity in step (8) and the determination method of the threshold value are as follows: According to and , several comprehensive similarity calculation methods are adopted, so that for each source domain sample, several comprehensive similarities are calculated. A plurality of comprehensive similarity threshold values are set, and the screening of the source domain samples with the comprehensive similarity greater than the threshold value is performed, thereby obtaining a plurality of first migration sample sets; For , screening source domain samples with organic matter within a preset range forms a second migration sample set; The intersection of the first migration sample set and the second migration sample set is calculated as the source domain migration sample set; Acquiring a union of a source domain migration sample set and a training set as augmented training dataset for soil organic matter spectral inversion learning ; Using the random forest algorithm to Perform organic matter inversion model training, and use the test set Verify the constructed organic matter inversion model, calculate the test precision index, select the similarity calculation method with the highest test precision index and the comprehensive similarity threshold value as the final value of the comprehensive similarity calculation method and the comprehensive similarity threshold value.

3. The field-plot-scale soil organic matter spectroscopy inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (3) specifically comprises the following steps: Source domain sample dataset Divided into spectral datasets Environment variable dataset and organic matter dataset In the formula, This represents the spectral reflectance data of the i-th source domain sample. express Spectral reflectance of the k1th spectral band, This represents the environmental variables of the i-th source domain sample. express The value of the k2th environment variable, Indicates the first Organic matter content of each source domain sample The total number of spectral bands, This represents the total number of environment variables. This represents the number of samples in the source domain.

4. The field-plot-scale soil organic matter spectroscopy inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (4) specifically comprises the following steps: (4-1) Based on the environmental variable dataset , the Spearman correlation coefficient matrix of :​ , wherein denotes a function that computes the Spearman correlation coefficient, denotes the Spearman correlation coefficient of the k2th environmental variable, denotes the k2th environmental variable value of the ith source domain sample, is the number of source domain samples, is the total number of environmental variables; (4-2) Based on the environmental variable dataset , calculate the variance inflation factor matrix ; , wherein, denotes a difference inflation factor computation function, denotes the variance inflation factor of the k2th environment variable; (4-3) For the environmental variable dataset of each source domain sample, remove the environmental variables with Spearman correlation coefficient greater than a threshold value and the environmental variables with variance inflation factor greater than a threshold value , form a dataset ; , In the formula, denotes the environmental variable of the i-th source domain sample after screening, is the total number of environmental variables after screening.

5. The field-plot-scale soil organic matter spectroscopy inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (5) specifically comprises the following steps: (5-1) dividing the target domain sample data set into a training set and a test set in a preset ratio and a test set , denotes the j1th target domain training sample, is the total number of target domain training samples, denotes the j2th target domain test sample, is the total number of target domain training test samples; (5-2) The training set Divided into spectral datasets Environment variable dataset And organic matter content dataset In the formula, The spectral reflectance data of the j1th target domain training sample are shown. express Spectral reflectance of the k1th spectral band, This represents the environment variable for the j1-th target domain training sample. express The value of the k2th environment variable, This represents the organic matter content of the j1th target domain training sample. The total number of spectral bands, This represents the total number of environment variables.

6. The field-plot-scale soil organic matter spectroscopic inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (6) specifically comprises the following steps: (6-1) Normalizing each element of the source domain's environmental variable dataset 1 and the target domain's environmental variable dataset respectively, so that its value is in the range of 0-1; (6-2) Calculation 1 any element of and average Euclidean distance of any element of , wherein, denotes the i-th element in 1 and the j1-th element in the average Euclidean distance, denotes the k2-th environment variable value of denotes the k2-th environment variable value of is the total number of environment variables in 1; (6-3) Based on the average Euclidean distance, calculate 1 any element and Similarity between any element in , , In the formula, is and the similarity of, is the number of source domain samples, is the total number of target domain training samples.

7. The field-plot-scale soil organic matter spectroscopy inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (7) specifically comprises the following steps: (7-1) Normalizing each element of the source domain's spectral dataset and the target domain's spectral dataset respectively, so that its value is in the range of 0-1; (7-2) Calculation the average Euclidean distance of any element in the average Euclidean distance of any element in , wherein, denotes the i-th element in with the j1-th element in the average Euclidean distance, denotes the spectral reflectance of the k1-th spectral band of denotes the spectral reflectance of the k1-th spectral band of is the number of spectral bands; (7-3) Based on the average Euclidean distance, the similarity between any element in and any element in is calculated and stored in the spectral similarity dataset , , In the formula, is and the similarity of is the number of source domain samples, is the total number of target domain training samples.

8. The field-plot-scale soil organic matter spectroscopy inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (8) specifically comprises the following steps: (8-1) Read an arbitrary element from , read the element corresponding to from , calculate the comprehensive similarity based on and as the comprehensive similarity of the ith source domain sample and the j1th target domain training sample , and store it in the comprehensive similarity dataset , is the number of source domain samples, is the number of target domain training set samples;​​ (8-2) screen all source domain samples whose comprehensive similarity with the target domain training sample is greater than a threshold value as the first migration sample set .​ 9. The field -scale soil organic matter spectral inversion modeling sample transfer learning method according to claim 1, characterized in that, Step (9) specifically comprises the following steps: (9-1) Calculate the data set of the organic matter content of the target domain training set the mean value and the standard deviation ; (9-2) From the source domain of organic matter data set In the organic matter content in the interval [0.1%, 1.5%] of the data corresponding to the source domain sample, as the second migration sample set , , Standard deviation of the multiple.​ 10. A computer program product comprising a computer program, characterized in that: The computer program is executed by the processor to realize the method in any one of claims 1-7.

Images