Soil sample screening method based on instance transfer learning

Abstract

The application discloses a soil sample screening method based on instance migration learning, and relates to the technical field of migration learning. The method comprises the following steps: acquiring a source domain soil sample set and a target domain initial soil sample set; screening a plurality of soil samples with Mahalanobis distance greater than a set threshold from the source domain soil sample set and the target domain initial soil sample set to obtain a candidate sample set; training a model through the target domain initial sample set to obtain an initial prediction model, adding the candidate samples one by one to train a temporary prediction model and determining an error; updating the sample set according to the sample with the maximum error reduction and recursively screening to obtain a migration sample subset; taking the migration sample subset with the minimum prediction error as an optimal subset, combining the optimal subset and the target domain initial sample set into a final sample set and training to obtain a final prediction model. The application can improve the effectiveness of training data and improve the precision and generalization ability of the model in the prediction of soil properties in the target region.

Description

Technical Field

[0001] This invention relates to the field of transfer learning technology, and in particular to a method for screening soil samples based on instance transfer learning. Background Technology

[0002] Soil organic carbon (SOC) is an important indicator of soil fertility and carbon cycle, and its accurate prediction is of great significance for sustainable agricultural development and climate change assessment. Visible-near-infrared spectroscopy (vis-NIR) combined with machine learning provides a rapid means for SOC prediction.

[0003] Due to the significant spatial heterogeneity of SOC distribution across different regions, general models based on the Soil Spectral Library (SSL) are difficult to generalize to local areas, especially when the sample size in the target area is small, resulting in low prediction accuracy.

[0004] Existing transfer learning methods often rely on random sampling or simple similarity indicators for sample selection, lacking a refined mechanism, making it difficult to effectively remove noisy samples and weakening the model's cross-domain generalization ability. Summary of the Invention

[0005] Therefore, it is necessary to provide a soil sample screening method based on instance transfer learning to address the aforementioned technical problems.

[0006] This invention provides a soil sample screening method based on instance transfer learning, comprising:

[0007] Obtain the global soil spectral library and the local soil sample set of the target area, and use the global soil spectral library as the source soil sample set and the local soil sample set as the initial soil sample set of the target area.

[0008] From the source domain soil sample set, multiple soil samples whose Mahalanobis distance from the initial target domain soil sample set is greater than a set threshold are selected to obtain a candidate sample set.

[0009] The linear regression model is trained based on the initial soil sample set of the target domain to obtain the initial soil organic carbon prediction model; soil samples from the candidate sample set are added one by one to the initial soil sample set of the target domain. After each addition of a sample, the linear regression model is trained based on the updated initial soil sample set of the target domain to obtain the temporary soil organic carbon prediction model, and the prediction error of each temporary soil organic carbon prediction model is determined.

[0010] Soil samples corresponding to the temporary soil organic carbon prediction model with the largest decrease in prediction error compared to the initial soil organic carbon prediction model are selected as migration samples. These migration samples are added to the initial set of soil samples in the target domain and removed from the candidate sample set. This process is repeated recursively to obtain a subset of migration samples. In each round of recursive selection, the linear regression model is trained based on the subset of migration samples to determine the prediction error corresponding to that subset. After the recursive selection is completed, the subset of migration samples with the smallest prediction error is selected as the optimal subset.

[0011] The optimal subset is merged with the initial set of soil samples in the target domain to form the final set of soil samples in the target domain. The deep learning model or machine learning model is then trained using the final set of soil samples in the target domain to obtain the final prediction model of soil organic carbon in the target area.

[0012] Optionally, multiple soil samples from the source domain soil sample set whose Mahalanobis distance to the initial target domain soil sample set is greater than a set threshold are selected to obtain a candidate sample set, which specifically includes:

[0013] The initial soil sample set for the target domain is represented by the following formula:

[0014] ;

[0015] The mean vector and covariance matrix of the initial set of soil samples in the target domain are determined based on the following formula:

[0016] ;

[0017] ;

[0018] For any global sample in the source domain soil sample set, the Mahalanobis distance of each global sample is determined based on the mean vector and covariance matrix according to the following formula:

[0019] ;

[0020] All global samples in the source domain soil sample set are sorted in ascending order of Mahalanobis distance. Global samples with a Mahalanobis distance greater than a set threshold are aggregated into a candidate sample set based on the following formula:

[0021]

[0022] in, The Mahalanobis distance, Soil samples from the initial soil sample set of the target domain. For global samples in the source domain soil sample set, It is the mean vector. Let covariance matrix be the variance matrix. MThe total number of samples in the candidate sample set. This is the initial set of soil samples for the target domain. For the candidate sample set, ( , ) are candidate samples.

[0023] Optionally, determining the prediction error of each provisional soil organic carbon prediction model specifically includes:

[0024] The initial prediction error of the soil organic carbon initial prediction model is determined based on the following formula;

[0025] ;

[0026] Soil samples from the candidate sample set are added one by one to the initial soil sample set of the target domain. After each addition, a deep learning model or a traditional machine learning model is trained based on the updated target domain soil sample set to obtain a provisional soil organic carbon prediction model. The prediction error of each provisional soil organic carbon prediction model is determined based on the following formula:

[0027] ;

[0028] in, This is the initial prediction error. The prediction error of the provisional soil organic carbon prediction model. n This represents the initial number of target region samples in the initial set of soil samples for the target domain. This refers to the initial target region samples in the initial set of soil samples for the target domain. For the updated target domain soil sample set, soil samples are used.

[0029] Optionally, the reduction in prediction error compared to the initial soil organic carbon prediction model can be determined based on the following formula:

[0030] ;

[0031] in, The percentage decrease in prediction error compared to the initial prediction model for soil organic carbon;

[0032] Soil samples corresponding to the temporary soil organic carbon prediction model with the largest decrease in prediction error compared to the initial soil organic carbon prediction model are selected as migration samples. These samples are added to the initial soil sample set of the target domain and removed from the candidate sample set. The recursive selection continues based on the updated initial soil sample set of the target domain and the updated candidate sample set until the number of added samples reaches the preset upper limit, at which point the recursive selection stops.

[0033] Optionally, the soil spectral data in the global soil spectral library are visible-near infrared spectra, and the visible-near infrared spectra are preprocessed, including noise band removal, resampling, smoothing filtering, and first derivative transformation.

[0034] Optionally, training the final prediction model for soil organic carbon includes:

[0035] Obtain historical soil sample data and corresponding observations for the target area;

[0036] Historical soil sample data of the target area are input into the final soil organic carbon prediction model to obtain the predicted value;

[0037] With the goal of maximizing the coefficient of determination between observed and predicted values ​​and minimizing the root mean square error between observed and predicted values, the final prediction model for soil organic carbon is trained to obtain the trained final prediction model for soil organic carbon.

[0038] Optionally, the coefficient of determination and root mean square error between the observed and predicted values ​​are determined based on the following formula:

[0039] ;

[0040] ;

[0041] in, As the coefficient of determination, The root mean square error, For the observed values, For predicted values, The average of the predicted values. n The number of samples.

[0042] The soil sample screening method based on instance transfer learning provided in this invention has the following advantages compared with the prior art:

[0043] This invention integrates local similarity calculation based on Mahalanobis distance with a recursive optimization mechanism based on prediction error feedback. It identifies high-value migration samples that are most similar to the target region samples and significantly improve modeling performance in the global soil spectral library. Through recursive screening and dynamic performance evaluation of candidate samples, it effectively avoids interference and overfitting problems caused by the introduction of redundant samples, improves the effectiveness of training data, and enhances the accuracy and generalization ability of the model in soil prediction of the target region. Attached Figure Description

[0044] Figure 1 A graph showing the impact of recursive sample addition on RMSE for a soil sample screening method based on instance transfer learning provided in one embodiment;

[0045] Figure 2 PCA diagram showing the feature distribution relationship between local samples and screened transfer samples in a soil sample screening method based on instance transfer learning provided in one embodiment;

[0046] Figure 3 A comparison chart of SOC prediction performance before and after adding transfer samples is provided for different models of a soil sample screening method based on instance transfer learning in one embodiment. Figure 3 (a) shows a comparison of SOC prediction performance before and after adding transfer samples to 1D-CNN. Figure 3 (b) in the figure shows the comparison of SOC prediction performance before and after adding transfer samples in Cubist.

[0047] Figure 4 This is a flowchart illustrating a soil sample screening method based on instance transfer learning, as provided in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0049] This invention provides a soil sample screening method based on instance transfer learning, such as... Figure 4 As shown, the method includes:

[0050] Obtain the global soil spectral library and the local soil sample set of the target area. Use the global soil spectral library as the source soil sample set and the local soil sample set as the initial soil sample set of the target area.

[0051] From the source domain soil sample set, multiple soil samples whose Mahalanobis distance from the initial target domain soil sample set is greater than a set threshold are selected to obtain a candidate sample set.

[0052] The linear regression model is trained based on the initial soil sample set of the target domain to obtain the initial soil organic carbon prediction model. Soil samples from the candidate sample set are added one by one to the initial soil sample set of the target domain. After each addition of a sample, the linear regression model is trained based on the updated initial soil sample set of the target domain to obtain the provisional soil organic carbon prediction model, and the prediction error of each provisional soil organic carbon prediction model is determined.

[0053] Soil samples corresponding to the provisional soil organic carbon prediction model with the largest decrease in prediction error compared to the initial soil organic carbon prediction model are selected as migration samples. These migration samples are added to the initial set of soil samples in the target domain and removed from the candidate sample set. This recursive selection process continues to obtain a subset of migration samples. In each round of recursive selection, the linear regression model is trained based on the migration sample subset to determine the prediction error corresponding to that subset. After the recursive selection is completed, the migration sample subset with the smallest prediction error is selected as the optimal subset.

[0054] The optimal subset is merged with the initial set of soil samples in the target domain to form the final set of soil samples in the target domain. The deep learning model or machine learning model is then trained using the final set of soil samples in the target domain to obtain the final prediction model of soil organic carbon in the target area.

[0055] The specific implementation includes:

[0056] 1. Target Domain Model Construction: The initial prediction model is trained using soil samples collected from the target area, and its prediction error is calculated through cross-validation to guide the subsequent sample selection process.

[0057] 2. Preliminary Sample Screening: The K-nearest neighbors algorithm combined with Mahalanobis distance is used to calculate the similarity between samples in the source domain and samples in the target domain, and the samples with the highest similarity are selected. M The candidate sample set consists of 1 source domain sample.

[0058] Using Mahalanobis distance as a similarity metric, the K-nearest neighbor (KNN) method was employed in a global soil spectral library to select the samples most similar to those in the target region. M These samples constitute the candidate sample set.

[0059] (1) Let the local sample set of the target region be:

[0060] ;

[0061] (2) The mean vector and covariance matrix of the local samples are calculated as follows:

[0062] ;

[0063] ;

[0064] (3) For any global sample Its Mahalanobis distance from the local center is:

[0065] ;

[0066] (4) All global samples are sorted by Sort in ascending order and select the first few. MThe candidate sample set consists of 10 samples:

[0067] ;

[0068] in, The Mahalanobis distance, Soil samples from the initial soil sample set of the target domain. For global samples in the source domain soil sample set, It is the mean vector. Let covariance matrix be the variance matrix. M The total number of samples in the candidate sample set. This is the initial set of soil samples for the target domain. For the candidate sample set, ( , ) are candidate samples.

[0069] 3. Forward recursive filtering.

[0070] Samples are sequentially added to the target domain training set from the candidate sample set. After each addition, the sample is combined with the current training set to form new training data, and a temporary linear regression prediction model is constructed to evaluate the impact of the sample on the model's prediction error (multiple intermediate models). The prediction error of the intermediate model is calculated through cross-validation and compared with the error of the previous model. The sample that causes the most significant reduction in error is added to the training set and removed from the candidate set. This process is repeated until the number of added samples reaches a preset upper limit M or the prediction error no longer decreases significantly.

[0071] (1) Using initial target region samples ( n The prediction model was trained and its initial prediction error was evaluated using 5-fold cross-validation.

[0072] ;

[0073] (2) Sequentially sort the candidate samples ( , Adding to the training set constitutes Calculate the prediction error of the provisional soil organic carbon prediction model and define:

[0074] ;

[0075] (3) Define the decrease caused by this candidate sample as:

[0076] ;

[0077] in, This is the initial prediction error. The prediction error of the provisional soil organic carbon prediction model. n This represents the initial number of target region samples in the initial set of soil samples for the target domain. This refers to the initial target region samples in the initial set of soil samples for the target domain. For the updated soil samples in the target domain soil sample set, The percentage decrease in prediction error compared to the initial prediction model for soil organic carbon.

[0078] Select The largest sample is added to the training set and removed from the candidate set, and the sample pool is updated to continue the selection process.

[0079] 4. Determining the optimal subset.

[0080] For each round of recursive screening, the source domain sample combination and the target domain sample are jointly modeled, and the prediction error of each round of model is recorded. The error values ​​corresponding to all historical combinations are compared, and a subset of source domain samples that minimizes the prediction error is selected as the final screening result.

[0081] The selection process is repeated, continuing to choose samples from the candidate set that show the greatest improvement in RMSE and adding them to the local sample set, until the preset maximum number of candidate samples M is reached. The final training set is... , where A is the optimal subset obtained by recursive selection.

[0082] 5. Transfer modeling and prediction.

[0083] The optimal subset is merged with the initial set of soil samples in the target domain to form the final set of soil samples in the target domain. The deep learning model or machine learning model is then trained using the final set of soil samples in the target domain to obtain the final soil organic carbon prediction model for predicting soil organic carbon in the target area.

[0084] 6. Evaluate the accuracy of the model prediction results.

[0085] The coefficient of determination (R²) is uniformly used to determine the model's prediction accuracy. 2 The root mean square error (RMSE) was used as the evaluation parameters. In all modeling experiments, the above two indicators were used to compare and analyze the performance of different sample configurations and modeling methods, so as to comprehensively evaluate the improvement effect of the proposed soil sample screening method based on instance transfer learning (KNN-FRSS method) on model accuracy and stability.

[0086] Using the optimized training sample set A final prediction model was constructed, and the target soil properties were predicted based on data from the target area. The coefficient of determination (R²) was consistently used to determine the prediction accuracy of the model. 2The root mean square error (RMSE) is used as an evaluation parameter. The specific formula is as follows:

[0087] ;

[0088] ;

[0089] in, These are observed values. It is a predicted value. It is the average of the predicted values. n This refers to the number of samples. The above metrics comprehensively reflect the model's predictive accuracy and generalization ability in the target region. Specific implementation examples:

[0091] This embodiment uses the publicly available European LUCAS soil spectral database to verify the applicability and effectiveness of the proposed KNN-FRSS sample screening method in soil organic carbon (SOC) prediction tasks.

[0092] 1. Data Preparation: This study uses soil chemical properties and visible-near-infrared (Vis-NIR) spectra collected by Eurostat during the European Commission Land Use / Cover Area Frame Statistical Survey (LUCAS) conducted from 2008 to 2012, encompassing a total of 19,036 soil sampling points. This survey covered 23 EU member states, including France, Italy, and Sweden, with a total area of ​​4.38 million square kilometers (Panagos, Meusburger et al. 2014). A uniformly designed stratified random sampling method was used for sample collection. Five soil samples were collected from each point using a cross-sectional method. The locations were recorded using GPS, and the samples were then sent to the laboratory for air drying, grinding, and mixing. Approximately 500g of the samples were retained for subsequent analysis. SOC content was determined by the dry burning method according to the ISO10694-1995 international standard; spectral acquisition was performed using a FOSS XDS near-infrared spectrometer with a wavelength range of 400–2500 nm and an interval of 0.5 nm, and the output raw spectrum contained 4200 bands.

[0093] To simulate small-sample modeling conditions, this study selected 210 soil samples from the French South-Pyrenees region (local samples n=210) from the LUCAS database as the target area samples, and 18,826 soil samples from other regions (excluding the French South-Pyrenees) as the LUCAS global dataset (N) to provide cross-regional knowledge transfer support. To ensure the independence and realism of the model evaluation, the local samples were randomly divided into training and validation sets in a 7:3 ratio. Transferred samples were only added to the training set, while the validation set remained unchanged.

[0094] Table 1. Descriptive statistics of soil organic carbon (SOC) content (g kg) in different datasets -1 )

[0095]

[0096] Table 1 shows the statistical characteristics of SOC content in the global sample set (N=18826) and the local sample set (n=210). It can be seen that the global sample has a wider SOC range, a higher mean, and a significantly higher coefficient of variation than the local sample, reflecting stronger heterogeneity; in contrast, the local sample is more concentrated. To improve modeling performance, it is necessary to select migration samples from the global sample set that are closer in characteristics to the local samples, thus reducing the inter-domain distribution differences.

[0097] 2. Spectral Preprocessing: All samples from the LUCAS spectral database underwent uniform preprocessing before use, including noise band removal, Savitzky-Golay smoothing filtering, and first-order derivative transformation to enhance the correlation between soil properties and spectral characteristics. To ensure the quality of spectral data and reduce the impact of noise on subsequent analysis, this study performed multi-step preprocessing on both global and local soil spectral data. First, the spectral data was resampled to 10nm intervals. Resampling not only reduced data dimensionality but also preserved the main spectral features. Then, the Savitzky-Golay filter was used to smooth the spectral data to reduce high-frequency noise while preserving the main trends and features of the spectrum. The smoothing parameters were set to zero-order derivative, polynomial order 2, and window width of 15nm. The combination of these preprocessing methods made the spectral data more stable and suitable for subsequent modeling and analysis.

[0098] 3. Transfer Sample Screening: Using the KNN-FRSS method proposed in this invention, high-quality transfer samples that are most similar to the training samples of the target region in the spectral feature space are screened from the global samples and added to the target training set as external supplementary samples to improve the representativeness and sufficiency of the modeling data. The entire process is divided into two stages:

[0099] In the initial screening stage: Mahalanobis distance was used as the sample similarity metric. The Mahalanobis distance between each global sample and the mean of the local training set samples was calculated. This comprehensively considered the covariance structure between different spectral bands, more accurately reflecting the overall feature similarity between samples. The samples were sorted in ascending order of distance, and the top 500 samples (M=500) that were most similar to the spectrum of the target region were selected to form the initial candidate sample set for subsequent fine screening.

[0100] Forward Recursive Selection Phase: A model performance feedback mechanism is introduced, employing a forward recursive strategy to dynamically select the optimal samples from the candidate pool. In this phase, an initial prediction model is built using local training set samples, and its initial prediction error RMSE on a fixed validation set is calculated. Subsequently, for each sample in the initial candidate sample pool, it is attempted to be added to the current training set to build a temporary model, and its RMSE on the validation set is evaluated. In each round, the sample that causes the most significant decrease in RMSE is selected to be added to the training set, while the sample is removed from the candidate pool. The above process is repeated until the number of added samples reaches the preset upper limit M=500. Finally, a subset A of transfer samples that can significantly improve prediction accuracy is determined.

[0101] The intermediate model refers to a temporary SOC prediction model built based on the current training set after each round of sample addition, used to evaluate the actual contribution of the newly added samples to the model performance. The entire sample selection process is implemented using Python programming, ensuring its objectivity and repeatability.

[0102] Figure 1 This paper presents the performance trend (measured by RMSE) of the KNN+MGFS method under different sample sizes. It also demonstrates the impact of different sample sizes extracted by the KNN-FRSS method on model performance (in terms of RMSE). The results show that RMSE generally decreases with increasing sample size. Initially, when the sample size is insufficient, the model prediction error is large, indicating that the samples at this stage are insufficient to effectively represent global features, limiting the modeling effect. When 48 samples are introduced, the model RMSE reaches its lowest value (15.29 g kg⁻¹), indicating optimal performance. Subsequently, as the sample size further increases, RMSE shows a significant increase near 100 samples, presumably related to the large distributional differences between some selected samples and the target domain or the presence of outliers, leading to a decrease in model generalization ability. After exceeding 100 samples, RMSE tends to stabilize but does not decrease significantly further, indicating that additional samples have limited effect on improving model performance and may even introduce redundant interference.

[0103] Based on the comprehensive analysis, the KNN+MGFS method performed optimally when the number of samples A was 48. At this size, the SOC content of the selected samples ranged from 8.20 to 99.50 g kg⁻¹, exhibiting high consistency in spectral characteristics. This facilitated the construction of a stable and representative training set, improving the model's predictive performance in the target region. The relatively concentrated distribution and high consistency in spectral characteristics provided more stable and representative training samples for subsequent model training, significantly enhancing the model's predictive performance in the target domain.

[0104] 4. To further verify the similarity between the selected migration sample subset (A) and the local sample set (n) in the spectral feature space using the KNN-FRSS method, principal component analysis (PCA) was used to perform a distribution visualization analysis on the local samples (n=210) and the finally selected migration sample subset (A=48). The results are as follows: Figure 2 As shown.

[0105] The first principal component (PC1) and the second principal component (PC2) collectively explained 98.54% of the total variance (PC1 accounting for 95.69% and PC2 accounting for 2.85%), indicating that the first two principal components can fully reflect the main differences between samples. In terms of distribution, local samples are mainly concentrated in the central region of the graph, while migrated samples highly overlap with them in the PC1-PC2 space. The 95% confidence ellipses of the two groups of samples are basically consistent, indicating that the KNN-FRSS method successfully screened migrated samples with high similarity to local samples in feature distribution within the high-dimensional spectral feature space. This result verifies the effectiveness of the KNN-FRSS method in maintaining sample consistency and enhancing data representativeness from the perspective of feature distribution, contributing to improved stability and generalization performance in subsequent modeling.

[0106] 5. Based on the final training set consisting of local samples and transfer samples selected by the KNN-FRSS method, two modeling models, one-dimensional convolutional neural network (1D-CNN) and Cubist, were constructed to predict soil organic carbon (SOC) content. These models were then used to evaluate the applicability and robustness of the proposed sample selection method under different modeling frameworks. During the modeling process, spectral preprocessed data (after resampling and Savitzky-Golay smoothing) was used as input features, and the corresponding measured SOC values ​​were used as output labels to construct the model training dataset. The validation set remained unchanged during model training to ensure the independence of the evaluation results and a true reflection of the generalization ability.

[0107] (1) Construction of 1D-CNN model.

[0108] A unified one-dimensional convolutional neural network (CNN) structure was used to model the sample data, as shown in Table 2. The network structure included an input layer, four convolutional layers, a batch normalization layer (BatchNorm), a ReLU activation function, a max pooling layer, and a fully connected regression output layer. The input was single-channel one-dimensional spectral data, which underwent feature extraction layer by layer through convolutional and pooling layers, ultimately outputting a 16-dimensional one-dimensional feature vector. This vector was flattened and fed into the fully connected layer for continuous value prediction of SOC content. The network loss function was mean squared error (MSELoss), the optimizer was AdamW, the learning rate was set to 0.0005, and the weight decay parameter was 5e-5. Batch training was used during model training, and the model was iterated on the training set until the loss converged. This model was implemented on the PyTorch platform.

[0109] Table 2 Parameters of a one-dimensional convolutional neural network model

[0110]

[0111] (2) Cubist model construction.

[0112] The Cubist model is an ensemble learning method based on regression trees, combining rule induction and linear modeling strategies to achieve good regression results on high-dimensional continuous variable data. This study uses the Cubist package in R for modeling. The input features are the same spectral variables as in 1D-CNN, and the output label is SOC content. The default parameters for the Cubist model are: a maximum of 100 rules per tree and a minimum sample size of 5 per model.

[0113] 6. The model prediction accuracy is uniformly based on the coefficient of determination (R²). 2 The prediction results of each modeling method under different sample configurations are shown in Table 3 and . Figure 3 As shown.

[0114] Table 3 Comparison of SOC prediction accuracy under different model and sample combinations

[0115]

[0116] From Table 2 and Figure 3 The results show that the transfer samples selected using KNN-FRSS significantly improved the SOC prediction performance across different model frameworks. Taking 1D-CNN as an example, the test set R... 2 The value increased from 0.78 to 0.88, and the RMSE increased from 8.12 g·kg⁻¹. - ¹ Reduced to 6.52 g·kg - ¹; The Cubist model also shows a similar trend, R 2The value increased from 0.71 to 0.78, and the RMSE increased from 10.09 g·kg⁻¹. - ¹ decreased to 8.22 g·kg - ¹ This shows that the KNN-FRSS method is not only applicable to deep learning models, but also to traditional regression methods, demonstrating good versatility and robustness. Figure 3 The effectiveness of this method was further verified. After introducing the selected samples, the fit between the model's predicted values ​​and the true values ​​was significantly enhanced, and the data points were closer to the ideal diagonal, reflecting a reduction in model bias and an improvement in prediction accuracy.

[0117] In summary, KNN-FRSS, as a transfer sample selection strategy that integrates local similarity and model feedback, can effectively alleviate the problem of sample scarcity in the target area and demonstrates superior predictive performance and cross-domain adaptability in soil spectral modeling.

[0118] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A soil sample screening method based on instance transfer learning, characterized in that, include: Obtain the global soil spectral library and the local soil sample set of the target area, and use the global soil spectral library as the source soil sample set and the local soil sample set as the initial soil sample set of the target area. From the source domain soil sample set, multiple soil samples whose Mahalanobis distance from the initial target domain soil sample set is greater than a set threshold are selected to obtain a candidate sample set. The linear regression model is trained based on the initial soil sample set of the target domain to obtain the initial soil organic carbon prediction model; soil samples from the candidate sample set are added one by one to the initial soil sample set of the target domain. After each addition of a sample, the linear regression model is trained based on the updated initial soil sample set of the target domain to obtain the temporary soil organic carbon prediction model, and the prediction error of each temporary soil organic carbon prediction model is determined. Soil samples corresponding to the temporary soil organic carbon prediction model with the largest decrease in prediction error compared to the initial soil organic carbon prediction model are used as migration samples. The migration samples are added to the initial set of soil samples in the target domain and removed from the candidate sample set. Continue recursively filtering to obtain a subset of transfer samples; in each round of recursive filtering, train the linear regression model based on the subset of transfer samples to determine the prediction error corresponding to the subset of transfer samples; after the recursive filtering is completed, take the subset of transfer samples with the smallest prediction error as the optimal subset. The optimal subset is merged with the initial set of soil samples in the target domain to form the final set of soil samples in the target domain. The deep learning model or machine learning model is then trained using the final set of soil samples in the target domain to obtain the final prediction model of soil organic carbon in the target area.

2. The soil sample screening method based on instance transfer learning as described in claim 1, characterized in that, The process of selecting multiple soil samples from the source domain soil sample set whose Mahalanobis distance from the initial target domain soil sample set is greater than a set threshold, to obtain a candidate sample set, specifically includes: The initial soil sample set for the target domain is represented by the following formula: ； The mean vector and covariance matrix of the initial set of soil samples in the target domain are determined based on the following formula: ； ； For any global sample in the source domain soil sample set, the Mahalanobis distance of each global sample is determined based on the mean vector and covariance matrix according to the following formula: ； All global samples in the source domain soil sample set are sorted in ascending order of Mahalanobis distance. Global samples with a Mahalanobis distance greater than a set threshold are aggregated into a candidate sample set based on the following formula: in, The Mahalanobis distance, Soil samples from the initial soil sample set of the target domain. For global samples in the source domain soil sample set, It is the mean vector. Let covariance matrix be the variance matrix. M The total number of samples in the candidate sample set. This is the initial set of soil samples for the target domain. For the candidate sample set, ( , ) are candidate samples.

3. The soil sample screening method based on instance transfer learning as described in claim 1, characterized in that, The determination of the prediction error of each provisional soil organic carbon prediction model specifically includes: The initial prediction error of the soil organic carbon initial prediction model is determined based on the following formula; ； Soil samples from the candidate sample set are added one by one to the initial soil sample set of the target domain. After each addition, a deep learning model or a traditional machine learning model is trained based on the updated target domain soil sample set to obtain a provisional soil organic carbon prediction model. The prediction error of each provisional soil organic carbon prediction model is determined based on the following formula: ； in, This is the initial prediction error. The prediction error of the provisional soil organic carbon prediction model. n This represents the initial number of target region samples in the initial set of soil samples for the target domain. This refers to the initial target region samples in the initial set of soil samples for the target domain. For the updated target domain soil sample set, soil samples are used.

4. The soil sample screening method based on instance transfer learning as described in claim 3, characterized in that, The decrease in prediction error compared to the initial soil organic carbon prediction model is determined based on the following formula: ； in, The percentage decrease in prediction error compared to the initial prediction model for soil organic carbon; Soil samples corresponding to the temporary soil organic carbon prediction model with the largest decrease in prediction error compared to the initial soil organic carbon prediction model are used as migration samples. These samples are added to the initial soil sample set of the target domain and removed from the candidate sample set. The recursive selection continues based on the updated initial soil sample set of the target domain and the updated candidate sample set until the number of added samples reaches the preset upper limit, at which point the recursive selection stops.

5. The soil sample screening method based on instance transfer learning as described in claim 1, characterized in that, The soil spectral data in the global soil spectral library are visible-near infrared spectra, and the visible-near infrared spectra are preprocessed, including noise band removal, resampling, smoothing filtering, and first derivative transformation.

6. The soil sample screening method based on instance transfer learning as described in claim 1, characterized in that, Training the soil organic carbon final prediction model specifically includes: Obtain historical soil sample data and corresponding observations for the target area; Historical soil sample data of the target area are input into the soil organic carbon final prediction model to obtain the predicted value; The soil organic carbon final prediction model is trained with the goal of maximizing the coefficient of determination between the observed and predicted values ​​and minimizing the root mean square error between the observed and predicted values, resulting in the trained soil organic carbon final prediction model.

7. The soil sample screening method based on instance transfer learning as described in claim 6, characterized in that, The coefficient of determination and root mean square error between observed and predicted values ​​are determined based on the following formula: ； ； in, As the coefficient of determination, The root mean square error, For the observed values, For predicted values, The average of the predicted values. n The number of samples.

Images