Decision tree-based lycium barbarum field pesticide residue analysis method and system

Abstract

The invention discloses a decision tree-based lycium barbarum field pesticide residue analysis method and system, and belongs to the technical field of lycium barbarum field pesticide residue analysis, and the method comprises the following steps: acquiring spectral information of lycium barbarum samples of different lycium barbarum varieties, producing areas and harvesting times by using a near infrared spectrometer, and determining the content of pesticide residues on the surfaces of lycium barbarum through a high performance liquid chromatography, establishing an associated data set of the spectral characteristics and the pesticide residue content; the method comprises the following steps: preprocessing the obtained wolfberry spectral data, including denoising, normalization and the like, improving the quality and consistency of spectral information, then extracting a characteristic wave band which can best reflect the pesticide residue content difference from mass spectral data by adopting a principal component analysis method, and reducing the data dimension; according to the extracted characteristic wave band, a quantitative correlation model of the spectral characteristics and the pesticide residue content is established by adopting a partial least squares regression method, parameters of the model are optimized through cross validation and external validation, and the robustness and prediction precision of the model are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of pesticide residue analysis in wolfberry fields, and in particular relates to a method and system for pesticide residue analysis in wolfberry fields based on a decision tree. Background Art

[0002] As a popular tonic food, the quality and safety of wolfberry has always been a concern. In the process of wolfberry cultivation, farmers often use various pesticides to increase yields and prevent pests and diseases, resulting in pesticide residues on the surface of wolfberry fruits, which endangers the health of consumers. Traditional pesticide residue detection methods require pre-treatment of wolfberry samples, which is time-consuming and labor-intensive, and it is difficult to achieve rapid screening of large batches of samples. Near-infrared spectroscopy is a rapid and non-destructive detection technology that can obtain spectral information without destroying wolfberry fruits. However, how to accurately determine the pesticide residue content from massive spectral data and establish a correlation model between spectral characteristics and pesticide residue content is a technical problem that needs to be solved urgently.

[0003] In addition, due to the uneven distribution of pesticide residues on the surface of wolfberry fruit, as well as differences in wolfberry varieties, origins, harvesting times and other factors, the spectral information obtained has large volatility and uncertainty, which poses a challenge to the establishment and optimization of the model. At the same time, how to screen out the characteristic bands with the greatest correlation with pesticide residue content from massive spectral data and build a robust and efficient decision tree model is also a technical difficulty. How to balance the complexity and generalization ability of the model to avoid overfitting and underfitting requires in-depth research in algorithm design and parameter tuning. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for analyzing pesticide residues in wolfberry fields based on a decision tree to solve the problems existing in the above prior art.

[0005] To achieve the above object, the present invention provides a method for analyzing pesticide residues in wolfberry fields based on a decision tree, comprising:

[0006] Obtain a dataset of the correlation between the spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvest times;

[0007] Preprocessing the associated data set to obtain a characteristic band data set;

[0008] Building a model based on partial least squares regression method, training the model with the characteristic band data set to obtain a correlation model;

[0009] Real-time wolfberry spectrum data is acquired, the real-time wolfberry spectrum data is calculated based on the association model to generate a calculation result, the calculation result is judged by a decision tree algorithm, and the wolfberries are divided into qualified products and unqualified products.

[0010] Preferably, the process of obtaining a data set of association between spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvest times comprises:

[0011] Near-infrared spectrometer was used to obtain the spectral information of wolfberry samples of different wolfberry varieties, origins and harvest times;

[0012] The pesticide residue content on the surface of wolfberry was determined by HPLC;

[0013] A correlation data set between spectral features and pesticide residue content is established based on the spectral information of the wolfberry sample and the pesticide residue content on the surface of the wolfberry.

[0014] Preferably, the process of preprocessing the associated data set to obtain the characteristic band data set includes:

[0015] dividing the associated data set into an initial spectral data set and a pesticide residue content data set;

[0016] After performing denoising and normalization preprocessing on the initial spectral data set, a spectral data matrix is ​​constructed;

[0017] Performing principal component analysis on the spectral data matrix, and obtaining the cumulative contribution rate of the principal components of the data matrix through eigenvalue decomposition and eigenvector extraction;

[0018] According to the cumulative contribution rate of the principal component, a new characteristic band is determined to obtain spectral characteristic data after dimension reduction;

[0019] The spectral feature data after dimension reduction is combined with the pesticide residue content data set to obtain the characteristic band data set.

[0020] Preferably, the process of obtaining the cumulative contribution rate of the principal components of the data matrix includes:

[0021] Performing principal component analysis on the spectral data matrix to obtain the contribution rate of each principal component;

[0022] According to the contribution rate of each principal component, a cumulative contribution rate curve is drawn, and the inflection point position of the curve is determined to complete the acquisition of the cumulative contribution rate of the principal component.

[0023] Preferably, the process of constructing a model based on partial least squares regression method and training the model through the characteristic band data set to obtain the correlation model includes:

[0024] A prediction model is established by using a partial least squares regression algorithm, taking the characteristic band as an independent variable and the pesticide residue content as a dependent variable;

[0025] The prediction model is trained based on the characteristic band data set to obtain the association model.

[0026] Preferably, the prediction model is trained based on the characteristic band data set to obtain the correlation model, which includes:

[0027] Determining the optimal number of principal components of the prediction model by a cross-validation method;

[0028] The prediction model is used to predict the pesticide residue in the characteristic band data set, and the predicted value is compared with the measured value to evaluate the prediction accuracy of the prediction model; if the prediction accuracy does not reach the preset threshold, the model parameters are adjusted and the quantitative relationship model is re-established until the prediction accuracy meets the requirements.

[0029] Preferably, the process of determining the optimal number of principal components of the prediction model by a cross-validation method comprises:

[0030] Using a K-fold cross-validation method, the preprocessed data set is divided into K subsets;

[0031] For each fold of cross-validation, a partial least squares regression model is constructed using the corresponding training set;

[0032] Analyze the results of the K-fold cross validation to determine the optimal number of principal components;

[0033] According to the optimal number of principal components, a partial least squares regression model is trained on the preprocessed data set to obtain the prediction model.

[0034] Preferably, the process of judging the calculation results by a decision tree algorithm and dividing wolfberries into qualified and unqualified products includes: constructing a decision tree algorithm, inputting the calculation results into the decision tree algorithm, and performing qualitative judgment through the branch structure and node threshold of the decision tree, if the pesticide residue content is lower than a preset threshold, the wolfberry sample is judged to be a qualified product, otherwise it is judged to be an unqualified product.

[0035] To achieve the above object, the present invention also provides a wolfberry field pesticide residue analysis system based on a decision tree, comprising:

[0036] A data acquisition module is used to obtain a data set of association between spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvest times;

[0037] A preprocessing module, used for preprocessing the associated data set to obtain a characteristic band data set;

[0038] A model building module, used to build a model based on partial least squares regression method, train the model through the characteristic band data set, and obtain a correlation model;

[0039] The discrimination module is used to obtain real-time wolfberry spectrum data, calculate the real-time wolfberry spectrum data based on the association model, generate a calculation result, discriminate the calculation result through a decision tree algorithm, and classify the wolfberries into qualified products and unqualified products.

[0040] Compared with the prior art, the present invention has the following advantages and technical effects:

[0041] The present invention discloses a method for rapid and non-destructive detection of the pesticide residue content of wolfberry based on near-infrared spectroscopy technology and chemometric methods. The spectral information of wolfberry samples of different varieties, origins and harvesting times is collected, and a correlation data set is established with the pesticide residue content determined by high performance liquid chromatography. After pre-processing the spectral data, principal component analysis is used to extract characteristic bands, and then a quantitative correlation model between spectral characteristics and pesticide residue content is established through partial least squares regression, and the pesticide residue content of wolfberry is qualitatively judged in combination with a decision tree algorithm. The present invention can realize rapid, accurate and non-destructive detection of the pesticide residue content of wolfberry, providing a powerful technical means for the quality and safety control of wolfberry. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 This is a flow chart of a method for analyzing pesticide residues in wolfberry fields based on a decision tree according to an embodiment of the present invention;

[0044] Figure 2 A flow chart for model construction of an embodiment of the present invention;

[0045] Figure 3 This is a model training flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Embodiment 1

[0049] like Figure 1-3 As shown, this embodiment provides a method for analyzing pesticide residues in wolfberry fields based on a decision tree, comprising:

[0050] S101. For wolfberry samples of different wolfberry varieties, origins and harvesting times, their spectral information was obtained by near-infrared spectrometer, and the pesticide residue content on the wolfberry surface was determined by high performance liquid chromatography to establish a correlation data set between spectral characteristics and pesticide residue content.

[0051] The near-infrared spectral information and the pesticide residue content determination results of the wolfberry sample are obtained, and a mapping relationship data set between the spectral characteristics and the pesticide residue content is established; for the mapping relationship data set, a partial least squares regression algorithm is used, with the spectral characteristics as the independent variable and the pesticide residue content as the dependent variable, to construct a pesticide residue content prediction model; the pesticide residue content prediction model is applied to actual detection, the near-infrared spectral information of the wolfberry sample to be tested is input, and the corresponding pesticide residue content prediction result is output through the prediction model.

[0052] Specifically, in order to establish a prediction model for the pesticide residue content in wolfberry, it is first necessary to collect representative wolfberry samples. According to the attributes of wolfberry varieties, origins and harvesting time, 500 wolfberry samples were collected, and the spectral information of each sample was obtained using a near-infrared spectrometer. At the same time, the content of 10 common pesticide residues in each sample was determined by high-performance liquid chromatography, including dimethoate, dichlorvos, and methyl parathion. In the data preprocessing stage, 400 samples were selected from 500 samples as the modeling set and 100 samples as the validation set by the Kennard-Stone algorithm, and the spectral data of all samples were standardized to remove the influence of spectral baseline drift and scattering effects. In the modeling stage, the partial least squares regression algorithm was used to establish a prediction model for 10 pesticide residues, and the optimal number of principal components was determined by 10-fold cross-validation, and the average relative error was less than 10%. For pesticide residues with insufficient sample data, the pesticide residue model was initialized using some parameters of other pesticide residue models through the transfer learning method, and then fine-tuned with a small number of samples to obtain better prediction performance. In practical applications, the established prediction model is used to analyze the near-infrared spectral data of the wolfberry samples to be tested, and the predicted values ​​of the contents of 10 kinds of pesticide residues can be obtained within 2 minutes, providing a powerful tool for the rapid screening of wolfberry pesticide residues. Through 3 months of actual testing, the predicted values ​​and measured values ​​of the pesticide residue content of 2,000 wolfberry samples were obtained, and the correlation coefficients between the predicted values ​​and the measured values ​​were all above 95, indicating that the prediction model has high accuracy and stability, and can meet the needs of rapid detection of wolfberry pesticide residues.

[0053] S102, preprocessing the acquired wolfberry spectral data, including denoising, normalization, etc., to improve the quality and consistency of spectral information. Then, principal component analysis is used to extract characteristic bands that best reflect the difference in pesticide residue content from massive spectral data to reduce data dimension.

[0054] The original spectral data collected from different batches of wolfberry samples are obtained to obtain an initial spectral data set; after denoising and normalization preprocessing the spectral data, a spectral data matrix is ​​constructed; principal component analysis is performed on the spectral data matrix, and the principal component eigenvectors of the data matrix are obtained through eigenvalue decomposition and eigenvector extraction; according to the cumulative contribution rate of the principal components, the first few principal components with the largest contribution rates are determined as new characteristic bands to obtain spectral characteristic data after dimension reduction; the spectral characteristic data are used as the input of the machine learning model, and combined with the measured value of the pesticide residue content, a regression prediction model of spectral characteristics and pesticide residue content is trained.

[0055] Specifically, in the detection of pesticide residues in wolfberry, a near-infrared spectrometer is first used to collect spectra of wolfberry samples from different batches. The wavelength range is 800-2500nm and the sampling interval is 2nm to obtain the initial high-dimensional spectral data. Then, the wavelet transform is used to decompose the spectral data into 5 levels, and the wavelet coefficients of the 3rd to 5th levels are selected to reconstruct the spectral signal, which effectively removes the high-frequency noise interference in the spectrum. Then, the denoised spectral data is normalized to the maximum and minimum values, and the spectral intensity is uniformly mapped to between 0 and 1, eliminating the scale difference between samples from different batches. After normalization, the spectral data of each sample is converted into a 1×700 row vector, and the spectral vectors of multiple samples are spliced ​​row by row to form a high-dimensional data matrix. The matrix is ​​subjected to principal component analysis, and the principal component eigenvector is obtained by eigenvalue decomposition. According to the cumulative contribution rate curve, the first four principal components are selected as new characteristic bands. Each principal component is a linear combination of the original spectral data, and the main information of the spectral data is extracted. Finally, a partial least squares regression model was constructed with these four principal component features as input and the measured values ​​of pesticide residue content as output. The model parameters were optimized through cross-validation, and finally an efficient and accurate spectral prediction model for wolfberry pesticide residue content was established, which provided a new idea for the rapid detection of wolfberry pesticide residues.

[0056] The principal component analysis of the spectral data matrix is ​​performed through eigenvalue decomposition and eigenvector extraction, and the principal component eigenvectors are determined to extract the main information of the spectral data.

[0057] According to the size of the eigenvalues, select at least one eigenvector corresponding to the maximum eigenvalue as the principal component eigenvector; use the principal component matrix composed of the principal component eigenvectors to replace the spectral data matrix to obtain the spectral data after dimension reduction; input the spectral data after dimension reduction into a support vector machine to obtain a spectral data classification model; input the target spectral data to be classified into the classification model to determine the category to which the target spectral data belongs.

[0058] Specifically, the spectral data is first obtained and converted into a matrix form with a matrix size of 500×1000. Then the matrix is ​​decomposed by eigenvalue to obtain 1000 eigenvalues ​​and corresponding eigenvectors. According to the eigenvalue sorting from large to small, the eigenvectors corresponding to the first 50 largest eigenvalues ​​are selected as the principal component eigenvectors. Since the number of selected principal component eigenvectors meets the preset threshold of 40, the original spectral data matrix is ​​projected on these 50 eigenvectors to obtain a 50×500 principal component matrix after dimensionality reduction. The principal component analysis method is used to replace the original spectral data matrix with the principal component matrix, and the dimensionality reduction and feature extraction of the spectral data from 1000 dimensions to 50 dimensions are realized. Then, the radial basis kernel function in the support vector machine algorithm is used to train the principal component matrix after dimensionality reduction, and the model parameters are optimized by grid search and cross-validation methods. Finally, a high-precision spectral data classification model is established. The classification model is used to predict the newly collected spectral data, and the Euclidean distance between the new data and the center of each category is calculated to determine its category, and the classification accuracy reaches more than 95%. According to the classification results, the spectral data is automatically marked and classified for storage, and data of different categories are stored in corresponding folders to facilitate the classification management and application of spectral data.

[0059] According to the cumulative contribution rate curve, the first few principal components with the largest contribution rates are selected to obtain the spectral feature data after dimension reduction to eliminate redundant information.

[0060] Acquire spectral feature data, perform principal component analysis on the spectral feature data, and obtain the contribution rate of each principal component; draw a cumulative contribution rate curve based on the contribution rate of each principal component, and determine the inflection point position of the curve; if the number of principal components corresponding to the inflection point position is a preset threshold, use the preset threshold number of principal components as the feature data after dimensionality reduction; use the spectral feature data after dimensionality reduction as the input of the machine learning algorithm to train the corresponding model.

[0061] Specifically, first obtain a set of spectral feature data, which contains 1000 samples, each sample has a reflectance value of 500 bands. Perform principal component analysis on this set of data, and obtain the contribution rate of each principal component by calculating the eigenvalue and eigenvector of the covariance matrix. According to the calculation results, the contribution rate of principal component 1 is 60%, the contribution rate of principal component 2 is 25%, the contribution rate of principal component 3 is 10%, and the contribution rate of the remaining principal components is low. Plot the cumulative contribution rate curve in the coordinate system, and it is found that the curve has an obvious inflection point at principal component 3, and the corresponding cumulative contribution rate has reached 95%, so the first three principal components are selected as the feature data after dimensionality reduction. Through principal component selection, the original 500-dimensional spectral feature data is reduced to 3 dimensions, which greatly compresses the data dimension, and the data after dimensionality reduction still retains most of the information of the original data, removing the redundancy and noise in the spectral data. Next, the 3D spectral feature data after dimension reduction was used as the input of the support vector machine (SVM), and the radial basis kernel function was used. The penalty coefficient C was set to 10, and the model was trained through 5-fold cross validation. Finally, an SVM classification model with excellent performance was obtained. The model can be used to predict and analyze the newly collected spectral data, which can quickly and accurately determine the category of the sample and provide important data support for related businesses.

[0062] The partial least squares regression model was trained in combination with the measured values ​​of pesticide residue content, and the model parameters were optimized through cross-validation to obtain an accurate prediction of the pesticide residue content in wolfberry.

[0063] Obtain the known measured values ​​of wolfberry pesticide residue content and the corresponding spectral data, and construct a partial least squares regression model, with the spectral data as the independent variable and the pesticide residue content as the dependent variable. Use the cross-validation method to randomly divide the data set into a training set and a validation set, and determine the optimal model parameters through grid search. Obtain the spectral data of the newly collected wolfberry sample and input it into the prediction model to obtain the predicted value of the pesticide residue content. If the prediction error exceeds the preset threshold, obtain more sample data, retrain the optimization model, and improve the prediction accuracy until the prediction accuracy requirement is met.

[0064] Specifically, 100 wolfberry samples were first collected, and their pesticide residue content was determined by gas chromatography-mass spectrometry. At the same time, near-infrared spectral data of these samples were collected. The spectral data was used as the independent variable matrix X, and the pesticide residue content was used as the dependent variable vector y to construct a partial least squares regression model. The 5-fold cross-validation method was used to randomly divide the data set into 5 subsets, and 4 subsets were selected as training sets each time, and the remaining 1 subset was used as the validation set. Through grid search, the optimal parameter combination of the model was found within the range of 1 to 10 principal components and pretreatment methods including standardization, normalization, and mean centering. The results showed that when the number of principal components was 7 and the pretreatment method was standardization, the root mean square error of the model on the validation set was the smallest, which was 0.5 mg / kg. The partial least squares regression model was retrained according to the optimal parameters to obtain a pesticide residue content prediction model. For the 20 newly collected wolfberry samples, their spectral data were measured and input into the prediction model to obtain the predicted value of the pesticide residue content. By comparing the predicted values ​​with the actual measured values, it was found that the prediction errors were all within 1 mg / kg, which met the accuracy requirements for pesticide residue detection. In order to further improve the prediction performance of the model, an additional 50 wolfberry samples were collected and added to the original data set, and cross-validation and parameter optimization were re-performed to obtain an updated prediction model. The root mean square error of the new model on the validation set was reduced to 0.3 mg / kg, and the prediction accuracy was further improved. Finally, by comprehensively utilizing machine learning methods such as partial least squares regression and cross-validation, combined with spectral data and pesticide residue measurement values, an efficient and accurate wolfberry pesticide residue content prediction model was constructed, providing strong technical support for wolfberry quality and safety testing.

[0065] S103. Based on the extracted characteristic bands, a quantitative correlation model between spectral characteristics and pesticide residue content is established using partial least squares regression method. Through cross-validation and external validation, the parameters of the model are optimized to improve the robustness and prediction accuracy of the model.

[0066] Acquire spectral data of agricultural product samples and extract characteristic bands related to pesticide residues; adopt partial least squares regression algorithm, take the characteristic bands as independent variables and the pesticide residues as dependent variables, and establish a quantitative relationship model; acquire test set agricultural product samples, use the quantitative model to predict the pesticide residues of the test set samples, compare the predicted values ​​with the measured values, and evaluate the prediction accuracy of the quantitative model; if the prediction accuracy does not reach a preset threshold, adjust the model parameters and re-establish the quantitative relationship model until the prediction accuracy meets the requirements.

[0067] Specifically, the spectral data of the collected agricultural product samples are first collected using a visible light-near infrared spectrometer in the wavelength range of 350-2500nm, with spectral data collected at intervals of 1nm. Then, characteristic band extraction algorithms, such as continuous wavelet transform and genetic algorithm, are used to screen out 10 characteristic bands with the highest correlation with pesticide residues from the full-band spectral data, such as 450nm, 680nm, 970nm, etc. Then, the spectral absorption values ​​of these 10 characteristic bands are used as independent variables, and the pesticide residues determined by high-performance liquid chromatography are used as dependent variables to establish a partial least squares regression model. The optimal number of principal components of the model is determined to be 8 through cross-validation, and the determination coefficient R of the model is 100. 2 The model is embedded in a portable spectrometer, and the spectral data of agricultural products are collected on site, and the prediction results of pesticide residues can be obtained within 5 seconds, providing a new technology for fast, non-destructive and environmentally friendly agricultural product quality and safety testing. At the same time, the spectral-pesticide residue data are regularly supplemented, the model is updated and iteratively optimized online, and the applicability and robustness of the model are continuously improved, providing reliable big data support for agricultural product quality and safety supervision.

[0068] Based on the full-band spectral data, the characteristic band extraction algorithm is used to determine the characteristic band with the highest correlation with pesticide residues.

[0069] Acquire full-band spectral data, preprocess the full-band spectral data, remove outliers and noise data, and obtain preprocessed full-band spectral data; use a feature band extraction algorithm to analyze the preprocessed full-band spectral data and extract at least one candidate feature band; calculate the correlation coefficient between each candidate feature band and the pesticide residue, sort the candidate feature bands according to the correlation coefficient, and obtain the target feature band with the highest correlation; use the target feature band as the input of the pesticide residue prediction model, train the machine learning model, and obtain the pesticide residue prediction result.

[0070] Specifically, a hyperspectral imager is first used to obtain full-band spectral data of crop samples, with a wavelength range of 400-2500nm and a spectral resolution of 10nm. The acquired spectral data is denoised by wavelet transform method, Daubechies wavelet basis is selected, the decomposition layer number is 5, and the spectral data is subjected to 5-layer wavelet decomposition to remove high-frequency noise. Then the Pearson correlation coefficient between the spectral data and the pesticide residue is calculated, the correlation coefficient threshold is set to 6, and the bands with correlation coefficients greater than 6 are extracted. The stepwise regression analysis method is adopted, with pesticide residue as the dependent variable and the extracted candidate feature bands as the independent variable, and stepwise regression analysis is performed to screen out the feature band combination with the greatest contribution to pesticide residue. If the number of feature bands screened out is less than 10, the correlation coefficient threshold is reduced to 5, and the candidate feature bands are re-extracted until no less than 10 feature bands are extracted. Finally, the support vector machine model is selected, and the pesticide residue prediction model is constructed with the 10 screened feature bands as input and the pesticide residue as output. The 5-fold cross-validation method was used to divide the data set into a training set and a test set. Parameter optimization was performed on the training set, and model evaluation was performed on the test set. After optimization, the radial basis kernel function was selected as the kernel function of the support vector machine model, the penalty factor C was 10, the parameter γ of the radial basis kernel function was 1, and the determination coefficient R of the final model on the test set was 1. 2 It reaches 85, and the root mean square error RMSE is 0.2 mg / kg, indicating that the model can accurately predict the pesticide residues in crops.

[0071] Through the cross-validation method, the training set and validation set samples are obtained to determine the optimal number of principal components for the partial least squares regression model.

[0072] The invention provides a method for obtaining an original data set, preprocessing the original data set to obtain a preprocessed data set; using a K-fold cross-validation method to divide the preprocessed data set into K subsets; for each fold of cross-validation, using a corresponding training set to construct a partial least squares regression model; comprehensively analyzing the results of the K-fold cross-validation to determine an optimal number of principal components; training a partial least squares regression model on the preprocessed data set according to the optimal number of principal components to obtain an optimized model; and applying the optimized model to a prediction task of a target variable.

[0073] Specifically, after obtaining the original data set, the data is first preprocessed. The maximum likelihood estimation method is used to fill missing values. For continuous variables, the 3σ principle is used to identify outliers and replace them with the median; for categorical variables, the mode is used to fill out outliers. Then, the minimum-maximum standardization method is used to standardize the data and scale the data to the interval [0,1]. Next, a 10-fold cross-validation is used to randomly divide the preprocessed data set into 10 subsets of equal size. In each fold cross-validation, one subset is selected as the validation set, and the remaining 9 subsets constitute the training set. The partial least squares regression model is constructed using the training set data. By grid search, the search range of the number of principal components is set from 1 to min (number of samples, number of variables), and a series of candidate models are obtained. The candidate models are applied to the validation set, and performance indicators such as mean square error and determination coefficient are calculated. The results of the 10-fold cross-validation are comprehensively analyzed, and the average values ​​of the model performance indicators under different numbers of principal components are calculated. The number of principal components with the smallest average mean square error and the largest average determination coefficient is selected as the optimal number of principal components. According to the determined optimal number of principal components, the partial least squares regression model is retrained on all preprocessed data sets to obtain the final optimized model. The optimized model is applied to the actual prediction task, new sample data is input, and the estimated value of the target variable is obtained through model prediction to support business decision-making and analysis. For example, in product quality prediction, by collecting various parameters in the product production process, such as temperature, pressure, raw material properties, etc., a partial least squares regression model is constructed to predict the quality indicators of the product, timely discover and adjust problems in the production process, and improve product quality and production efficiency.

[0074] By building a quantitative model, we can obtain the predicted values ​​of the external test set samples and judge the prediction accuracy and robustness of the model.

[0075] Obtaining external test set sample data, and preprocessing the sample data to obtain preprocessed sample data;

[0076] For a preset machine learning algorithm, a cross-validation method is used to optimize the hyperparameters of the machine learning algorithm to obtain an optimized machine learning model;

[0077] The prediction results of the optimized machine learning model are compared with the true labels of the test set samples, and the prediction accuracy, precision and recall of the machine learning model are calculated.

[0078] Specifically, when obtaining high-quality external test set sample data, it can be obtained through public data sets, business data collection, etc., such as downloading data sets related to business scenarios from the Kaggle platform. After obtaining the data, it is necessary to clean and preprocess the data, such as removing features with more missing values ​​and processing outliers. Then select the appropriate machine learning algorithm according to the characteristics of the data. For example, if the data presents a linear relationship, a linear regression algorithm can be selected; if the data presents a nonlinear relationship, a decision tree or support vector machine algorithm can be selected. After selecting the algorithm, the hyperparameters of the algorithm are optimized through grid search and other methods, such as tuning the penalty coefficient C and kernel function parameter γ of the support vector machine, evaluating the performance of the model under different parameter combinations through 5-fold cross validation, and selecting the parameters with the best performance on the validation set. Use the optimized model to predict the external test set, compare the prediction results with the true labels, and calculate the accuracy, precision, recall, F1 value and other evaluation indicators through Python's sklearn library. If the evaluation indicators meet expectations, such as an accuracy rate of more than 90%, and the performance is stable on multiple external test sets, the model performance can be considered to meet the requirements; otherwise, the model needs to be further optimized, such as building more effective features through feature engineering, increasing the amount of training data, or trying other machine learning algorithms such as ensemble learning. Model optimization is an iterative process that requires continuous attempts and adjustments until a model that meets business needs is obtained.

[0079] S104. Based on the established association model, a decision tree algorithm is used to qualitatively identify the pesticide residue content of wolfberry. By setting a pesticide residue content threshold, wolfberry samples are divided into qualified and unqualified products, and the branch structure and node threshold of the decision tree are optimized to improve the accuracy and recall rate of the identification.

[0080] Obtaining a pre-established wolfberry pesticide residue content association model and the pesticide residue detection data of the wolfberry sample to be identified;

[0081] The pesticide residue detection data is input into a decision tree algorithm model, and a qualitative judgment is made through the branch structure and node threshold of the decision tree. If the pesticide residue content is lower than a preset threshold, the wolfberry sample is judged to be a qualified product, otherwise it is judged to be an unqualified product;

[0082] According to the discrimination results, the discrimination accuracy and recall rate indicators are calculated, and the branch structure and node threshold of the decision tree are adjusted to obtain an optimized discrimination model;

[0083] The optimized discrimination model was applied to new wolfberry samples using a cross-validation method to obtain discrimination results, which were compared with actual pesticide residue detection results.

[0084] Specifically, first, according to the pre-established wolfberry pesticide residue content association model, the pesticide residue detection data of the wolfberry sample to be judged is obtained. For example, the organophosphorus pesticide residues in the wolfberry sample are detected by high performance liquid chromatography-mass spectrometry to obtain the pesticide residue content data of the sample, such as 0.2mg / kg, 0.5mg / kg, etc. Then, the obtained pesticide residue detection data is input into the decision tree algorithm model, and qualitative judgment is performed through the branch structure and node threshold of the decision tree. Assuming that the preset threshold is 1mg / kg, if the pesticide residue content is lower than the threshold, such as 0.2mg / kg, the wolfberry sample is judged to be qualified; if the pesticide residue content is higher than or equal to the threshold, such as 15mg / kg, the wolfberry sample is judged to be unqualified. For the discrimination results, the discrimination accuracy and recall rate indicators are statistically used to evaluate the discrimination performance of the decision tree algorithm model. For example, 100 wolfberry samples are discriminated, of which 80 are actually qualified and 20 are actually unqualified. The discrimination results are 75 qualified and 25 unqualified. The discrimination accuracy is (75+15) / 100=90%, and the recall rate is 75 / 80=975%. According to the results of the discrimination performance evaluation, the branch structure and node threshold of the decision tree are adjusted to optimize the discrimination model. For example, the threshold is adjusted to 0.8 mg / kg, and the discrimination is re-performed to improve the recall rate. The 5-fold cross-validation method is used to apply the optimized discrimination model to new wolfberry samples and obtain the discrimination results. The discrimination results are compared with the actual pesticide residue detection results to further verify the effectiveness of the optimized discrimination model and ensure that it can accurately identify qualified and unqualified products. For example, when another 50 wolfberry samples were discriminated, the discrimination accuracy reached more than 95% and the recall rate reached more than 98%, which means that the optimized discrimination model has good generalization ability and can be applied to the rapid detection and classification of wolfberry pesticide residues in actual production.

[0085] The pesticide residue content data of wolfberry samples were obtained by high performance liquid chromatography-mass spectrometry to obtain the basic data for input into the decision tree model.

[0086] According to the working principle of high performance liquid chromatography-mass spectrometry, the optimal chromatographic column, mobile phase composition and mass spectrometry detection parameters were determined to establish a pesticide residue detection method. Wolfberry samples were obtained, pre-treated, the pesticide residue components were extracted, the interfering matrix was removed, and the target analytes were enriched so that their concentrations met the instrument detection limit requirements. The pesticide residue content data were randomly divided into a training set and a test set, and a decision tree algorithm was used to establish a pesticide residue content prediction model. The model was verified using the test set data, and the model parameters were continuously optimized to improve the prediction accuracy.

[0087] Specifically, according to the working principle of the high performance liquid chromatography-mass spectrometry, a C18 reverse phase column was selected, the mobile phase was an acetonitrile-water system, gradient elution, a flow rate of 8 mL / min, and a column temperature of 35 °C. The mass spectrometer used an electrospray ion source, positive ion mode, spray voltage of 3500 V, ion transmission temperature of 320 °C, and collision gas pressure of 35 psi. 100 wolfberry samples were collected, including 30 from Xinjiang, Ningxia, and Inner Mongolia, and 10 from different varieties. The samples were subjected to pre-treatment steps such as grinding, extraction, and purification. The extract was concentrated to 1 mL by nitrogen blowing and injected after passing through a 22 μm filter membrane. Each sample was injected 3 times and the average value was taken. The test data was imported into SPSS software, and the outliers exceeding the mean ± 3 times the standard deviation were eliminated to calculate the content of each pesticide residue component. Compared with GB2763-2019 "National Food Safety Standard Maximum Residue Limits of Pesticides in Food", it was determined whether the pesticide residue content exceeded the standard. The data was randomly divided into a training set and a test set in a ratio of 8:2, and the prediction model was established using the C5 decision tree algorithm. The optimal feature was selected by the information gain ratio, and the decision tree was recursively constructed until the samples in the training set belonged to the same category or the feature was empty. The model performance was evaluated using the test set, and the pruning threshold was continuously adjusted to optimize the decision tree structure and improve the prediction accuracy. After 10 cross-validations, the model accuracy was stabilized at more than 95%, which can be used for the rapid prediction and evaluation of the pesticide residue content in wolfberry.

[0088] The branch structure and node threshold of the decision tree are adjusted according to the discriminant performance evaluation results to determine the optimal discriminant model parameters.

[0089] The branch structure and node threshold parameters of the decision tree discriminant model are obtained. For the branch structure and threshold parameters, a grid search algorithm is used to traverse different parameter combinations to generate at least one candidate decision tree discriminant model; for the candidate discriminant model, the discriminant accuracy and recall performance indicators on the validation data set are obtained, and a weighted scoring method is used to calculate the performance score of the candidate discriminant model; the candidate discriminant models are sorted according to the performance score, and the candidate discriminant model with the highest performance score is selected as the optimized decision tree discriminant model.

[0090] Specifically, first, by evaluating the discriminant performance of the decision tree on the test set, it is found that there is room for optimization of some branch structures and node threshold parameters. For example, in terms of age attributes, the original decision tree model sets the threshold to 35 years old, but through analysis, it is found that adjusting the threshold to 38 years old can improve the discrimination accuracy. For these branch structures and threshold parameters that need to be optimized, a grid search algorithm is used to set different parameter value ranges, such as the age threshold from 30 to 50 years old, with a step length of 1 year, and the maximum depth from 3 to 10, with a step length of 1, to generate multiple candidate decision tree discriminant models. Then, the performance of each candidate model is evaluated on the validation data set, and indicators such as accuracy, recall, and F1 value are calculated. For example, the accuracy of a candidate model on the validation set is 85%, the recall rate is 80%, and the F1 value is 85%. Then, a weighted scoring method is used to comprehensively consider various indicators and set different weights, such as the accuracy weight is 4, the recall weight is 3, and the F1 value weight is 3, and the performance score of each candidate model is calculated. The candidate models are sorted from high to low according to the performance score, and the model with the highest score is selected as the optimized discriminant model. Finally, the branch structure and node threshold parameters of the optimized model are updated to the original decision tree model to obtain the optimized decision tree discriminant model.

[0091] The optimized discriminant model was applied to new wolfberry samples using the 5-fold cross-validation method to determine the generalization ability of the model.

[0092] New wolfberry sample data to be identified is obtained, the sample data is preprocessed, and key feature parameters are extracted; the sample data is divided into a training set and a test set by a cross-validation method, and a pre-established wolfberry discrimination model is optimized and trained using the training set data to obtain an optimized discrimination model; if the generalization ability of the optimized discrimination model meets the preset requirements, it is applied to the actual wolfberry quality discrimination task to perform quality judgment on the newly collected wolfberry samples.

[0093] Specifically, firstly, according to the pre-established wolfberry discrimination model, the new wolfberry sample data to be discriminated is obtained, the sample data is preprocessed, and key feature parameters such as color, shape, and size are extracted. Then, the 10-fold cross-validation method is used to randomly divide the sample data into 10 subsets, 9 subsets are selected as training sets each time, and the remaining 1 subset is used as a test set. The training set data is used to optimize the support vector machine discrimination model, and the optimal model hyperparameters are obtained through grid search to obtain the optimized discrimination model. Then, the optimized discrimination model is used to discriminate and predict the test set data, obtain the discrimination results of the samples, and calculate the discrimination accuracy of the model on the test set. For 10 different cross-validation data division methods, the above training and testing process is repeated, and 10 cross-validations are performed to obtain 10 discrimination accuracy results. The average discrimination accuracy of 10 cross-validations is calculated as an indicator for evaluating the generalization ability of the optimized discrimination model. If the average accuracy is higher than 90%, it is considered that the model meets the requirements of practical applications. If the generalization ability of the model does not meet the requirements, the original wolfberry discrimination model is optimized according to the samples with incorrect discrimination, such as introducing new features, adjusting the model structure, etc., and the cross-validation process is re-executed. If the generalization ability of the model meets the requirements, the optimized wolfberry discrimination model is applied to the actual wolfberry quality discrimination task. After extracting features from the newly collected wolfberry samples, the trained model is used to automatically determine the quality level, thereby realizing rapid screening and grading of wolfberry quality.

[0094] This embodiment also provides a wolfberry field pesticide residue analysis system based on a decision tree, comprising:

[0095] A data acquisition module is used to obtain a data set of association between spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvest times;

[0096] A preprocessing module, used for preprocessing the associated data set to obtain a characteristic band data set;

[0097] A model building module, used to build a model based on partial least squares regression method, train the model through the characteristic band data set, and obtain a correlation model;

[0098] The discrimination module is used to obtain real-time wolfberry spectrum data, calculate the real-time wolfberry spectrum data based on the association model, generate a calculation result, discriminate the calculation result through a decision tree algorithm, and classify the wolfberries into qualified products and unqualified products.

[0099] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for analyzing pesticide residues in wolfberry fields based on a decision tree, characterized in that: The following steps are involved: Obtain a dataset of the correlation between the spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvest times; Preprocessing the associated data set to obtain a characteristic band data set; Building a model based on partial least squares regression method, training the model with the characteristic band data set to obtain a correlation model; Real-time wolfberry spectrum data is acquired, the real-time wolfberry spectrum data is calculated based on the association model to generate a calculation result, the calculation result is judged by a decision tree algorithm, and the wolfberries are divided into qualified products and unqualified products.

2. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 1, characterized in that: The process of obtaining a data set of association between spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvesting times includes: Near-infrared spectrometer was used to obtain the spectral information of wolfberry samples of different wolfberry varieties, origins and harvest times; The pesticide residue content on the surface of wolfberry was determined by HPLC; A correlation data set between spectral features and pesticide residue content is established based on the spectral information of the wolfberry sample and the pesticide residue content on the surface of the wolfberry.

3. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 1, characterized in that: The process of preprocessing the associated data set to obtain a characteristic band data set includes: dividing the associated data set into an initial spectral data set and a pesticide residue content data set; After performing denoising and normalization preprocessing on the initial spectral data set, a spectral data matrix is ​​constructed; Performing principal component analysis on the spectral data matrix, and obtaining the cumulative contribution rate of the principal components of the data matrix through eigenvalue decomposition and eigenvector extraction; According to the cumulative contribution rate of the principal component, a new characteristic band is determined to obtain spectral characteristic data after dimension reduction; The spectral feature data after dimension reduction is combined with the pesticide residue content data set to obtain the characteristic band data set.

4. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 3, characterized in that: The process of obtaining the cumulative contribution rate of the principal components of the data matrix includes: Performing principal component analysis on the spectral data matrix to obtain the contribution rate of each principal component; According to the contribution rate of each principal component, a cumulative contribution rate curve is drawn, and the inflection point position of the curve is determined to complete the acquisition of the cumulative contribution rate of the principal component.

5. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 1, characterized in that: The process of constructing a model based on the partial least squares regression method and training the model through the characteristic band data set to obtain the correlation model includes: A prediction model is established by using a partial least squares regression algorithm, taking the characteristic band as an independent variable and the pesticide residue content as a dependent variable; The prediction model is trained based on the characteristic band data set to obtain the association model.

6. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 5, characterized in that: The process of training the prediction model based on the characteristic band data set to obtain the association model includes: Determining the optimal number of principal components of the prediction model by a cross-validation method; The prediction model is used to predict the pesticide residue in the characteristic band data set, and the predicted value is compared with the measured value to evaluate the prediction accuracy of the prediction model; if the prediction accuracy does not reach the preset threshold, the model parameters are adjusted and the quantitative relationship model is re-established until the prediction accuracy meets the requirements.

7. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 6, characterized in that: The process of determining the optimal number of principal components of the prediction model by the cross-validation method includes: Using a K-fold cross-validation method, the preprocessed data set is divided into K subsets; For each fold of cross-validation, a partial least squares regression model is constructed using the corresponding training set; Analyze the results of the K-fold cross validation to determine the optimal number of principal components; According to the optimal number of principal components, a partial least squares regression model is trained on the preprocessed data set to obtain the prediction model.

8. The method for analyzing pesticide residues in wolfberry fields based on decision trees according to claim 1, characterized in that: The process of judging the calculation results by a decision tree algorithm and classifying wolfberries into qualified products and unqualified products includes: constructing a decision tree algorithm, inputting the calculation results into the decision tree algorithm, and performing qualitative judgment through the branch structure and node threshold of the decision tree. If the pesticide residue content is lower than a preset threshold, the wolfberry sample is judged to be a qualified product, otherwise it is judged to be an unqualified product.

9. The analysis system of the method for analyzing pesticide residues in wolfberry fields based on a decision tree according to any one of claims 1 to 8, characterized in that: include: A data acquisition module is used to obtain a data set of association between spectral characteristics and pesticide residue content of wolfberry samples of different wolfberry varieties, origins and harvest times; A preprocessing module, used for preprocessing the associated data set to obtain a characteristic band data set; A model building module, used to build a model based on partial least squares regression method, train the model through the characteristic band data set, and obtain a correlation model; The discrimination module is used to obtain real-time wolfberry spectrum data, calculate the real-time wolfberry spectrum data based on the association model, generate a calculation result, discriminate the calculation result through a decision tree algorithm, and classify the wolfberries into qualified products and unqualified products.

Images