Method, device and system for predicting starch content in potato quality, and storage medium

Abstract

The invention discloses a potato quality starch content prediction method, device and system, and a storage medium. The method comprises the following steps: acquiring potato test data and preprocessing; performing feature standardization processing on the preprocessed data; segmenting the standardized data set into a training set and a test set; and training the random forest regression model by using the training set, and screening out features having significant influence on starch content prediction through feature importance analysis. By adopting the technical scheme, the starch content of the potatoes can be accurately predicted.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a method and device, a system and a storage medium for predicting the starch content of potato quality. Background Art

[0002] As one of the most important food crops in the world, the quality of potato directly affects starch production, food processing and market value. Starch content is one of the important indicators to measure potato quality, and has a great impact on potato planting, harvesting, processing and other links. Traditionally, the determination of potato starch content relies on chemical analysis or physical detection methods. Although these methods are accurate, they are time-consuming, labor-intensive, and costly, and they are difficult to meet the needs of large-scale planting and rapid quality assessment. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device, system and storage medium for predicting the starch content of potato quality, so as to achieve efficient and accurate prediction of the starch content of potato and provide a scientific basis for potato planting, quality management and other aspects.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A method for predicting potato quality starch content, comprising:

[0006] Obtain potato trial data and pre-process them;

[0007] Perform feature standardization on the preprocessed data;

[0008] Split the standardized data set into training set and test set;

[0009] The random forest regression model was trained using the training set, and the features that had a significant impact on the prediction of starch content were screened out through feature importance analysis.

[0010] Preferably, preprocessing includes: data cleaning, screening and formatting.

[0011] Preferably, the random forest regression model is trained using the training set, including:

[0012] Resample the data set with sample size equal to N and get the bootstrap sample

[0013] Based on bootstrap samples Building a regression tree The decision tree grows from the root node as follows until the pre-pruning parameters of the tree are met:

[0014] 1) Randomly select a random subset of m eigenvalues ​​from the p eigenvalues ​​Θ b , Or m = [log2p], [] means rounding;

[0015] 2) From θ b Determine the current "best" grouping variable, group and generate two child nodes;

[0016] The output includes B-trees Random forests;

[0017] The output regression prediction result is

[0018] The present invention also provides a device for predicting potato quality starch content, comprising:

[0019] An acquisition module is used to acquire potato test data and perform preprocessing;

[0020] A standardization processing module is used to perform feature standardization processing on the preprocessed data;

[0021] A partitioning module is used to split the standardized data set into a training set and a test set;

[0022] The prediction module is used to train the random forest regression model using the training set and screen out features that have a significant impact on starch content prediction through feature importance analysis.

[0023] Preferably, preprocessing includes: data cleaning, screening and formatting.

[0024] Preferably, the prediction module comprises:

[0025] The first processing unit is used to perform resampling bootstrapping on a data set with a sample size equal to N to obtain a bootstrap sample

[0026] The second processing unit is used to Building a regression tree The decision tree grows from the root node as follows until the pre-pruning parameters of the tree are met:

[0027] 1) Randomly select a random subset of m eigenvalues ​​from the p eigenvalues ​​Θ b , Or m = [log2p], [] means rounding;

[0028] 2) From θ b Determine the current "best" grouping variable, group and generate two child nodes;

[0029] The third processing unit is used to output a B-tree Random forests;

[0030] The fourth processing unit is used to output the regression prediction result as

[0031] The embodiment of the present invention further provides a potato quality starch content prediction system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a potato quality starch content prediction method when executed by the processor.

[0032] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is run, the method for predicting starch content in potato quality is executed.

[0033] The invention obtains and pre-processes data from potato test data, including data cleaning, screening and formatting; standardizes features to eliminate dimensional differences between different features and improve the stability and accuracy of the model; divides the data set into a training set and a test set for model training and verification; uses a random forest regression model for training, and screens out features that have a significant impact on starch content prediction through feature importance analysis; in addition, the invention also intuitively displays the impact of two important features on potato starch content by drawing a 3D regression surface graph; finally, by adjusting model parameters, such as the number of trees, the maximum depth, etc., the model performance is optimized, and the 1-R 2 The random forest regression model of the present invention can accurately predict the starch content of potatoes, provides strong technical support for potato quality evaluation and breeding, and has important application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 Schematic diagram of the process of predicting potato quality starch content of the present invention;

[0036] Figure 2 A flow chart for constructing a random forest regression model according to an embodiment of the present invention;

[0037] Figure 3 This is a feature importance analysis diagram of an embodiment of the present invention;

[0038] Figure 4 3D surface graphs of the linear model and random forest regression model of the embodiment of the present invention;

[0039] Figure 5 This is the error graph (OOB) of the random forest regression test of the embodiment of the present invention;

[0040] Figure 6 This is a random forest regression error diagram (1-R 2 );

[0041] Figure 7 This is a random forest regression error graph (RMSE) of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Embodiment 1:

[0045] like Figure 1 As shown, an embodiment of the present invention provides a method for predicting starch content in potato quality, comprising:

[0046] S1. Data acquisition and preprocessing: obtain raw data from potato trial data, and perform data cleaning, screening and formatting;

[0047] S2, feature standardization processing: perform feature standardization processing on the preprocessed data;

[0048] S3, data set segmentation: the standardized data set is segmented into a training set and a test set, where the training set is used for model training and the test set is used for model verification;

[0049] S4. Random forest regression model training: Use the training set to train the random forest regression model, build multiple decision trees through multiple iterations, and form an integrated learning model to improve the robustness and accuracy of the prediction;

[0050] S5. Feature importance analysis: Through feature importance analysis, the features that have a significant impact on the prediction of potato starch content are screened out;

[0051] S6. Regression surface plot: Based on the selected important features, a 3D regression surface plot is plotted to intuitively show the effects of two important features on potato starch content.

[0052] S7. Model parameter optimization and performance evaluation: Adjust the parameters of the random forest regression model to optimize model performance; calculate and evaluate the model's predictive performance indicators.

[0053] As an implementation method of an embodiment of the present invention, in step S1, after inputting potato test data (as shown in Table 1), the test data is cleaned, screened and formatted. During the cleaning process, missing values, outliers, etc. in the data are checked and processed to ensure the accuracy and completeness of the data; during the screening process, irrelevant or redundant information is removed to reduce the complexity of the model.

[0054] Table 1

[0055]

[0056]

[0057] As an implementation of an embodiment of the present invention, in step S2, the feature data is standardized by using the StandardScaler class to eliminate the dimensional differences between different features, thereby improving the stability of the model and the prediction accuracy.

[0058] The normalization formula is:

[0059]

[0060] Where x is the original eigenvalue, μ is the mean of the feature, and σ is the standard deviation of the feature.

[0061] The feature standardized data are shown in Table 2.

[0062] Table 2

[0063]

[0064]

[0065] As an implementation of the embodiment of the present invention, in step S3, the standardized data set is divided into a training set and a test set.

[0066] When using the train_test_split function for splitting, you need to specify the random_state parameter to ensure that the results of each split are consistent, so as to facilitate the comparison of the performance of different models. The test_size parameter specifies the proportion of the test set to the total data set, which is usually set between 0.2 and 0.3 to ensure that the test set is representative enough. The data set split ensures the generalization ability of the model on unseen data and reduces overfitting.

[0067] As an implementation method of an embodiment of the present invention, in step S4, the random forest regression model is trained by a training set, and multiple decision trees are constructed through multiple iterations to form an integrated learning model. Random forest is an integrated learning method based on the resampling bootstrap method. It uses a random method to establish a forest containing multiple decision trees, where each tree is a basic learner, and the "whole" forest corresponds to integrated learning. The random forest regression model is composed of regression decision trees. The basic principle of the regression decision tree is: since the target value of the regression tree is numerical, its heterogeneity is usually measured by variance. The mathematical definition of the variance of the regression tree node t is

[0068]

[0069] Among them, N t is the number of samples of node t, y i (t) is the sample observation X in node t i The target value of node t; y(t) is the average value of the target value in node t.

[0070] If node t is the parent node, when branching, according to the basic principle of determining data grouping during tree growth, find the left child node t that can make t left The variance S 2 (t left ) and the right child node t teft The variance S 2 (t right ) The grouping variables and grouping limits when the minimum value is taken. From the parent node to the child node, the heterogeneity of the target value decreases to

[0071]

[0072] where the “best” grouping variable and group limits are ΔS 2 (t) Take the maximum eigenvalue and group limit.

[0073] Compared with other regression models, the regression tree constructed by the above division principle has a "natural" high variance feature, that is, the model parameters (grouping variables and group limits, etc.) will change greatly with the change of the training set, resulting in large differences in the predicted values ​​of the sample observation X0 of each model, showing high variance of the prediction. The general method to solve the problem of high variance in prediction is ensemble learning, which can give predictions with better prediction performance. The regression tree constructs a random forest regression model with better prediction performance through the ensemble learning method. To train the random forest regression model, it is necessary to perform b = 1, 2, ..., B iterations as follows to obtain a random forest including B trees.

[0074] like Figure 2 As shown, training a random forest regression model involves:

[0075] The first step is to resample the data set with a sample size equal to N and obtain the bootstrap sample

[0076] The second step is based on the bootstrap sample Building a regression tree The decision tree grows from the root node as follows until the pre-pruning parameters of the tree are met:

[0077] 1. Randomly select a random subset of m eigenvalues ​​from the p eigenvalues ​​Θ b , Or m = [log2p], [] means rounding;

[0078] 2. From θ b Determine the current "best" grouping variable, group and generate two child nodes;

[0079] The third step is to output B trees. Random forests;

[0080] The fourth step is prediction. The regression prediction result is

[0081] As an implementation of an embodiment of the present invention, in step S5, the feature importance analysis is: if a certain feature value plays an important role in the prediction of the target value, then adding random noise to the feature value of the model OOB will significantly affect the calculation result of the model OOB.

[0082] First of all, Calculate the OOB-based test error, denoted as

[0083] Then, to measure the importance of the jth eigenvalue to the target value, the following calculation is performed:

[0084] 1. Random shuffle The order of OOB values ​​at the jth eigenvalue is recalculated The OOB-based error is denoted as

[0085] 2. Calculate the jth eigenvalue before and after adding noise The OOB error changes:

[0086]

[0087] Repeat the above steps B times to get B Calculate the mean It is the change in the OOB error of the random forest sum caused by adding noise to the jth eigenvalue. The larger the change, the more important the jth eigenvalue. The feature importance analysis is as follows: Figure 3 shown.

[0088] As an implementation method of an embodiment of the present invention, in step S6, in order to intuitively demonstrate the prediction performance of the model under different feature combinations. First, two key features are selected according to the training data, and a feature grid covering each area of ​​the feature space is generated. Then, each point on the feature grid is predicted using the trained linear regression model and the random forest regression model to obtain the corresponding starch content prediction value. Then, two subgraphs are drawn using the Matplotlib library to respectively display the regression surfaces of the linear regression model (as a comparison) and the random forest regression model. During the drawing process, the plot_wireframe function is used to draw the grid lines, the plot_surface function is used to draw the regression surface, and the scatter function is used to draw the original data points, such as Figure 4 In this way, the prediction performance of the two models can be compared intuitively, and visual support can be provided for the prediction of potato quality starch content.

[0089] As an implementation method of an embodiment of the present invention, in step S7, in the model parameter optimization stage, the key parameters of the random forest regression model, such as the number of trees (n_estimators) and the maximum depth (max_depth), are adjusted. By setting different maximum depth values ​​in a loop, the random forest regression model and the decision tree regression model (for comparison) are trained respectively, and the training error, test error, and cross-validation error of each model are calculated. In order to comprehensively evaluate the model performance, two error indicators are used: 1-R 2 (1-coefficient of determination), which is used to measure the accuracy of model predictions; and RMSE (root mean square error), which is used to measure the deviation between the model prediction value and the true value.

[0090]

[0091] Among them, m is the number of samples, yi is the true value of the sample, is the true mean of the sample, is the sample prediction value.

[0092] In order to more intuitively show the changes in model performance under different parameters, a graph showing the error changes with the number of trees and tree depth is drawn, including training error, test error and cross-validation error. Figure 5 , 6 , 7 for easy comparison and analysis. Finally, by comprehensively comparing the model performance under different parameters, the optimal random forest regression model parameters were selected to ensure the accuracy and reliability of the model in practical applications. This step not only improved the prediction accuracy of the model, but also provided strong support for the rapid and accurate prediction of the starch content of potato quality in the subsequent period.

[0093] Embodiment 2:

[0094] The embodiment of the present invention also provides a device for predicting potato quality starch content, comprising:

[0095] An acquisition module is used to acquire potato test data and perform preprocessing;

[0096] A standardization processing module is used to perform feature standardization processing on the preprocessed data;

[0097] A partitioning module is used to split the standardized data set into a training set and a test set;

[0098] The prediction module is used to train the random forest regression model using the training set and screen out features that have a significant impact on starch content prediction through feature importance analysis.

[0099] As an implementation method of the embodiment of the present invention, preprocessing includes: data cleaning, screening and formatting.

[0100] As an implementation of an embodiment of the present invention, the prediction module includes:

[0101] The first processing unit is used to perform resampling bootstrapping on a data set with a sample size equal to N to obtain a bootstrap sample S b * ;

[0102] The second processing unit is used to Building a regression tree The decision tree grows from the root node as follows until the pre-pruning parameters of the tree are met:

[0103] 1) Randomly select a random subset of m eigenvalues ​​from the p eigenvalues ​​Θ b , Or m = [log2p], [] means rounding;

[0104] 2) From θ b Determine the current "best" grouping variable, group and generate two child nodes;

[0105] The third processing unit is used to output a B-tree Random forests;

[0106] The fourth processing unit is used to output the regression prediction result as

[0107] Embodiment 3:

[0108] The embodiment of the present invention further provides a potato quality starch content prediction system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a potato quality starch content prediction method when executed by the processor.

[0109] Embodiment 4:

[0110] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the computer program is run, the method for predicting starch content in potato quality is executed.

[0111] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for predicting potato quality starch content, characterized in that: include: Obtain potato trial data and pre-process them; Perform feature standardization on the preprocessed data; Split the standardized data set into training set and test set; The random forest regression model was trained using the training set, and the features that had a significant impact on the prediction of starch content were screened out through feature importance analysis.

2. The method for predicting potato quality starch content according to claim 1, wherein: Preprocessing includes: data cleaning, filtering and formatting.

3. The method for predicting potato quality starch content as claimed in claim 2, characterized in that: The random forest regression model is trained using the training set, including: Resample the data set with sample size equal to N and get the bootstrap sample Based on bootstrap samples Building a regression tree The decision tree grows from the root node as follows until the pre-pruning parameters of the tree are met: 1) Randomly select a random subset of m eigenvalues ​​from the p eigenvalues ​​Θ b , Or m = [log2p], [] means rounding; 2) From θ b Determine the current "best" grouping variable, group and generate two child nodes; The output includes B-trees Random forests; The output regression prediction result is 4. A potato quality starch content prediction device, characterized in that: include: An acquisition module is used to acquire potato test data and perform preprocessing; A standardization processing module is used to perform feature standardization processing on the preprocessed data; A partitioning module is used to split the standardized data set into a training set and a test set; The prediction module is used to train the random forest regression model using the training set and screen out features that have a significant impact on starch content prediction through feature importance analysis.

5. The potato quality starch content prediction device according to claim 4, characterized in that: Preprocessing includes: data cleaning, filtering and formatting.

6. The potato quality starch content prediction device according to claim 5, characterized in that: The prediction module includes: The first processing unit is used to perform resampling bootstrapping on a data set with a sample size equal to N to obtain a bootstrap sample The second processing unit is used to Building a regression tree The decision tree grows from the root node as follows until the pre-pruning parameters of the tree are met: 1) Randomly select a random subset of m eigenvalues ​​from the p eigenvalues ​​Θ b , Or m = [log2p], [] means rounding; 2) From θ b Determine the current "best" grouping variable, group and generate two child nodes; The third processing unit is used to output a B-tree Random forests; The fourth processing unit is used to output the regression prediction result as 7. A potato quality starch content prediction system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the method for predicting starch content in potato quality according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that: The storage medium stores a computer program, which, when running, executes the method for predicting potato quality starch content according to any one of claims 1 to 3.

Images