A collaborative prediction method for sugar content of multiple pear varieties based on characteristic band selection

Abstract

The present invention provides a collaborative prediction method for the sugar content of multiple varieties of pears based on characteristic band selection, which belongs to the field of spectral analysis. The present invention collects spectral data of multiple varieties of pears as training samples, uses the training samples to train a 1D Grad‑CAM model, obtains the Grad‑CAM value L of each spectral band, extracts the common characteristic bands of pears with the same skin color according to the Grad‑CAM value L, and constructs a sugar content prediction model for multiple varieties of pears, which reduces the amount of data and the amount of calculation, improves the versatility, robustness and prediction accuracy of multi-variety prediction, saves resources and time to a great extent, and has broad application prospects. The present invention solves the problem that it is impossible to construct the same sugar content prediction model for different varieties of pears under the influence of internal and external characteristic changes, spectral reflection intensity and characteristic peak changes between different varieties of pears, reduces resource and time consumption, and improves the versatility of the model.

Description

Technical Field

[0001] The invention belongs to the field of spectral analysis, and in particular relates to a collaborative prediction method for the sugar content of multiple varieties of pears based on characteristic band selection. Background Art

[0002] The development of chemometrics, coupled with the introduction of modern optics and computer data processing technology, has promoted the application of spectral analysis technology in the fields of agriculture and industry. It has been widely used in the qualitative and quantitative analysis of near-infrared spectroscopy, Raman spectroscopy, X-ray and other spectra. In recent years, near-infrared spectroscopy has developed rapidly in the non-destructive detection of agricultural product quality. Due to the differences in the material structure composition between different fruits, the developed models cannot be used universally among multiple varieties, resulting in the need to develop different models for different varieties of fruits, which consumes a lot of resources and time, and current research has not yet proposed an effective solution. Therefore, it is very necessary to develop an effective multi-variety prediction model and realize multi-variety collaborative prediction ("one type for multiple uses"). To address this problem, the present invention designs a multi-variety pear sugar content collaborative prediction method based on characteristic band selection. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides a method for collaborative prediction of the sugar content of multiple varieties of pears based on characteristic band selection, which realizes the multi-variety collaborative prediction function and solves the problem that it is impossible to construct the same sugar content prediction model for different varieties of pears under the influence of changes in internal and external characteristics, spectral reflectance intensity and characteristic peak changes among different varieties of pears.

[0004] The present invention achieves the above technical objectives through the following technical means.

[0005] A collaborative prediction method for the sugar content of multiple varieties of pears based on characteristic band selection includes the following processes:

[0006] Step 1: Build a single variety convolutional neural network model, train weight parameters, optimize hyperparameters, form a single variety sugar content prediction one-dimensional convolutional neural network model, input the test set, and output the optimal sugar content prediction value;

[0007] Step 2: Based on the sugar content prediction value output in step 1, calculate the Grad-CAM value L of the spectral data of different varieties of pears, that is, L Grad-CAM , according to different L Grad-CAM Create Grad-CAM heatmaps of different spectra;

[0008] Step 3: Follow the specific L Grad-CAMrange, select the spectral feature bands that meet the requirements in the original spectrum, and build a collaborative prediction model for the sugar content of pear varieties with similar skin color based on 1D Grad-CAM spectral feature extraction, that is, a multi-variety convolutional neural network model; then train the weight parameters of the multi-variety convolutional neural network model, optimize the hyperparameters of the multi-variety convolutional neural network model, save the optimal multi-variety convolutional neural network model, and input the test set of each variety of pear respectively, and output the predicted values ​​of the sugar content of different varieties of pears.

[0009] Furthermore, the specific process of step 2 is as follows:

[0010] Step 2.1: First, use the sugar content prediction value of the output layer of the single variety convolutional neural network model in step 1 to perform partial derivative operations on all feature maps A output by the last convolutional layer to obtain a set of weights. The weight calculation formula is as follows:

[0011]

[0012] In the formula, α k represents the weight of the kth channel of feature map A; k represents the kth channel in feature map A; z represents the width multiplied by the height of the feature map; i represents the 1D spectral matrix coordinate; θ represents the sign of the partial derivative; y represents the sugar content prediction value; Represents the data of feature map A at position i in channel k;

[0013] Next, the feature map value and the corresponding weight are used for weighted summation, the weighted result is processed using the ReLU function, and the processed result is normalized to the maximum and minimum, and finally the output is L Grad-CAM Value, L Grad-CAM The calculation formula is as follows:

[0014]

[0015] In the formula, A k Represents the data of the kth channel in the feature map A; n represents the number of feature map channels;

[0016] Step 2.2: Use the third-order nonlinear function to calculate the L obtained in step 2.1 Grad-CAM Interpolate and convert L Grad-CAM Expanded to the size of the original spectral data;

[0017] Step 2.3: Calculate the L of all spectra of each pear after amplification Grad-CAM And take the average value, and then perform maximum and minimum normalization on the average value, so that L Grad-CAM Finally, in the range of 0 to 1, the Grad-CAM heat map of different spectra is drawn.

[0018] Furthermore, in step 3, the specific LGrad-CAM The range is selected according to actual conditions, not less than m, and m≥0.75.

[0019] Furthermore, in step 3, the classification basis of fruits with similar skin colors is the skin color, spectral shape, and clustering of t-SNE visualization results.

[0020] Furthermore, the single-variety convolutional neural network model includes 1 input layer, 3 convolutional layers, 1 stretching layer, 1 fully connected layer, and 1 output layer. The multi-variety convolutional neural network model has the same structure as the single-variety convolutional neural network model but different hyperparameters; both the single-variety convolutional neural network model and the multi-variety convolutional neural network model use the RMSprop algorithm to train weight parameters.

[0021] The present invention has the following beneficial effects:

[0022] Compared with the traditional method that can only model a single variety and the model established in one variety cannot be applied to another variety, the present invention selects feature bands through gradient-weighted class activation mapping (Grad-CAM) to construct a collaborative prediction model for the sugar content of multiple varieties of pears with similar skin colors, extracts the common characteristics of pears with similar skin colors, reduces the amount of data, and improves the versatility, robustness and prediction accuracy of multiple varieties. It saves resources and time to a great extent and has broad application prospects. The present invention solves the problem that it is impossible to build the same sugar content prediction model for different varieties of pears under the influence of changes in internal and external characteristics, spectral reflectance intensity and characteristic peak changes between different varieties of pears, and brings better effects on reducing resource and time consumption and improving model versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the convolutional neural network model of the present invention;

[0024] Figure 2 This is a flow chart of the collaborative prediction method for the sugar content of multiple varieties of pears based on characteristic band selection according to the present invention;

[0025] Figure 3 Schematic diagram of 1D Grad-CAM spectral feature extraction according to the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0027] Embodiment 1:

[0028] The collaborative prediction method for the sugar content of multiple varieties of pears based on characteristic band selection of the present invention is as follows Figure 2 As shown, the process includes the following:

[0029] Step 1: Construct a single variety convolutional neural network model, which includes an input layer, a convolutional layer, a stretching layer, a fully connected layer, and an output layer; then input the training set spectral data of different varieties of pears and the known soluble solid content (SSC) values ​​into the single variety convolutional neural network model;

[0030] The root mean square propagation (RMSprop) algorithm is used to train the weight parameters of the single variety convolutional neural network model, optimize the hyperparameters of the single variety convolutional neural network model, save the optimal weight parameters of each single variety convolutional neural network model, form a single variety sugar content prediction one-dimensional convolutional neural network model, input the test set into the trained single variety sugar content prediction one-dimensional convolutional neural network model, and output the optimal prediction result.

[0031] Step 2: Calculate the Grad-CAM value L (i.e. L Grad-CAM ), according to different L Grad-CAM Create a visual heat map of different spectral data, namely the Grad-CAM heat map;

[0032] Step 2.1: Reference Figure 3 First, calculate the score gradient of the sugar content prediction value obtained in step 1, that is, use the sugar content prediction value of the output layer of the model in step 1 to perform partial derivative operations on all feature maps A output by the last convolutional layer to obtain a set of weights. The weight calculation formula is as follows:

[0033]

[0034] In the formula, α k represents the weight of the kth channel of feature map A; k represents the kth channel in feature map A; z represents the width multiplied by the height of the feature map; i represents the 1D spectral matrix coordinate; represents the sign of partial derivative; y represents the predicted value of sugar content; Represents the data of feature map A at position i in channel k; Figure 3 In the k-1 ,ω k Respectively represent the data in the 1st, 2nd, k-1, and kth channels of feature map A;

[0035] Next, the feature map value and the corresponding weight are used for weighted summation, the weighted result is processed using the ReLU function, and the processed result is normalized to the maximum and minimum, and finally an exact number is output, namely LGrad-CAM Value, L Grad-CAM The calculation formula is as follows:

[0036]

[0037] In the formula, A k Represents the data of the kth channel in the feature map A; n represents the number of feature map channels;

[0038] Step 2.2: For L obtained in step 2.1 Grad-CAM Upsampling is performed by using a third-order nonlinear function to Grad-CAM Interpolate and convert L Grad-CAM Zoom in to the size of the original spectral data;

[0039] Step 2.3: Calculate L for all spectra of each pear after amplification Grad-CAM And take the average value; then perform maximum and minimum normalization on the average value, the purpose is to make L Grad-CAM Finally, in the range of 0 to 1, the Grad-CAM thermograms of different varieties of pears are drawn, that is, the Grad-CAM thermograms of different spectra.

[0040] Step 3: Follow the specific L Grad-CAM Range, select the spectral feature bands that meet the requirements in the original spectrum, and build a collaborative prediction model for the sugar content of pear varieties with similar skin color (i.e., a multi-variety convolutional neural network model) based on spectral feature extraction of one-dimensional gradient weighted class activation mapping (1D Grad-CAM);

[0041] The RMSprop algorithm was used to train the weight parameters and optimize the hyperparameters. The optimal weight parameters of the multi-variety convolutional neural network model were saved, and the test set of each variety was input into the trained multi-variety convolutional neural network model. The prediction results of the multi-variety convolutional neural network model were output to obtain the predicted sugar content corresponding to each input spectral curve, that is, the predicted sugar content of different varieties of pears.

[0042] Among them, the specific L Grad-CAM The range is selected according to the actual situation. The larger the value, the more important the extracted spectral band is, but it also makes the amount of data smaller. In this embodiment, L is selected Grad-CAM The range is not less than 0.9;

[0043] Among them, the classification basis of similar skin color is peel color, spectral shape, and clustering of t-distributed stochastic neighbor embedding (t-SNE, t-SNE) visualization results.

[0044] In practical applications, the present invention can be used not only for predicting the sugar content of different varieties, but also for predicting acidity, hardness and other aspects.

[0045] Embodiment 2:

[0046] This embodiment is applied to the quantitative analysis of visible\near infrared spectra to predict the sugar content values ​​of multiple varieties of pears. The selected data is a reflectance spectrum collected by a self-designed online system, including spectra of three varieties of pears, Fengshui, Qiushui and Ruoguang. The sample pears come from the Jinling Crisp Pear Garden in Jiangning District. A single sample collects three spectral data, which are divided into the calyx, equatorial part and pedicel. The training sets are 189, 193, and 196 respectively, and the test sets are 48, 49, and 49 respectively. A hyperspectral imaging spectrometer (SOC710VP, Surface Optics Corporation, USA) is used to collect spectral data, and the band range of the spectral data is selected from 400 to 800nm. Take the pulp with a spectral detection point position, width, and depth of about 1cm, and use a saccharimeter (ATAGO PAL-1, Tokyo, Japan) to determine the soluble solids (SSC). The minimum SSC contents of Fengshui pear, Qiushui pear and Ruoguang pear are 12.0, 10.4 and 13.0°Bx, the maximum SSC contents are 20.9, 20.9 and 25.0°Bx, the average SSC contents are 15.92, 14.82 and 15.77°Bx, and the standard deviations are 1.77, 1.46 and 1.40°Bx.

[0047] The collaborative prediction method of sugar content of multiple varieties of pears based on characteristic band selection is as follows:

[0048] Step 1: Build Figure 1 The single variety convolutional neural network model shown includes 1 input layer, 3 convolutional layers, 1 stretching layer, 1 fully connected layer, and 1 output layer; then the training set spectral data of the three varieties of pears, Fengshui, Qiushui, and Ruoguang, and the known SSC values ​​are input into the single variety convolutional neural network model; in this embodiment, the dimension of the input spectral data is 307;

[0049] The RMSprop algorithm was used to train the weight parameters of the single-variety convolutional neural network model (the weight parameters are the weights of the connections between model layers), and the hyperparameters of the single-variety convolutional neural network model were optimized. Convolutional layer 1 includes 7 convolution kernels with a step size of 5, convolutional layer 2 includes 5 convolution kernels with a step size of 3, and convolutional layer 3 includes 3 convolution kernels with a step size of 1. The number of neurons in the fully connected layer is 128, and the number of neurons in the output layer is 1; the random inactivation rate is 0.2, and the hyperparameter optimization results in network training are that the model batch sizes corresponding to the three varieties of Fengshui, Qiushui, and Ruoguang are 15, 15, and 16 respectively, the learning rates are 0.02, 0.0035, and 0.02 respectively, the regularization coefficients are all 0.001, and the number of model training rounds is 3000 rounds;

[0050] The optimal weight parameters of each single variety convolutional neural network model are saved to form a one-dimensional convolutional neural network model for predicting the sugar content of a single variety. The test set is input into the trained one-dimensional convolutional neural network model for predicting the sugar content of a single variety, and the prediction results of the model on the test set are output.

[0051] Step 2: Reference Figure 3 , create Grad-CAM heat maps of spectra of different varieties;

[0052] Step 2.1: First, use the sugar content prediction value of the output layer of the single variety convolutional neural network model to perform partial derivative operations on all feature maps A output by the last convolutional layer, that is, perform global average pooling on the gradient information of all channels to obtain a set of weights; then use the feature map value and the corresponding weight to perform weighted summation, and perform ReLU processing on the weighted result, and finally perform maximum and minimum normalization on the result to obtain L Grad-CAM ;

[0053] Step 2.2: For the obtained L Grad-CAM Upsample and interpolate using a third-order nonlinear function to convert L Grad-CAM Enlarge the original spectrum data to a size of 1×307;

[0054] Step 2.3: For each variety, all L Grad-CAM Take the average value to indicate the Grad-CAM value corresponding to the average spectrum of the variety, and then perform maximum and minimum normalization to fix the Grad-CAM value in the range of 0 to 1, which is convenient for drawing the heat map and according to L Grad-CAM Select the characteristic bands and finally draw the spectral band feature extraction map of each variety, namely the Grad-CAM heat map.

[0055] Step 3: Select L Grad-CAMThe characteristic bands of the spectra (including training set and test set) with a value of not less than 0.9 are used to select important bands in the sugar content prediction process and reduce the amount of data; the characteristic bands of each variety are taken as a union instead of an intersection, in order to extract the common characteristics of pears with similar skin colors while avoiding the loss of spectra containing important information in each variety due to the intersection;

[0056] Construct a multi-variety convolutional neural network model based on 1D Grad-CAM spectral feature extraction, the structure is as follows Figure 1 As shown;

[0057] The extracted characteristic bands of Fengshui, Qiushui and Ruoguang and the known SSC values ​​were input into the multi-variety convolutional neural network model. The multi-variety convolutional neural network model used the RMSprop algorithm to train the weight parameters and optimize the hyperparameters. The number of neurons in the fully connected layer of the multi-variety convolutional neural network model was 32, the number of neurons in the output layer was 1, and the random inactivation rate was 0.5. The hyperparameter optimization results in the network training were as follows: the model batch size was 15, the learning rate was 0.07, the regularization coefficient was 0.0077, the number of model training rounds was 3000, and the other parameters were the same as the single-variety convolutional neural network model.

[0058] Save the optimal weight parameters of the multi-variety convolutional neural network model, input the test set of each variety into the trained model, and output the prediction results of the model on the test set;

[0059] The root mean square error of prediction (RMSEP) of the test sets of Fengshui pear, Qiushui pear and Ruoguang pear in the multi-variety convolutional neural network model based on 1D Grad-CAM spectral feature extraction are 1.60, 0.92, and 1.05°Bx, respectively, and the RMSEP in the single-variety convolutional neural network model are 1.54, 0.91, and 1.00°Bx, respectively; by comparison, it can be seen that the RMSEP of the multi-variety pear sugar content collaborative prediction model based on 1D Grad-CAM spectral feature extraction and the single-variety convolutional neural network model is within 6%, and the test accuracy is still high. Compared with the single-variety model, the multi-variety pear sugar content collaborative prediction model based on 1D Grad-CAM spectral feature extraction has a simpler structure and less data volume, and one model is applicable to the sugar content prediction of three varieties of pears. The model has wider versatility and better robustness, which reflects the good generalization ability of the present invention.

[0060] Embodiment 3:

[0061] Comparative Example:

[0062] The prediction process of the multi-variety pear sugar content prediction model for full-band spectral data is as follows:

[0063] 1) Input the training set data into the multi-variety pear sugar content prediction model;

[0064] 2) The multi-variety pear sugar content prediction model uses the RMSprop algorithm to train the model weight parameters and optimize the hyperparameters of the neural network model. The number of neurons in the fully connected layer is 128, the number of neurons in the output layer is 1; the random dropout rate is 0.2. The hyperparameter optimization results in network training are: the model batch size is 15, the learning rate is 0.02, the regularization coefficient is 0.001, and the number of model training rounds is 3000; other parameters are the same as the single variety convolutional neural network model;

[0065] 3) Save the optimal weight parameters of the multi-variety convolutional neural network model, input the full-band test set of each variety into the trained model, and output the prediction results of the model on the test set;

[0066] 4) The RMSEP of the full-band test sets of Fengshui, Qiushui and Ruoguang in the multi-variety convolutional neural network model are 1.68, 0.98 and 1.08°Bx respectively. By comparing with 1.60, 0.92 and 1.05°Bx, it can be seen that the accuracy of the method of the present invention in the coordinated prediction model of multi-variety pears is better than the prediction accuracy of the full-band spectrum.

[0067] The specific L Grad-CAM The range is determined according to the actual and modeling results, and is generally not less than m, and the value of m is not less than 0.75. In Examples 1 and 2, m is preferably set to 0.9. The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above implementations. Without departing from the essence of the present invention, any obvious improvements, substitutions or modifications that can be made by those skilled in the art belong to the protection scope of the present invention.

Claims

1. A collaborative prediction method for the sugar content of multiple varieties of pears based on characteristic band selection, characterized in that: The process includes the following: Step 1: Build a single variety convolutional neural network model, train weight parameters, optimize hyperparameters, form a single variety sugar content prediction one-dimensional convolutional neural network model, input the test set, and output the optimal sugar content prediction value; Step 2: Based on the sugar content prediction value output in step 1, calculate the Gradient-weighted Class Activation Mapping (Grad-CAM) value L of the spectral data of different varieties of pears, that is, L Grad-CAM , according to different L Grad-CAM Create Grad-CAM heatmaps of different spectra; Step 3: Follow the specific L Grad-CAM range, select the spectral feature bands that meet the requirements in the original spectrum, and construct a collaborative prediction model for the sugar content of pear varieties with similar skin color based on spectral feature extraction of one-dimensional gradient weighted class activation mapping (1D Grad-CAM), that is, a multi-variety convolutional neural network model; train the weight parameters of the multi-variety convolutional neural network model, optimize its hyperparameters, save the optimal multi-variety convolutional neural network model, input the test set of each variety of pear respectively, and output the predicted values ​​of the sugar content of different varieties of pears; The specific process of step 2 is as follows: Step 2.1: First, use the sugar content prediction value of the output layer of the single variety convolutional neural network model in step 1 to perform partial derivative operations on all feature maps A output by the last convolutional layer to obtain a set of weights. The weight calculation formula is as follows: In the formula, α k represents the weight of the kth channel of feature map A; k represents the kth channel in feature map A; z represents the width multiplied by the height of the feature map; i represents the 1D spectral matrix coordinate; represents the sign of partial derivative; y represents the predicted value of sugar content; Represents the data of feature map A at position i in channel k; Next, the feature map value and the corresponding weight are used for weighted summation, the weighted result is processed using the ReLU function, and the processed result is normalized to the maximum and minimum, and finally the output is L Grad-CAM Value, L Grad-CAM The calculation formula is as follows: In the formula, A k Represents the data of the kth channel in the feature map A; n represents the number of feature map channels; Step 2.2: Use the third-order nonlinear function to calculate the L obtained in step 2.1 Grad-CAM Interpolate and convert L Grad-CAM Expanded to the size of the original spectral data; Step 2.3: Calculate the L of all spectra of each pear after amplification Grad-CAM And take the average value, and then perform maximum and minimum normalization on the average value, so that L Grad-CAM Finally, in the range of 0 to 1, the Grad-CAM heat map of different spectra is drawn.

2. The collaborative prediction method for sugar content of multiple varieties of pears based on characteristic band selection according to claim 1 is characterized in that: In step 3, the specific L Grad-CAM The range is selected according to actual conditions, not less than m, and m≥0.

75.

3. The collaborative prediction method for sugar content of multiple varieties of pears based on characteristic band selection according to claim 1, characterized in that: In the step 3, the classification basis of fruits with similar skin colors is the skin color, spectral shape, and clustering of t-distributed stochastic neighbor embedding (t-SNE, t-SNE) visualization results.

4. The collaborative prediction method for sugar content of multiple varieties of pears based on characteristic band selection according to claim 1, characterized in that: The single-variety convolutional neural network model includes 1 input layer, 3 convolutional layers, 1 stretching layer, 1 fully connected layer, and 1 output layer. The multi-variety convolutional neural network model has the same structure as the single-variety convolutional neural network model but different hyperparameters. Both the single-variety convolutional neural network model and the multi-variety convolutional neural network model use the root mean square propagation (RMSprop) algorithm to train weight parameters.

Images