A Method for Transferring Near-Infrared Deep Neural Network Models between Instruments

Abstract

The present invention discloses a method for transferring a near-infrared deep neural network model between instruments, including measuring the near-infrared spectra of a substance to be measured using multiple different models of portable visible near-infrared spectrometers; unifying the wavelength ranges and resolutions of the near-infrared spectra of the multiple different models of portable visible near-infrared spectrometers; preprocessing the near-infrared spectra of the multiple different models of portable visible near-infrared spectrometers; dividing the datasets of the near-infrared spectra of the multiple different models of portable visible near-infrared spectrometers; developing a 1D-Inception-Resnet quantitative network for the main model spectrometer; adjusting the network parameters of the main model spectrometer using data from different slave model spectrometers; validating and calibrating the model using data from different slave model spectrometers; the present invention can correct and transfer the 1D-Inception-Resnet network model of the main instrument to the slave spectrometer, and repair the prediction of the spectra collected by the slave spectrometer.

Description

[0001] The present invention relates to a method for transferring near-infrared deep neural network models between instruments, belonging to the technical field of instrument calibration technology and chemometric model migration. Background Art

[0002] Near-infrared spectroscopy, with its advantages of rapidity, non-destructiveness, and environmental friendliness, plays a vital role in the quantitative detection of chemical components. However, in actual field applications, when spectrometer optical components such as light sources and gratings are replaced due to wear and aging, the optical characteristics of the analyte in the near-infrared spectral region (wavelength or wavenumber range, absorption intensity) can also change. These measurement discrepancies, caused by the instrument's inherent components, can render the near-infrared spectral prediction model of the older instrument inaccurate for quantitatively predicting spectra collected by the newer instrument, necessitating repeated chemical calibration and modeling analysis.

[0003] With the development of big data and artificial intelligence, convolutional neural networks have become one of the most popular deep network frameworks. Researchers have developed a one-dimensional convolutional neural network algorithm to analyze spectral data, capturing more local spectral features that are hidden in the original spectrum and have low recognition accuracy, thereby establishing a more accurate near-infrared spectral model. However, deep neural network models also rely on instrument characteristics and require adjustment before use with new instruments.

[0004] The deep neural network model is adjusted by using the underlying weights of the network model on the old spectrometer as the initial weights for the new instrument's network model. Spectra collected by the new instrument are then used to adjust the global network weights to accommodate the new instrument's predictions, avoiding the extensive sample labeling and retraining of the deep network from scratch. This approach involves retraining the model on the old instrument using data from the new instrument. However, it's important to note that retraining is not the same as retraining from scratch; it involves adjusting the weight parameters of the neural network model on the old instrument. The most prominent drawbacks of retraining are overfitting and gradient explosion or dispersion during training. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a method for calibrating and transferring chemometric models between multiple portable visible-near-infrared spectrometers of different models using a 1D-Inception-Resnet network, so as to solve the problem that near-infrared chemometric models cannot be freely used between various spectrometers of different models.

[0006] The Inception module independently designed by the present invention can extract spectral feature information from multiple computer vision perspectives. The features after splicing and fusion are more conducive to the training of neural networks. At the same time, the module with the Residual structure is equivalent to changing the learning objective of the network. Instead of learning a complete output, the entire network learns the difference between the expected output and the input of the Inception module, which is easier to fit and optimize than the non-linear layer mapping without the residual module, effectively alleviating the problem of gradient explosion or disappearance caused by the deepening of the network depth.

[0007] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0008] A method for transferring a near-infrared deep neural network model between instruments specifically includes the following steps:

[0009] (1) Use multiple different models of portable visible near-infrared spectrometers to measure the near-infrared spectra of the substance to be measured;

[0010] (2) Unify the wavelength ranges and resolutions of the near-infrared spectra of multiple different models of portable visible near-infrared spectrometers;

[0011] (3) Preprocess the near-infrared spectra of multiple different models of portable visible near-infrared spectrometers;

[0012] (4) Divide the data sets of the near-infrared spectra of multiple different models of portable visible near-infrared spectrometers;

[0013] (5) Develop a 1D-Inception-Resnet quantitative network for the main model spectrometer;

[0014] (6) Adjust the parameters of the 1D-Inception-Resnet quantitative network of the main model spectrometer with the data of different slave model spectrometers instead of developing from scratch;

[0015] (7) Use the data of different slave model spectrometers to verify and correct the model.

[0016] A further improvement of the technical solution of the present invention is that in the step (2), 3 times of Hermite interpolation is used to unify the wavelength range and resolution.

[0017] A further improvement of the technical solution of the present invention lies in: in step (3), linear mapping is performed on the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models. First, calculate the standard deviation std of the spectral data of each group of spectrometers of different models, and then set 10 slope seeds: [0.95 0.96 0.97 0.98 0.99 1.01 1.02 1.03 1.04 1.05]. Finally, perform linear mapping: seeds * spectral data + 0.1 * std to complete data augmentation and enhance the robustness of the training model.

[0018] A further improvement of the technical solution of the present invention lies in: in step (3), standard normal variate transformation (SNV) is performed on the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models to reduce baseline drift. The calculation formula is as follows:

[0019]

[0020] where x i,k is the k-th wavelength value of the i-th sample spectrum, x i is the mean of the k-th wavelength value of the i-th sample spectrum, k = 1, 2,..., m, m is the number of wavelength points, i = 1, 2,..., n, n is the number of samples, and X i,SNV is the transformed spectrum.

[0021] A further improvement of the technical solution of the present invention lies in: in step (4), the combined x-y distance SPXY algorithm is used for dataset division, and the near-infrared spectral data of multiple portable visible near-infrared spectrometers of different models are divided into an independent calibration set, a validation set, and a prediction set according to 3:1:1.

[0022] A further improvement of the technical solution of the present invention lies in: in step (5), the overall network framework of the developed 1D-Inception-Resnet is as follows:

[0023] Sequence input layer - Convolution layer - Batch normalization layer - ELU activation function -

[0024] Convolution layer - Batch normalization layer - ELU activation function - (1D-Inception module)

[0025] -(Residual module) - Flatten layer - Fully connected layer - Softplus activation layer

[0026] - Fully connected layer - Softplus activation layer - Fully connected layer - Softplus activation layer

[0027] - Fully connected layer - Regression layer.

[0028] A further improvement of the technical solution of the present invention lies in that: in the step (5), the 1D-Inception module is composed of 4 convolutional paths spliced by a dimensional splicing layer:

[0029] Path 1: Convolutional layer - Batch normalization layer - ELU activation function

[0030] Path 2: Max pooling layer - Convolutional layer - Batch normalization layer - ELU activation function

[0031] Path 3: Convolutional layer - Batch normalization layer - ELU activation function - Convolutional layer

[0032] - Batch normalization layer - ELU activation function

[0033] Path 4: Convolutional layer - Batch normalization layer - ELU activation function - Convolutional layer

[0034] - Batch normalization layer - ELU activation function.

[0035] A further improvement of the technical solution of the present invention lies in that: in the step (5), the Residual module is formed by skip connection of the input and output of the 1D-Inception module through an element-wise addition layer.

[0036] A further improvement of the technical solution of the present invention lies in that: in the step (6), the parameters of the 1D-Inception-Resnet quantitative network of the main model spectrometer are adjusted with data from spectrometers of different models.

[0037] A further improvement of the technical solution of the present invention lies in that: in the step (7), the evaluation indexes for verifying the model calibration transfer are the correlation coefficient R2 and the root mean square error RMSEP respectively.

[0038] Due to the adoption of the above technical solution, the technical progress obtained by the present invention is:

[0039] After interpolating, linearly mapping and standard normal transformation of the near-infrared spectral data collected by multiple portable visible near-infrared spectrometers of different models, the present invention constructs a 1D-Inception-Resnet network framework and trains a neural network quantitative network model of the main spectrometer with strong robustness. The quantitative neural network parameters of the main spectrometer are adjusted with the near-infrared data of spectrometers of different models to reduce the spectral feature differences caused by their own components between the main instrument and different slave instruments. According to actual needs, the 1D-Inception-Resnet network model of the main instrument can be flexibly calibrated and transferred to the slave spectrometer to repair the prediction of the spectra collected by the slave spectrometer.

[0040] The design of the Inception module of the present invention widens the neural network structure compared with the existing network design concept of stacking convolutional layers in the depth direction. In addition, the sizes and numbers of the convolutional kernels of the four convolutional channels in the independently designed Inception module are set differently, which means extracting near-infrared spectral features from four different visual perspectives and enriching the learning content of the network.

[0041] The design of the residual module of the present invention widens and deepens the network while effectively overcoming the problems of gradient explosion or dispersion during network training and accelerating the network training. Brief Description of the Drawings

[0042] Figure 1 is the flowchart of the present invention;

[0043] Figure 2 is the 1D-Inception-Resnet network framework of the present invention;

[0044] Figure 3 A and 3B are the comparative prediction results of the network model after adjusting the 1D-Inception-Resnet network of the main spectrometer from the spectra of the spectrometers in Example 1 of the present invention.

[0045] Figure 4 A and 4B are the comparative prediction results of the network model after adjusting the 1D-Inception-Resnet network of the main spectrometer from the spectra of the spectrometers in Example 2 of the present invention. Detailed Description of the Invention

[0046] The present invention will be further described in detail below with reference to the embodiments:

[0047] A method for transferring a near-infrared deep neural network model between instruments, as Figure 1 shown, includes the following steps:

[0048] (1) Measuring the near-infrared spectra of the substance to be measured with multiple different models of portable visible near-infrared spectrometers.

[0049] (2) Unifying the wavelength ranges and resolutions of the near-infrared spectra of multiple different models of portable visible near-infrared spectrometers.

[0050] (3) Preprocessing the near-infrared spectra of multiple different models of portable visible near-infrared spectrometers.

[0051] After linearly mapping and performing standard normal variate (SNV) processing on the near-infrared spectra of multiple different models of portable visible near-infrared spectrometers, the baseline drift is eliminated and the robustness of the training model is enhanced.

[0052] (4) Use the SPXY algorithm to divide the near-infrared spectrum dataset of multiple portable visible near-infrared spectrometers of different models.

[0053] (5) Develop a 1D-Inception-Resnet quantitative network for the main model spectrometer, as Figure 2 shown, the structure is:

[0054] Sequence input layer - Convolution layer - Batch normalization layer - ELU activation function -

[0055] Convolution layer - Batch normalization layer - ELU activation function - (1D-Inception module)

[0056] -(Residual module) - Flatten layer - Fully connected layer - Softplus activation layer

[0057] - Fully connected layer - Softplus activation layer - Fully connected layer - Softplus activation layer

[0058] - Fully connected layer - Regression layer.

[0059] Among them, the 1D-Inception module is composed of 4 convolutional paths spliced through a dimensional splicing layer:

[0060] Path 1: Convolution layer - Batch normalization layer - ELU activation function

[0061] [[ID=2Q]]Path 2: Max pooling layer - Convolution layer - Batch normalization layer - ELU activation function

[0062] Path 3: Convolution layer - Batch normalization layer - ELU activation function - Convolution layer

[0063] - Batch normalization layer - ELU activation function

[0064] Path 4: Convolution layer - Batch normalization layer - ELU activation function - Convolution layer

[0065] - Batch normalization layer - ELU activation function.

[0066] And the Residual module is composed of the input and output of the 1D-Inception module connected by an element-wise addition layer for skip connection.

[0067] (6) Adjust the parameters of the 1D-Inception-Resnet quantitative network of the main model spectrometer with data from different sub-model spectrometers, rather than developing from scratch.

[0068] (7) Verify the transfer model using data from different sub-model spectrometers.

[0069] Example 1:

[0070] (1) Use the visible near-infrared handheld and micro-spectrometer datasets publicly available from the Gilles Chaix team. Select the near-infrared spectral data of sugarcane under TellSpec and NIRscan Nano spectrometers (master and slave spectrometers respectively), as well as the two chemical component values of ADF and IVOMD to verify the proposed algorithm.

[0071] (2) Perform linear mapping on the near-infrared spectral data of sugarcane under TellSpec and NIRscan Nano spectrometers.

[0072] (3) Select the wavelength range of the near-infrared spectral data of sugarcane under TellSpec and NIRscan Nano spectrometers between 908 and 1640 nm. Then, through piecewise cubic Hermite interpolation, change the resolution of the two spectra to 1 nm, that is, the dimensions of the two spectra are unified to 733 dimensions.

[0073] (4) Perform linear mapping on the near-infrared spectral data of sugarcane under TellSpec and NIRscan Nano spectrometers, and amplify the near-infrared spectral data of sugarcane from 60 to 600 samples.

[0074] (5) Perform SNV transformation on the near-infrared spectral data of sugarcane under TellSpec and NIRscan Nano spectrometers to eliminate spectral baseline drift.

[0075] (6) Use the SPXY algorithm to divide the near-infrared spectral data of sugarcane under TellSpec and NIRscan Nano spectrometers into calibration sets (360), validation sets (120), and prediction sets (120) respectively.

[0076] (7) Build a 1D-Inception-Resnet network framework: composed of convolutional layers, batch normalization layers, flattening layers, concat layers, addition layers, elu activation functions, softplus activation functions, and fully connected layers.

[0077] (8) Train the neural network model of the Tellspec master spectrometer: select Adam as the regressor; the maximum number of iterations is 500; the initial learning rate is 0.001; add early stopping and dropout mechanisms.

[0078] (9) Perform calibration transfer on the neural network of the Tellspec master spectrometer using the calibration set spectral data of the NIRscan nano slave spectrometer: train the initialized fully connected layer and update the convolutional layer parameters.

[0079] (10) Test the calibrated network model with the prediction set of the Tellspec slave spectrometer, and record the evaluation metrics in Table 1.

[0080] Table 1 Results of the ADF and IVOMD Quantitative Prediction Models Before and After Migration in Example 1

[0081]

[0082] Evaluation of the transfer results in Example 1:

[0083] As can be seen from Table 1: Using the quantitative network of the main spectrometer Tellspec to predict the spectra of the secondary spectrometer NIRscan nano, in the ADF task, R2 = 0.4712 and RMSEP = 9.8909; in the IVOMD task, R2 = 0.6768 and RMSEP = 12.2084. After model transfer, in the ADF task, R2 = 0.9814 and RMSEP = 1.3279; in the IVOMD task, R2 = 0.9697 and RMSEP = 2.7405. By comparison, the linear correlation degree is significantly improved, and the root mean square prediction error is greatly reduced.

[0084] Combined with Figure 3 -A and Figure 4 -A, the points (cross symbols) formed by the true values and predicted values before calibration are far from the Y = X line, and the points (squares) after calibration are concentrated on the Y = X line, indicating that after model calibration, the prediction of the spectra collected by the secondary spectrometer NIRscan nano is repaired.

[0085] Example 2:

[0086] (1) Use the visible near-infrared handheld and micro-spectrometer dataset publicly available from the Gilles chaix team. Select the sugarcane near-infrared spectral data under the TellSpec and NIR1700 spectrometers (the main and secondary spectrometers respectively) and the two chemical component values of ADF and IVOMD to verify the proposed algorithm.

[0087] (2) Perform linear mapping on the sugarcane near-infrared spectral data under the TellSpec and NIR1700 spectrometers

[0088] (3) Select the wavelength range of the sugarcane near-infrared spectral data under the TellSpec and NIR1700 spectrometers between 908 and 1640 nm, and then change the resolution of the two spectra to 1 nm through piecewise cubic Hermite interpolation, that is, the dimensions of the two spectra are unified to 733 dimensions.

[0089] (4) Perform linear mapping on the sugarcane near-infrared spectral data under the TellSpec and NIR1700 spectrometers, and amplify the sugarcane near-infrared spectral data from 60 to 600 samples.

[0090] (5) Perform SNV transformation on the near-infrared spectral data of sugarcane under TellSpec and NIR1700 spectrometers to eliminate spectral baseline drift.

[0091] (6) Use the SPXY algorithm to divide the near-infrared spectral data of sugarcane under TellSpec and NIR1700 spectrometers into calibration sets (360), validation sets (120), and prediction sets (120) respectively.

[0092] (7) Build a 1D-Inception-Resnet network framework: It consists of a convolutional layer, a batch normalization layer, a flattening layer, a concat layer, an addition layer, an elu activation function, a softplus activation function, and a fully connected layer.

[0093] (8) Train the neural network model of the Tellspec main spectrometer: The regressor selects Adam; the maximum number of iterations is 500; the initial learning rate is 0.001; early stopping and dropout mechanisms are added.

[0094] (9) Perform calibration transfer on the neural network of the Tellspec main spectrometer using the calibration set spectral data of the NIR1700 spectrometer: Train the initialized fully connected layer and update the convolutional layer parameters.

[0095] (10) Use the prediction set of the Tellspec slave spectrometer to test the calibrated network model, and record the evaluation metrics in Table 2.

[0096] Table 2 Results of the ADF and IVOMD quantitative prediction models before and after migration in Example 2

[0097]

[0098] Evaluation of the transfer results in Example 2:

[0099] As can be seen from Table 2: When using the quantitative network of the main spectrometer Tellspec to predict the spectra of the slave spectrometer NIR 1700, in the ADF task, R2 = 0.7053 and RMSEP = 8.8005; in the IVOMD task, R2 = 0.2848 and RMSEP = 48.1181. After model transfer, in the ADF task, R2 = 0.9864 and RMSEP = 1.2683; in the IVOMD task, R2 = 0.9718 and RMSEP = 2.5949. Similarly, through comparison, it can be obtained that the linear correlation degree is significantly improved and the root mean square prediction error is greatly reduced.

[0100] Combined with Figure 3 -B and Figure 4-B. Before calibration, the points (circles) formed by the true values and predicted values are far from the Y = X line. After calibration, the points (squares) are concentrated on the Y = X line. Similarly, it shows that after model calibration, the prediction of the spectra collected by spectrometer NIR1700 is corrected.

[0101] In Examples 1 and 2 of the present invention, the migration of the sugarcane chemical component prediction model between instruments is realized by fine-tuning the weights of the global convolutional layer. The fine-tuning method based on 1D-Inception-Resnet is diverse and flexible. This network structure can also be used for other research objects (not limited to sugarcane, other substances), and the fine-tuning method can be designed according to the actual situation. For example, the convolutional layer parameters of channels 1 and 2 in the Inception module are frozen, and the convolutional layer parameters of channels 3 and 4 are adjusted, or channels 1, 2, and 3 are frozen, and only channel 4 is adjusted, etc. There is no need to list them one by one here.

Claims

1. A method for transferring a near-infrared deep neural network model between instruments, characterized in that: Specifically, it includes the following steps: (1) Measure the near-infrared spectra of the substance to be measured using multiple portable visible near-infrared spectrometers of different models; (2) Unify the wavelength ranges and resolutions of the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models; (3) Preprocess the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models; In step (3), perform linear mapping on the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models to enhance the robustness of the training model; In step (3), perform standard normal variate transformation SNV on the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models to reduce baseline drift; (4) Divide the datasets of the near-infrared spectra of multiple portable visible near-infrared spectrometers of different models; (5) Develop a 1D-Inception-Resnet quantitative network for the main model spectrometer; In step (5), the overall network framework of the developed 1D-Inception-Resnet is as follows: Sequence input layer - Convolution layer - Batch normalization layer - ELU activation function - Convolution layer - Batch normalization layer - ELU activation function - 1D-Inception module - Residual module - Flatten layer - Fully connected layer - Softplus activation layer - Fully connected layer - Softplus activation layer - Fully connected layer - Softplus activation layer - Fully connected layer - Regression layer; In step (5), the 1D-Inception module is composed of 4 convolutional paths spliced together through a dimensional splicing layer: Path 1: Convolution layer - Batch normalization layer - ELU activation function Path 2: Max pooling layer - Convolution layer - Batch normalization layer - ELU activation function Path 3: Convolution layer - Batch normalization layer - ELU activation function - Convolution layer - Batch normalization layer - ELU activation function Path 4: Convolution layer - Batch normalization layer - ELU activation function - Convolution layer - Batch normalization layer - ELU activation function; (6) Adjust the parameters of the 1D-Inception-Resnet quantitative network of the main model spectrometer using data from different slave model spectrometers instead of developing from scratch; (7) Use data from different slave model spectrometers to verify the calibration model.

2. A method for transferring a near-infrared deep neural network model between instruments according to claim 1, characterized in that: In step (2), use Hermite interpolation three times to unify the wavelength range and resolution.

3. A method for transferring a near-infrared deep neural network model between instruments according to claim 1, characterized in that: In step (4), use the combined x-y distance SPXY algorithm for dataset division, and divide the near-infrared spectral data of multiple portable visible near-infrared spectrometers of different models into independent calibration sets, validation sets, and prediction sets according to 3:1:1 respectively.

4. A method for transferring a near-infrared deep neural network model between instruments according to claim 1, characterized in that: In step (5), the Residual module is composed of the input and output of the 1D-Inception module connected by an element-wise addition layer through skip connection.

5. A method for transferring a near-infrared deep neural network model between instruments according to claim 1, characterized in that: In step (6), adjust the parameters of the 1D-Inception-Resnet quantitative network of the main model spectrometer using data from different slave model spectrometers.

6. A method for transferring a near-infrared deep neural network model between instruments according to claim 1, characterized in that: In step (7), the evaluation metrics for verifying model calibration transfer are the correlation coefficient R2 and the root mean square error RMSEP respectively.

Images