Distillate oil property prediction method based on deep learning feature extraction and partial least squares regression

Abstract

The invention discloses a distillate oil property prediction method based on deep learning feature extraction and partial least squares regression. The method comprises the following steps: firstly, carrying out classification training on a near infrared spectrum through a convolution-attention double-branch fusion network, and extracting high-dimensional spectral features with local and global information; then, historical samples are retrieved from a database based on prediction categories, a plurality of most similar samples are selected by adopting cosine similarity measurement to construct a correction set, and the spectral features and property labels are subjected to standardization processing; and finally, carrying out partial least squares regression modeling on the correction set, extracting latent variables to maximize covariance between spectral features and physicochemical properties, and inputting feature vectors of an oil sample to be detected into the trained PLS model to obtain a corresponding property prediction result. According to the method, the modeling requirement and the category specificity characteristics under the small sample condition are considered while the prediction precision is guaranteed, and the method is suitable for rapid property detection and intelligent analysis in the refining process.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of distillate oil property detection in refining enterprises, and specifically relates to a distillate oil property prediction method based on deep learning feature extraction and partial least squares regression. BACKGROUND

[0002] In the refining industry, the rapid detection of distillate oil properties directly affects the optimization of production processes and the quality of final products. Near-infrared spectroscopy technology, as a non-destructive detection method, has been widely used in distillate oil classification and property detection. However, existing methods still have the following problems:

[0003] Firstly, the number of samples with physicochemical property labels is very limited. Property detection often relies on special experimental equipment, long experimental periods, and high reagent costs, so it is difficult to obtain large-scale labeled property datasets in actual production. This limitation makes it difficult to train an end-to-end deep regression model directly, which lacks reliability and feasibility.

[0004] Secondly, the distillate oil property prediction task has obvious class specificity. Different oil categories have significant differences in property detection indicators. For example, gasoline needs to focus on octane number and distillation characteristics, diesel oil focuses on cetane number and sulfur content, and lubricating oil emphasizes viscosity and pour point performance. Due to the differences in property indicators of various categories, a single unified regression model cannot achieve ideal results in cross-category prediction tasks.

[0005] Based on the above shortcomings, the application proposes a new solution: learning deep feature representation through a spectral classification task and combining partial least squares regression to realize property prediction. This method uses the rich unlabeled spectral samples of the classification task to extract potential spectral features, effectively addressing the modeling difficulties under small sample conditions. At the same time, a dynamic correction set construction mechanism based on categories is introduced to ensure that the regression modeling process is more targeted, thereby achieving category-specific property prediction. Compared with traditional methods, the application not only improves prediction accuracy and generalization ability, but also considers model interpretability and engineering practicality, providing a new intelligent detection solution for the refining industry that is efficient and reliable.

[0006] In the previous study, we have applied for "a kind of intelligent classification method for near infrared spectrum property detection of distillate oil (application number: 202411931005.6)" and "a method for improving the precision of intelligent classification of distillate oil (application number: 202510486438.3)" and other related invention patents, both of which are aimed at the classification task of distillate oil spectrum and have played a role in improving the discrimination accuracy. However, its limitation is that it can only perform class recognition and cannot realize the prediction of physical and chemical properties. In actual production, the property prediction of distillate oil is particularly important for refining enterprises. How to realize the expansion and innovation from the classification task to the property prediction task, and improve the prediction accuracy and generalization ability of property prediction, has become a difficult problem that needs to be overcome in this application. SUMMARY

[0007] To solve the problems in the background art, the present application provides a distillate oil property prediction method based on deep learning feature extraction and partial least squares regression. First, a deep learning network is trained through a spectrum classification task to extract spectral features. Then, a correction set is dynamically constructed based on cosine similarity measurement. Finally, partial least squares regression is used to model the features to predict the properties of distillate oil. The specific steps are as follows:

[0008] Step 1: Read the original data in the distillate oil spectrum database, clean and preprocess the spectrum: in the cleaning stage, abnormal data is removed based on absorbance threshold; in the preprocessing stage, baseline correction, band cutting, vector normalization and S-G smoothing are performed to generate dataset X. According to the ratio of 8:2, the training set X train and the test set X test are divided.

[0009] Step 2: Construct a neural network structure with convolution-attention dual branch fusion, generate high-dimensional feature representation of distillate oil through feature concatenation module, and use the joint supervision mechanism of cross-entropy loss and center loss for network training. The composite loss function L used in network training is:

[0010]

[0011] Where, the center loss L1 is the Euclidean distance between the high-dimensional feature and the high-dimensional feature center of the distillate oil category, z i ∈R d represents the high-dimensional feature output by the neural network for the ith sample, belongs to the y i category, d is the feature dimension, represents the high-dimensional feature center of the distillate oil of the y i category in the feature space, the batch size is M, and λ is the loss weight; L2 is the cross-entropy loss function, W j ∈R d is the parameter matrix W d×Nthe jth column of B, b j ∈ R N is the bias term. The center loss in the network training process adopts a mini-batch update strategy, and sets the hyperparameter a to adjust the learning rate of the center update.

[0012] Step 3, input the training set X train into the network, and adjust the weight parameters of the model by minimizing the loss function L through the AdamW algorithm, to learn the nonlinear mapping relationship between the spectral absorbance and the fraction oil category.

[0013] Step 4, input the test set X test into the network to evaluate the generalization ability of the network weight parameter model, and determine whether the classification accuracy of the model on the test set is greater than 98%. If it is satisfied, go to step 5, otherwise modify the hyperparameters and go to step 3.

[0014] Step 5, input the near-infrared spectrum of the oil sample to be tested into the trained convolution-attention dual network to extract a high-dimensional feature vector Z.

[0015] Step 6, based on the predicted category obtained in step 5), retrieve historical samples belonging to the same category from the spectrum database, select the top 50 most similar samples according to the cosine similarity measure, and construct a correction set. The cosine similarity calculation formula is:

[0016]

[0017] where Z and V are high-dimensional feature vectors of the sample to be tested and the samples in the database, respectively, ‖·‖ represents the Euclidean norm of the vector, and · represents the dot product operation of the vector.

[0018] Step 7, standardize the spectrum data and property labels of the correction set, including mean centering and unit variance scaling of the spectrum data and property labels respectively:

[0019]

[0020] where X ∈ R n×m is the spectrum matrix, Y ∈ R n×p is the property matrix, n = 50 is the number of samples, m is the dimension, and p is the number of properties. μ X , σ X respectively represent the mean and standard deviation of the spectrum matrix array vector, μ Y , σ Y respectively represent the mean and standard deviation of the property matrix array vector.

[0021] Step 8. Extract latent variables to maximize the covariance between spectral features X' and property data Y' using partial least squares regression on the standardized calibration set to build the property prediction model. The PLS iteration process is as follows: in the kth iteration,

[0022] 1) Calculate the spectral projection direction w k and the property projection direction c k :

[0023]

[0024] where X k ∈R n×m and Y k ∈R n×p are the spectral residual matrix and the property residual matrix, respectively. When k is equal to 1, X1= X' and Y1= Y'.

[0025] 2) Calculate the spectral score vector t k ∈R n and the property score vector u k ∈R n :

[0026] t k = X k w k , u k = Y k c k

[0027] 3) Calculate the spectral loading vector p k ∈R m and the property loading vector q k ∈R p :

[0028]

[0029] 4) Update the spectral residual matrix X k+1 ∈R n×m and the property residual matrix Y k+1 ∈R n×p :

[0030] X k+1 = X k - t k p k T , Y k+1 = Y k - t k q k T

[0031] Based on the extracted k latent variables, a linear relationship between spectral features and oil properties is established:

[0032]

[0033] wherein T=[t1,t2,…,t K ]∈R n×K is a score matrix, P=[p1,p2,…,p K ]∈R m×K , Q=[q1,q2,…,q K ]∈R p×K is a load matrix, E x ,E y ,F is a residual matrix. Regression coefficient matrix B∈R m×p can be calculated by the following formula:

[0034] B=W(P T W) -1 Q T

[0035] wherein W=[w1,w2,…,w K ]∈R m×K is a spectral weight matrix.

[0036] Step 9, input the high-dimensional feature vector Z of the oil sample to be tested into the learned PLS regression (property prediction) model, and output the prediction result of the physical and chemical properties of the oil sample.

[0037] Beneficial effects:

[0038] The application discloses a distillate oil property prediction method based on deep learning feature extraction and partial least squares regression. The method uses a spectral classification task to pre-train a neural network, extracts potential features from a large number of spectra without property labels, and then combines PLS regression to realize property modeling under a small sample condition. In view of the differences between different oil categories in property detection indexes, the application adopts a category-based dynamic correction strategy, and only models within the same sample, so as to improve the prediction accuracy and adaptability, and is suitable for rapid property detection in a refining process. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The application is a distillate oil property prediction method based on deep learning feature extraction and partial least squares regression.

[0040] Figure 2 It is a structure diagram of a one-dimensional convolution self-attention network Conformer1D.

[0041] Figure 3 It is a schematic diagram of a center loss and cross-entropy loss joint supervision training method.

[0042] Figure 4A plot of accuracy curves for the model training process.

[0043] Figure 5 A framework diagram for gasoline property prediction. DETAILED DESCRIPTION

[0044] The embodiments of the present application are described in detail below, which are implemented on the premise of the technical solutions of the present application, and detailed implementation manners and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0045] By implementing the near-infrared spectrum rapid analysis technology in many domestic refining and chemical devices, the enterprise has collected a large amount of distillate oil spectrum data, and the data in the database of the enterprise is used to verify the effectiveness of the distillate oil property prediction method based on deep learning feature extraction and partial least squares regression proposed in the present application.

[0046] The overall system flowchart of the present application is shown in Figure 1 The specific steps include:

[0047] 1) Read the original data in the distillate oil spectrum database, and Table 1 is part of the spectrum data and labels.

[0048] Table 1 spectrum absorbance measurement data

[0049]

[0050] The original labeled spectrum data is cleaned and preprocessed to generate a data set X. The data cleaning is a threshold-based spectrum abnormal data cleaning; the data preprocessing includes baseline correction, wave number segment cutting, vector normalization and S-G smoothing. Table 2 is part of the abnormal spectrum data, and the absorbance is far above the threshold upper limit of 2.5, and Table 3 is the spectrum data after preprocessing. The cleaned and preprocessed data set X is divided into a training set X train and a test set X test according to the ratio of 8:2.

[0051] Table 2 abnormal spectrum data

[0052]

[0053] Table 3 original spectrum data and spectrum data after preprocessing

[0054]

[0055] 2) The architecture of the one-dimensional convolution self-attention network Conformer1D is shown in Figure 2 , and the high-dimensional feature expression of the distillate oil is generated through a feature splicing module. As Figure 3As shown, a joint supervision mechanism of cross-entropy loss and center loss is used for network training. The composite loss function L used for network training is:

[0056]

[0057] Wherein, the central loss L1 is the Euclidean distance between the high-dimensional feature centers and the high-dimensional feature centers of the distillate oil types, z i ∈R d This represents the high-dimensional feature output by the neural network after the i-th sample is processed, belonging to the y-th sample. i Class, d is the feature dimension, Represents the y-th element in the feature space i The high-dimensional feature centers of the distillate oil are defined, with a batch size of M and λ as the loss weight; L2 is the cross-entropy loss function, and W... j ∈R d The parameter matrix W∈R d×N The j-th column, b j ∈R N This is the bias term. The center loss during network training employs a mini-batch update strategy, and a hyperparameter α is set to adjust the learning rate for center updates.

[0058] 3) Transfer the training set X train In the input network, the initial learning rate of the network is set to 1e. -4 The learning rate decay rate is 0.1, the batch size is 256, and the regularization coefficient is 1e. -4 The dropout rate was 0.5, λ = 0.1, α = 0.01, and the total training duration was 100 epochs. The loss function L was minimized using the AdamW algorithm based on the defined loss function. Figure 4 The graph shows the accuracy of the model training process, with an accuracy of 99.17%, which meets the requirements.

[0059] 4) such as Figure 5 As shown, taking gasoline as an example, this category involves several key physicochemical properties, including density at 20℃, initial boiling point, 10% evaporation temperature, 50% evaporation temperature, 90% evaporation temperature, final boiling point, research octane number, and anti-knock index. The near-infrared spectrum of the oil sample to be tested is input into a trained convolutional-attention dual-branch network to extract a high-dimensional feature vector Z. Based on this feature vector, the category of the oil sample to be tested is first predicted, and it is successfully identified as gasoline. Next, historical samples of the same category are retrieved from the spectral database, and based on the cosine similarity metric, the top 50 samples most similar to the oil sample to be tested are selected, thus constructing a calibration set to provide data support for subsequent partial least squares regression modeling.

[0060] 5) Standardization of the spectral data and property labels of the calibration set, including mean centering and unit variance scaling of the spectral data and property labels, respectively.

[0061] 6) Extraction of latent variables by partial least squares regression on the standardized calibration set to maximize the covariance between the spectral features X' and the property data Y', and establishment of a property prediction model.

[0062] 7) Input of the high-dimensional feature vector Z of the oil sample to be tested into the trained PLS regression model to obtain the prediction result of the physical and chemical properties of the oil sample. To verify the effectiveness of the method, a control test is designed: on the one hand, the most similar top 50 gasoline samples are selected from the database according to the cosine similarity to construct the calibration set; on the other hand, the top 50 samples of the gasoline category in the database are directly selected as the calibration set. By comparing the performance of the two methods in property prediction, the root mean square error (RMSE), the relative root mean square error (rRMSE), and the determination coefficient (R 2 ) and other evaluation indexes are calculated, and the results are shown in Tables 4, 5 and 6. The experimental results show that the calibration set selected based on similarity is superior to the direct selection method in terms of various indicators, proving the effectiveness and superiority of the method in the task of distillate oil property prediction.

[0063] Table 4. Comparison of RMSE of oil property prediction under different calibration set construction methods

[0064]

[0065] Table 5. Comparison of rRMSE of oil property prediction under different calibration set construction methods

[0066]

[0067] Table 6. Comparison of R 2 of oil property prediction under different calibration set construction methods

[0068]

[0069]

Claims

1. A method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression, characterized in that, First, a deep learning network is trained through a spectral classification task to extract spectral features. Then, a calibration set is dynamically constructed based on cosine similarity. Finally, partial least squares (PLS) regression is used to model the features to predict the properties of distillate oils. The specific steps include the following: 1) Read the raw data from the distillate oil spectral database, clean and preprocess the spectra to generate dataset X, and divide it into training set X. train and test set X test ; 2) Construct a neural network structure that integrates convolution and attention branches, generate high-dimensional feature representations of distillate oil through a feature splicing module, and train the network using a joint supervision mechanism of cross-entropy loss and center loss; 3) Transfer the training set X train In the input network, the loss function L is minimized using the AdamW algorithm, the model's weight parameters are adjusted, and the nonlinear mapping relationship between spectral absorbance and distillate oil category is learned. 4) Through test set X test Evaluate the generalization ability of the network weight parameter model and determine whether the model's classification accuracy on the test set is greater than 98%. If it is satisfied, proceed to step 5); otherwise, modify the hyperparameters and proceed to step 3. 5) Input the near-infrared spectrum of the oil sample to be tested into the trained convolutional-attention dual-splitting network to extract the high-dimensional feature vector Z; 6) Based on the predicted category obtained in step 5), retrieve historical samples belonging to the same category from the spectral database, select the 50 most similar samples according to the cosine similarity measure, and construct a calibration set; 7) Standardize the spectral data and property labels of the calibration set, including mean centering and unit variance scaling for the spectral data and property labels respectively; 8) Partial least squares (PLS) regression is used on the standardized calibration set to extract latent variables to maximize the covariance between spectral features X′ and property data Y′, and a property prediction model is established. 9) Input the high-dimensional feature vector Z of the oil sample to be tested into the learned property prediction model, and output the physicochemical property prediction results of the oil sample.

2. The method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression according to claim 1, characterized in that, The cleaning stage uses an absorbance threshold-based method to remove outlier data; the preprocessing stage includes baseline correction, band clipping, vector normalization, and SG smoothing.

3. The method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression according to claim 1, characterized in that, The network training process employs a composite loss function L constructed by combining center loss and cross-entropy loss: Wherein, the central loss L1 is the Euclidean distance between the high-dimensional feature centers and the high-dimensional feature centers of the distillate oil types, z i ∈R d This represents the high-dimensional feature output by the neural network after the i-th sample is processed, belonging to the y-th sample. i Class, d is the feature dimension, Represents the y-th element in the feature space i The high-dimensional feature centers of the distillate oil are defined, with a batch size of M and λ as the loss weight; L2 is the cross-entropy loss function, and W... j ∈R d The parameter matrix W∈R d×N The j-th column, It is the yth i The deviation term of the class, b j ∈R N This is the bias term, and N is the number of categories.

4. The method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression according to claim 1, characterized in that, The center loss in the network training process adopts a mini-batch update strategy, and a hyperparameter α is set to adjust the learning rate of center update.

5. The method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression according to claim 1, characterized in that, The formula for calculating cosine similarity is: Where Z and V are the high-dimensional feature vectors of the sample to be tested and the samples in the database, respectively, ‖·‖ represents the Euclidean norm of the vector, and · represents the dot product operation of the vector.

6. The method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression according to claim 1, characterized in that, The formulas for mean centering and unit variance scaling are as follows: Where, X∈R n×m Let Y be the spectral matrix, Y∈R n×p This is the property matrix, where n = 50 is the number of samples, m is the dimension, and p is the number of properties; μ X , σ X Let μ represent the mean and standard deviation of the column vectors of the spectral matrix, respectively. Y , σ Y X' represents the mean and standard deviation of the column vectors of the property matrix, respectively; X' represents the standardized spectral characteristics; and Y' represents the standardized property data.

7. The method for predicting the properties of distillate oil based on deep learning feature extraction and partial least squares regression according to claim 1, characterized in that, The PLS iterative process is as follows: In the k-th iteration... 1) Calculate the spectral projection direction w k And the projection direction c of the properties k : Among them, X k ∈R n×m Y k ∈R n×p These are the spectral residual matrix and the property residual matrix, respectively. When k equals 1, X1 = X′ and Y1 = Y′. 2) Calculate the spectral score vector t k ∈R n And property score vector u k ∈R n : t k =X k w k ,u k =Y k c k 3) Calculate the spectral load vector p k ∈R m and property load vector q k ∈R p : 4) Update the spectral residual matrix X k+1 ∈R n×m and properties of the residual matrix Y k+1 ∈R n×p : X k+1 =X k -t k p k T ,Y k+1 =Y k -t k q k T 。 8. The method for predicting the properties of distillate oils based on deep learning feature extraction and partial least squares regression according to claims 1 and 7, characterized in that, Based on the extracted K latent variables, a linear relationship between spectral characteristics and oil properties is established: Where T = [t1, t2, ..., t K ]∈R n×K Let P be the score matrix, where P = [p1, p2, ..., p...]. K ]∈R m×K Q = [q1, q2, ..., q K ]∈R p×K Let E be the load matrix. x E y F represents the final spectral residual matrix, label residual matrix, and regression residual matrix, respectively; B is the regression coefficient matrix.

9. The method for predicting the properties of distillate oil based on deep learning feature extraction and partial least squares regression according to claim 8, characterized in that, The regression coefficient matrix B∈R m×p It can be calculated using the following formula: B=W(P T W) -1 Q T Where W = [w1, w2, ..., w K ]∈R m×K This is the spectral weighting matrix.

10. The method for predicting the properties of distillate oil based on deep learning feature extraction and partial least squares regression according to claim 8, characterized in that, K＝10。

Images