A method for predicting the near-infrared spectral properties of end-to-end distillate oils

Abstract

This invention discloses an end-to-end near-infrared spectral property prediction method for distillate oils. Using Conformer-1D as the backbone network, a multi-task learning framework combining classification and multi-property regression is constructed. Spectral features are extracted through a shared dimensionality-reduced MLP and a residual bottleneck interaction module. Then, classification prior information drives a gating module to allocate multi-expert network weights, achieving weighted fusion prediction of distillate oil properties. Simultaneously, a joint loss function including classification loss, regression loss, and expert load balancing loss is constructed. A three-stage training strategy of classification pre-training, multi-task training, and full-data joint fine-tuning is employed to optimize the model. This invention achieves end-to-end unified modeling for spectral feature extraction and multi-property prediction, fully utilizing category information and property association features, effectively alleviating the label sample imbalance problem, and improving the accuracy and stability of distillate oil property prediction.

Description

Technical Field

[0001] This invention belongs to the field of distillate oil property detection in refining and chemical enterprises, specifically a distillate oil property prediction method based on a deep learning multi-task framework. Background Technology

[0002] In the refining industry, the rapid acquisition of distillate oil properties is of great significance for production regulation and product quality control. Near-infrared spectroscopy, due to its non-destructive and rapid advantages, has been widely used in the field of oil analysis.

[0003] Previous work proposed "A Method to Improve the Accuracy of Intelligent Classification of Distillate Oils (Publication No.: CN120408270A)," which improved the accuracy of spectral classification and provided discriminative features for property prediction. Furthermore, "A Method for Predicting the Properties of Distillate Oils Based on Deep Learning Feature Extraction and Partial Least Squares Regression (Publication No.: CN121122479A)" was proposed, effectively improving prediction performance under conditions of small samples and unconventional properties. These methods demonstrate that features learned from classification tasks have good application potential in property prediction.

[0004] In prediction tasks primarily involving conventional properties such as density and distillation range, relevant data can be obtained online, and the data scale is relatively large, providing the conditions to support more complex model building. However, existing methods still employ chemometric methods or shallow network modeling. This approach has the following shortcomings: feature extraction and property prediction processes are independent, lacking a unified modeling mechanism, making it difficult to achieve a synergistic expression of spectral features for classification information and property prediction needs; furthermore, different properties are typically modeled separately, failing to effectively utilize the shared characteristics of spectral information, thus limiting overall prediction performance and application efficiency.

[0005] Therefore, it is necessary to further develop an end-to-end modeling method for predicting conventional properties based on existing classification feature extraction methods, so as to improve the overall performance and engineering application efficiency of distillate oil property prediction. Summary of the Invention

[0006] This invention addresses the problems existing in the background technology by proposing an end-to-end near-infrared spectral property prediction method for distillate oils. Using Conformer-1D as the backbone network, a multi-task joint modeling framework based on classification and multi-property regression is constructed, and an end-to-end prediction of distillate oil properties is achieved through a phased training strategy. Specifically, the Conformer-1D network is constructed and trained through the following steps: replacing the two-dimensional convolution, global average pooling, layer normalization, and batch normalization operations in the traditional Conformer with corresponding one-dimensional operations; replacing the downsampling and upsampling operations in the Feature Coupling Unit (FCU) with scaling operations based on one-dimensional sequence length, thereby constructing a one-dimensional Conformer network structure suitable for spectral data, and training it based on near-infrared spectral data of distillate oils. The prediction method specifically includes the following steps:

[0007] Step 1: Preprocess the near-infrared spectral data of distillate oil by baseline correction, band interception, and vector normalization.

[0008] Step 2: Extract high-dimensional fusion feature vectors from the preprocessed near-infrared spectral data of the distillate oil using the Conformer-1D backbone network. .

[0009] Step 3, fuse the feature vectors The input is processed by a shared dimensionality reduction MLP module for stepwise feature compression to obtain low-dimensional shared features. The shared dimensionality reduction MLP module consists of a two-layer network structure: the first layer is a linear layer, layer normalization, GELU activation function and dropout layer, which compresses 1600-dimensional features to 512 dimensions; the second layer is a linear layer and GELU activation function, which further compresses the 512-dimensional features to 128-dimensional unified shared features.

[0010] Step 4: Use the shared bottleneck interaction module to handle shared features. Feature extraction is performed to obtain features. The shared bottleneck interaction module adopts a residual bottleneck structure, sequentially reducing the 128-dimensional features to 64 dimensions through a three-layer MLP, then increasing them back to 128 dimensions, and then sharing the features. Perform residual addition and fusion.

[0011] Step 5, the classification head is based on shared features Output category probability ,Will As prior information input to the gating module, it generates the weight coefficients of each expert network. The gating module concatenates shared features and class probabilities, then performs MLP mapping and Softmax normalization to obtain the weight coefficients of each expert network. .

[0012] Step 6, feature Input multiple parallel expert networks to obtain prediction vectors for each set of properties. The expert network is a two-layer MLP structure that maps 128-dimensional features into a 5-dimensional property prediction vector containing the initial boiling point (IBP), 10% evaporation temperature (T10), 50% evaporation temperature (T50), 90% evaporation temperature (T90), and final boiling point (FBP). The number of expert networks is 8.

[0013] Step 7: Based on the weights of the gating modules, the outputs of each expert network are weighted and fused to output the joint prediction results of multiple properties of distillate oil. Its formula is:

[0014]

[0015] in, For the first The weight of each expert, , For the number of experts;

[0016] Step 8, construct the classification loss. Regression loss With load balancing losses Composition of multi-task joint loss function Classification loss The cross-entropy loss function is used. The regression loss uses the Huber loss function, with a threshold hyperparameter... :

[0017]

[0018] in, For true property values, This is the model's predicted value. The average loss over all samples within a batch is taken as the final regression loss. The load balancing loss is:

[0019]

[0020] in, Indicates the first sample in a batch The average gating weights of the experts. The multi-task joint loss function is:

[0021]

[0022] in, , This is the load balancing coefficient.

[0023] Step 9: Complete model training using a phased training strategy.

[0024] Step 9-1: Use only the class-labeled samples to pre-train the Conformer-1D backbone network for classification, and adopt a central joint supervision strategy to train for 100 rounds.

[0025] Step 9-2: Load pre-trained weights, freeze the backbone network parameters, train the high-level modules for 15 rounds, and set... Load balancing coefficient Train the high-level task modules, then unfreeze the backbone network, in order to The learning rate was fine-tuned for 40 rounds.

[0026] Step 9-3 involves joint fine-tuning using the full sample dataset, training for 20 epochs. An indicator function is introduced. Correcting the regression loss:

[0027]

[0028]

[0029] in, For samples with classification and property labels, The spectrum after preprocessing. These are category tags. It is a property label. The property values ​​predicted by the model. Indicates the sample index. For samples with classification labels, This represents the number of samples in the batch.

[0030] Step 10: After the model training is completed, input the near-infrared spectral data of the distillate oil to be tested into the model. After preprocessing, feature extraction, and multi-task joint inference, output the corresponding multi-property prediction results. .

[0031] Beneficial effects:

[0032] This invention achieves unified optimization of spectral feature extraction and property prediction by constructing an end-to-end modeling method that combines classification and multi-property regression, avoiding the information fragmentation problem of traditional staged modeling. By using classification results to guide the adaptive weighting allocation of a multi-expert network, the model can perform differentiated modeling for different distillate oil categories, improving prediction accuracy and adaptability. Simultaneously, the use of a multi-expert structure to jointly model various common properties fully utilizes the correlations between properties, reducing model complexity while improving overall prediction performance, and enhancing model stability through a staged training strategy. Attached Figure Description

[0033] Figure 1A method for predicting the near-infrared spectral properties of end-to-end distillate oils

[0034] Figure 2 Conformer-1D Structure Diagram

[0035] Figure 3 Multi-task network framework with Comformer-1D as the backbone network

[0036] Figure 4 Accuracy curve during the classification pre-training phase

[0037] Figure 5 Classification accuracy curve during multi-task training (frozen backbone) phase

[0038] Figure 6 Multi-task training (frozen backbone) phase property prediction root mean square error (RMSE) curve

[0039] Figure 7 Multi-task training (frozen backbone) phase property prediction coefficient of determination ( (curve graph)

[0040] Figure 8 Classification accuracy curve during multi-task training (thawed backbone) phase

[0041] Figure 9 Multi-task training (thawed backbone) phase property prediction root mean square error (RMSE) curve

[0042] Figure 10 Multi-task training (unfrozen backbone) phase property prediction coefficient of determination ( (curve graph)

[0043] Figure 11 Classification accuracy curve during the full-data joint fine-tuning stage

[0044] Figure 12 Full-data joint fine-tuning stage property prediction root mean square error (RMSE) curve

[0045] Figure 13 Coefficient of determination for the property prediction of the full data joint fine-tuning stage ( (curve graph) Detailed Implementation

[0046] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0047] The company has collected 60,435 unlabeled near-infrared spectral samples, covering eight typical distillate oil categories, and 35,863 labeled spectral samples. The predicted properties are all distillation range-related parameters: initial boiling point, 10% evaporation temperature, 50% evaporation temperature, 90% evaporation temperature, and final boiling point. Using this company's data, the effectiveness of the fine-grained distillate oil spectral classification method based on feature similarity weighted voting proposed in this invention is verified.

[0048] The overall system flowchart of the present invention is as follows: Figure 1 As shown, the specific steps include:

[0049] 1) Preprocess the data, including baseline correction, band truncation, and vector normalization.

[0050] 2) Construct a multi-task network with Comformer-1D as the backbone network. The Comformer-1D structure is as follows: Figure 2 As shown, the traditional Conformer's two-dimensional convolution, global average pooling, layer normalization, and batch normalization are replaced with one-dimensional operations; the feature coupling unit (FCU)'s downsampling and upsampling operations are both replaced with one-dimensional length scaling operations. The multi-task framework structure is as follows: Figure 3 As shown, based on Conformer-1D, a shared feature extraction module, a feature interaction module, a classification branch, and a multi-expert regression branch are added to achieve joint modeling of distillate oil classification and multi-property prediction.

[0051] 3) Construct a system based on classification loss. Regression loss With load balancing losses Composition of multi-task joint loss function .

[0052] 4) The Comformer-1D backbone network was pre-trained for classification using only class-labeled samples, employing a joint supervision strategy with central monitoring, and the initial learning rate of the network was set to [value missing]. The learning rate decay rate is Batch size is 256, regularization coefficient is... The discard rate was 0.5%. , , 100 rounds of training. Figure 4 This demonstrates the changes in accuracy during the classification pre-training phase. After training, the classification accuracy on the test set reaches [percentage missing]. .

[0053] 5) Load the pre-trained weights, freeze the backbone network parameters, train the high-level modules for 15 rounds, and set... Load balancing coefficient Train the high-level task modules, then unfreeze the backbone network, in order to The learning rate was fine-tuned for 40 rounds. Figure 5 , 6 Figure 7 shows the performance changes when the backbone network is frozen. The model converges quickly when only high-level task modules are updated. The classification accuracy during training and testing reaches [percentage missing]. and The test set for property prediction achieve RMSE from the initial Down to The error was significantly reduced. Figure 8 , 9 10. Multi-task training results after unfreezing the backbone network (40 rounds): The classification accuracy on the training and test sets stabilized at [value missing]. Test set Upgraded to RMSE further decreased to approximately The regression error continued to decrease.

[0054] 6) Joint fine-tuning was performed using the full sample dataset, with 20 training epochs. An indicator function was introduced. The regression loss is then corrected. Figure 11 , 12 Section 13 shows the training results during the full-data joint fine-tuning phase (20 epochs). In the early stages of training, the classification accuracy showed a slight decrease, from the initial... Gradually recovering. As training progresses, the training function changes from... Rapidly descending to After approximately the 10th round, all indicators tended to stabilize, and the model entered a stable convergence phase. Ultimately, the classification accuracy on the test set reached [percentage missing]. Nature prediction for RMSE is Furthermore, the curves of the test set and the training set show basically the same trend, with no overfitting phenomenon.

[0055] Table 1 presents the performance comparison results of different model architectures on classification and property prediction tasks. Using Conformer-1D as the regression baseline model, the accuracy for the single-class classification task is... The RMSE for the single-attribute regression task was 10.72. for This indicates that the bi-branch structure can effectively extract spectral features and perform modeling. Further introducing a multi-task framework only reduces the model's classification accuracy by a small margin. However, RMSE further decreased to , Upgraded to The results show that, within the multi-task framework of "classification first, then property," the classification task can provide effective feature constraints for property prediction, thereby improving prediction performance.

[0056] Table 1. Experimental Results Comparing the Effectiveness of Multi-Task Learning Frameworks

[0057] .

Claims

1. A method for predicting the near-infrared spectral properties of distillate oils from end to end, characterized in that, Using Conformer-1D as the backbone network, a multi-task joint modeling framework based on classification and multi-property regression is constructed. Through a phased training strategy, end-to-end prediction of distillate oil properties is achieved, specifically including the following steps: 1) Baseline correction, band interception, and vector normalization preprocessing were performed on the near-infrared spectral data of distillate oil; 2) The preprocessed spectral data is processed by the Conformer-1D backbone network to extract high-dimensional fusion feature vectors. ; 3) Fuse feature vectors The input is processed by a shared dimensionality reduction MLP module for stepwise feature compression to obtain low-dimensional shared features. ; 4) Sharing features through the shared bottleneck interaction module Feature recombination is performed to obtain features ; 5) Classification head based on shared features Output category probability ,Will As prior information input to the gating module, it generates the weight coefficients of each expert network. ; 6) Features Input multiple parallel expert networks to obtain prediction vectors for each set of properties. ; 7) The outputs of each expert network are weighted and fused according to the weights of the gating module to output the joint prediction results of multiple properties of distillate oil. Its formula is: in, For the first The weight of each expert, For the corresponding expert's output, For the number of experts; 8) Construct a system based on classification loss. Regression loss With load balancing losses Composition of multi-task joint loss function ; 9) A phased training strategy is used to complete model training; 10) After the model training is completed, the near-infrared spectral data of the distillate oil to be tested is input into the model. After preprocessing, feature extraction and multi-task joint inference, the corresponding multi-property prediction results are output. .

2. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, The shared dimensionality reduction MLP module contains a two-layer network structure: the first layer consists of a linear layer, layer normalization, GELU activation function, and Dropout layer; the second layer consists of a linear layer and GELU activation function.

3. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, The shared bottleneck interaction module adopts a residual bottleneck structure, sequentially reducing the 128-dimensional features to 64 dimensions through a three-layer MLP, then increasing them back to 128 dimensions, and then sharing the features. Perform residual addition and fusion.

4. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, The gating module concatenates shared features and class probabilities, then performs MLP mapping and Softmax normalization to obtain the weight coefficients of the expert network. .

5. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, The expert network is a two-layer MLP structure that maps features into property prediction vectors containing the initial boiling point, 10% evaporation temperature, 50% evaporation temperature, 90% evaporation temperature, and final boiling point. The number of expert networks is 8.

6. The method for predicting the near-infrared spectral properties of distillate oils from end to end according to claim 1, characterized in that, Classification loss The cross-entropy loss function is used.

7. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, Regression loss Using the Huber loss function, threshold hyperparameter in, For true property values, These are the model's predicted values.

8. The method for predicting the near-infrared spectral properties of distillate oils from end to end according to claim 1, characterized in that, The load balancing loss is: in, Indicates the first sample in a batch Average gating weight of each expert.

9. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, The joint loss function for multiple tasks is: in, , This is the load balancing coefficient.

10. The method for predicting the near-infrared spectral properties of distillate oils from end-to-end sources according to claim 1, characterized in that, The phased training strategy includes the following steps: 1) The Conformer-1D backbone network is pre-trained for classification using only class-labeled samples, and a joint supervision strategy is adopted. 2) Load the pre-trained weights, freeze the backbone network first, train the high-level task modules, and then unfreeze the backbone network for fine-tuning with a small learning rate. 3) Joint fine-tuning is performed using the full sample dataset, and an indicator function is introduced. Correcting the regression loss: in, For samples with classification and property labels, The spectrum after preprocessing. These are category tags. It is a property label. The property values ​​predicted by the model. Indicates the sample index. For samples with classification labels, This represents the number of samples in the batch.

Images