A distillate deep classification network lightweight method
By performing structured pruning and low-rank decomposition on the Conformer-1D network, and combining it with quantization techniques, the resource requirements of the distillate oil classification model were optimized, solving the problems of large model size and high computational resources, and enabling efficient deployment and fast inference on edge devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING RICHISLAND INFORMATION TECH CO LTD
- Filing Date
- 2026-04-17
- Publication Date
- 2026-06-19
AI Technical Summary
Deep neural networks have a large number of model parameters and high computational resource requirements in distillate oil classification, making them difficult to deploy efficiently on resource-constrained embedded platforms and mobile terminals.
Based on the Conformer-1D network, a structured pruning strategy is adopted to prune the convolutional branches. Combined with low-rank decomposition and post-training quantization techniques, the attention branches are weighted and compressed to construct a one-dimensional Conformer network, thereby optimizing the model size and computational efficiency.
While maintaining classification accuracy, the model size and inference latency were significantly reduced, the model's deployment capability on edge devices was improved, and efficient distillate oil classification was achieved.
Smart Images
Figure CN122245525A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of distillate oil property detection in refining and chemical enterprises, specifically a lightweight method for deep classification networks of distillate oil. Background Technology
[0002] In the refining industry, rapid detection of distillate oil properties directly impacts the optimization of production processes and the quality of final products. Near-infrared spectroscopy, as a non-destructive testing method, has been widely applied to the classification and property detection of distillate oils.
[0003] Previously, we applied for invention patents for "An intelligent classification method for distillate oils for near-infrared spectral property detection (application number: 202411931005.6)" and "A method to improve the accuracy of intelligent classification of distillate oils (application number: 202510486438.3)". We proposed a complete data preprocessing process, constructed a neural network structure that integrates convolution and attention mechanisms, and introduced a joint supervision strategy of center loss and cross-entropy loss to achieve high-precision classification of distillate oils.
[0004] However, the high performance of deep neural networks typically relies on large-scale model structures and high computational resources, which poses challenges to their deployment and inference in real-world industrial environments. On the one hand, the large number of model parameters consumes significant storage resources; on the other hand, it places high demands on the hardware capabilities of deployment environments such as embedded platforms, FPGAs, or mobile terminals, hindering widespread application under resource-constrained conditions. Therefore, it is desirable to perform structural compression and computational optimization of neural networks while maintaining classification accuracy. Summary of the Invention
[0005] This invention addresses the problems existing in the background technology by proposing a lightweight method for deep classification networks of distillate oils. Based on a trained Conformer-1D network, a structured pruning strategy is introduced for the convolutional branches, a low-rank decomposition method is adopted for the attention branches, and post-training quantization technology is combined. This effectively compresses the model size and accelerates inference speed without significantly sacrificing accuracy, thereby significantly improving the deployment and application capabilities of the intelligent classification model for distillate oils in practical industrial scenarios. Specifically, the Conformer-1D network is constructed and trained through the following steps: replacing the two-dimensional convolution, global average pooling, layer normalization, and batch normalization operations in the traditional Conformer with corresponding one-dimensional operations; replacing the downsampling and upsampling operations in the Feature Coupling Unit (FCU) with scaling operations based on one-dimensional sequence length, thus constructing a one-dimensional Conformer network structure suitable for spectral data, and training it based on near-infrared spectral data of distillate oils. The specific steps of this invention are as follows:
[0006] Step 1: Read the raw data from the distillate oil spectral database and perform spectral cleaning and data preprocessing: Data cleaning uses an absorbance threshold-based method to remove outlier data; the preprocessing stage includes baseline correction, band truncation, vector normalization, and SG smoothing to generate a dataset. And in accordance with The proportion of the training set is used to divide the training set. and test set .
[0007] Step 2: Construct a convolutional-attention dual-branch fusion neural network structure. Generate high-dimensional feature representations of distillate oil through a feature concatenation module. Train the network using a joint supervision mechanism of cross-entropy loss and center loss. The composite loss function used for network training... :
[0008]
[0009] Among them, the central loss The Euclidean distance between the high-dimensional feature centers of the high-dimensional features and the high-dimensional feature centers of the distillate oil species. Indicates the first The high-dimensional features output by the neural network after processing the sample belong to the first... Class, d is the feature dimension, Represents the first in the feature space The high-dimensional feature centers of the distillate oil of this class, with a batch size of M. For loss weights; Let cross-entropy be the loss function. It is a parameter matrix The List, This is the bias term. The center loss during network training employs a mini-batch update strategy, and a hyperparameter α is set to adjust the learning rate for center updates.
[0010] Step 3, train the set The loss function is minimized in the input network using the AdamW algorithm. The model's weight parameters are adjusted to learn the nonlinear mapping relationship between spectral absorbance and distillate oil category.
[0011] Step 4, using the test set To evaluate the generalization ability of the network weight parameter model, determine whether the model's classification accuracy on the test set is greater than [missing value]. If the condition is met, proceed to step 5; otherwise, modify the hyperparameters and proceed to step 3.
[0012] Step 5: Based on the trained network, apply a structured pruning strategy to the convolutional branches. Select the scaling factor learned by the batch normalization layer in the convolutional branch. As an important indicator of the role of each convolutional kernel in the feature extraction process. At that time, the convolution kernel corresponding to that channel is pruned. If so, then retain it.
[0013] Step 6: Perform singular value decomposition on the query (Q), key (K), and value (V) projection matrices in the attention mechanism branch of the attention branch, and the weight matrix in the multilayer perceptron (MLP) to perform approximate reconstruction.
[0014] Step 6-1, for any matrix Perform singular value decomposition:
[0015]
[0016] in, yes An orthogonal matrix has column vectors called left singular vectors; yes An orthogonal matrix has column vectors called right singular vectors. yes A rectangular diagonal matrix, with diagonal elements The number of singular values is equal to the number of non-zero singular values in the matrix. rank .
[0017] Step 6-2: Determine the dimensions to retain. Its selected value satisfies:
[0018]
[0019] Step 6-3, keep the previous steps The largest singular value and its corresponding left and right singular vectors , to obtain the matrix Approximate reconstruction:
[0020]
[0021] Step 7: Combining the overall network structure, post-training quantization techniques are used to compress the weights from 32-bit floating-point numbers to 8-bit integers (INT8). All weight parameters in the network are uniformly quantized to discrete integer values with even intervals. The specific quantization mapping relationship is as follows:
[0022]
[0023] in, The quantized integer value. The floating-point value to be quantized. Indicates rounding down to the nearest integer. Scaling factor , , These represent the maximum and minimum values of the original floating-point data, respectively. Zero-point offset, And using the test set Make minor adjustments.
[0024] Beneficial effects:
[0025] This invention discloses a lightweight method for deep classification networks of distillate oils. This method employs structured pruning, low-rank decomposition, and post-training quantization to achieve comprehensive optimization in terms of parameter size, storage, and inference speed. While significantly compressing the model size, classification accuracy remains within a usable range, and inference latency is greatly reduced, enabling efficient operation on edge devices such as Jetson and FPGA. Furthermore, it eliminates the need to retrain the entire model, directly lightweighting it based on a pre-trained Conformer-1D model, resulting in a simple process and low deployment cost. Attached Figure Description
[0026] Figure 1 This is a flowchart of a lightweight method for distillate oil depth classification network according to the present invention.
[0027] Figure 2 This is a structural diagram of the one-dimensional convolutional self-attention network Conformer1D. Detailed Implementation
[0028] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.
[0029] By implementing near-infrared spectroscopy rapid analysis technology in multiple refining and chemical plants in China, the company has collected a large amount of distillate oil spectral data. Based on this company's measured database, this invention uses the NVIDIA Jetson Orin Nano edge computing platform as the model deployment and inference testing hardware to verify the effectiveness of the proposed lightweight method.
[0030] The overall system flowchart of the present invention is as follows: Figure 1 As shown, the specific steps include:
[0031] 1) Read the raw data from the distillate oil spectral database, clean and preprocess the raw labeled spectral data, and generate a dataset. Data cleaning is threshold-based cleaning of spectral anomalies; data preprocessing includes baseline correction, wavenumber band truncation, vector normalization, and SG smoothing.
[0032] 2) The architecture of the one-dimensional convolutional self-attention network Conformer1D is as follows: Figure 2 As shown, the two-dimensional convolution, global average pooling, layer normalization, and batch normalization of the traditional Conformer are replaced with one-dimensional operations; the downsampling and upsampling operations of the Feature Coupling Unit (FCU) are both replaced with one-dimensional length scaling operations. A high-dimensional feature representation of distillate oil is generated through a feature concatenation module, and a joint supervision mechanism of cross-entropy loss and center loss is used for network training.
[0033] 3) The training set Input the network and set the initial learning rate of the network. The learning rate decay rate is Batch size is 256, regularization coefficient is... The discard rate was 0.5%. , The training process consists of 100 epochs. Based on the defined loss function, the AdamW algorithm is used to minimize the loss function. .
[0034] 4) Based on the trained network, apply a structured pruning strategy to the convolutional branches. Select the scaling factor learned by the batch normalization layers in the convolutional branches. As an important indicator of the role of each convolutional kernel in the feature extraction process. At that time, the convolution kernel corresponding to that channel is pruned. If so, then retain it.
[0035] 5) Perform singular value decomposition on the query (Q), key (K), and value (V) projection matrices in the attention mechanism branch of the attention branch, and the weight matrix in the multilayer perceptron (MLP).
[0036] 6) The post-training quantization technique is used to compress the weights from 32-bit floating-point numbers to 8-bit integers (INT8). All weight parameters in the network are uniformly quantized to discrete integer values with even intervals.
[0037] To verify the beneficial effects of the present invention, multiple control experiments were set up, and the results are shown in Table 1.
[0038] Table 1 Lightweighting Results and Classification Performance Analysis
[0039]
[0040] As shown in Table 1, without lightweight processing, the number of model parameters is... The model volume is Inference delay is The accuracy rate reached While it boasts high overall predictive performance, its large model size and low inference efficiency make it unsuitable for practical deployment.
[0041] After simultaneously introducing pruning and low-rank decomposition, the number of model parameters is reduced to The model size was reduced to Inference delay shortened to Frame rate increased to At this point, the model accuracy dropped slightly. After further introducing quantization methods, the model volume decreased significantly. Inference delay reduced to Frame rate increased to The lightweighting effect is further enhanced, and the model accuracy decreases to This indicates that while lightweight operations bring higher compression rates and inference speeds, they also introduce a certain loss of accuracy.
[0042] In summary, the proposed lightweight method achieves an effective trade-off between model size, inference efficiency, and classification accuracy.
Claims
1. A lightweight method for distillate oil depth classification networks, characterized in that, Based on the trained Conformer-1D, the convolutional branch adopts a structured pruning strategy, the attention branch adopts a low-rank decomposition method, and post-training quantization technology is introduced to effectively compress the model size and accelerate inference speed. Under the premise of controllable accuracy loss, the deployment capability of the distillate oil intelligent classification model is improved. The specific steps include the following: 1) Read the raw data from the distillate oil spectral database, perform data cleaning and preprocessing on the spectra, and generate a dataset. Divide the training set and test set ; 2) Conformer-1D, a neural network structure that integrates convolution and attention branches, is constructed. High-dimensional feature representations of distillate oil are generated through a feature splicing module. The network is trained using a joint supervision mechanism of cross-entropy loss and center loss. 3) The training set The loss function is minimized in the input network using the AdamW algorithm. Update the model weights and learn the nonlinear mapping relationship between spectral absorbance and distillate oil category; 4) Through the test set To evaluate the generalization ability of the network weight parameter model, determine whether the model's classification accuracy on the test set is greater than [missing value]. If the condition is met, proceed to step 5); otherwise, modify the hyperparameters and proceed to step 3. 5) Based on the trained network, apply a structured pruning strategy to the convolutional branches, and prune low-contribution convolutional kernels according to the importance index of each convolutional kernel in the feature extraction process; 6) Perform low-rank decomposition on the core weight matrix of the attention branch and then approximate reconstruction; 7) Combining the overall network structure, post-training quantization techniques are used to compress the weights from 32-bit floating-point numbers to 8-bit integers, and the test set is used... Make minor adjustments.
2. The lightweight method for distillate oil depth classification network according to claim 1, characterized in that, The network training process employs a composite loss function constructed by combining center loss and cross-entropy loss. : Among them, the central loss The Euclidean distance between the high-dimensional feature centers of the high-dimensional features and the high-dimensional feature centers of the distillate oil species. Indicates the first The high-dimensional features output by the neural network after processing the sample belong to the first... Class, d is the feature dimension, Represents the first in the feature space The high-dimensional feature centers of the distillate oil of this class, with a batch size of M. For loss weights; Let cross-entropy be the loss function. It is a parameter matrix The List, It is a deviation item.
3. The lightweight method for distillate oil depth classification network according to claim 1, characterized in that, The central loss during network training employs a mini-batch update strategy, and hyperparameters are set accordingly. The learning rate is used to adjust the center update.
4. The lightweight method for distillate oil depth classification network according to claim 1, characterized in that, Scaling factor learned from the batch normalization layer in the convolution branch As an evaluation indicator At that time, the convolution kernel corresponding to that channel is pruned. If so, then retain it.
5. The lightweight method for distillate oil depth classification network according to claim 1, characterized in that, The core weight matrices in the Conformer-1D network are all approximated in low rank using the Singular Value Decomposition (SVD) method. Specifically, these include: the query Q, key K, and value V projection matrices in the attention mechanism branch; and the weight matrices in the Multilayer Perceptron (MLP).
6. The lightweight method for distillate oil depth classification network according to claim 5, characterized in that, For any object to be decomposed matrix Its singular value decomposition is expressed as: in, yes An orthogonal matrix has column vectors called left singular vectors; yes An orthogonal matrix has column vectors called right singular vectors. yes A rectangular diagonal matrix, with diagonal elements These are singular values, and the number of non-zero singular values is equal to the matrix. rank .
7. The lightweight method for distillate oil depth classification network according to claim 6, characterized in that, Before keeping The largest singular value and its corresponding left and right singular vectors , to obtain the matrix Approximate reconstruction:
8. The lightweight method for distillate oil depth classification network according to claim 7, characterized in that, The selected value satisfies: in, Let [the number] be the candidate cutoff order. Representation matrix Rank.
9. The lightweight method for distillate oil depth classification network according to claim 1, characterized in that, In the post-training quantization process, all weight parameters in the network are uniformly quantized to discrete integer values with uniform intervals. The specific quantization mapping relationship is as follows: in, The quantized integer value. The floating-point value to be quantized. Indicates rounding down to the nearest integer. Scaling factor This is the zero-point offset.
10. The lightweight method for distillate oil depth classification network according to claim 9, characterized in that, , for: in, , These represent the maximum and minimum values of the original floating-point data, respectively.