Method and system for predicting content of sulfides and nitrogen compounds in catalytic cracking purge flue gas
By employing various machine learning algorithms and an improved Transformer model, the problem of accurately predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking was solved, achieving high-precision and high-efficiency prediction results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- EAST CHINA UNIV OF SCI & TECH
- Filing Date
- 2023-06-28
- Publication Date
- 2026-05-12
AI Technical Summary
In existing technologies, it is difficult to accurately predict the content of sulfides and nitrogen oxides in the flue gas purified by catalytic cracking processes.
Multiple machine learning algorithms were used for feature selection and cross-validation to generate a common feature set. An improved Transformer model was then used for nonlinear fitting to predict the content of sulfides and nitrogen oxides in the flue gas purified by catalytic cracking.
It improves prediction accuracy and computational efficiency, has a wide range of applications, and is suitable for predicting the component content of flue gas purified by catalytic cracking.
Smart Images
Figure CN116798541B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial catalytic cracking technology, and more specifically, to a method and system for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking. Background Technology
[0002] Catalytic cracking (FCC) is an important petroleum refining process that converts heavy petroleum into light gasoline and other high-value products. The process involves injecting hydrocarbon feedstock into a riser reactor, where it comes into contact with a high-temperature catalyst. This breaks down the hydrocarbon feedstock into smaller molecules. The cracked hydrocarbon vapors are separated and further processed, while the deactivated high-temperature catalyst is regenerated. During regeneration, the catalyst is heated and then recycled back to the riser reactor, providing the heat required for endothermic reactions and the heat needed for feedstock evaporation.
[0003] Catalytic cracking offers several advantages: First, it can convert heavy crude oil into light gasoline and other high-value-added products, thereby increasing product output, helping to meet market demand for high-quality gasoline and petrochemical products, and improving refinery profit margins. Second, it can adapt to various types and qualities of petroleum feedstocks, including heavy and low-quality feedstocks, thus enhancing refinery flexibility and adaptability. Furthermore, catalytic cracking can improve the yield and selectivity of petroleum fractions, while reducing energy consumption and environmental pollution in petroleum processing.
[0004] Therefore, catalytic cracking technology is not only of great significance for ensuring energy security and environmental protection, but also has broad application prospects.
[0005] However, catalytic cracking processes generate large volumes of flue gas with complex compositions, including nitrogen oxides and sulfides, posing potential threats to air quality and health. Therefore, predicting and controlling the levels of sulfides and nitrogen oxides in flue gas emissions is becoming increasingly important. Summary of the Invention
[0006] The purpose of this invention is to provide a method and system for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, thereby solving the problem in the prior art that it is difficult to accurately predict the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking processes.
[0007] To achieve the above objectives, the present invention provides a method for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, comprising the following steps:
[0008] Step S1: Obtain catalytic cracking process data;
[0009] Step S2: Use various machine learning algorithms to perform feature selection on the catalytic cracking process data, evaluate the importance score of each feature, select features with the required importance scores, and generate several feature subsets.
[0010] Step S3: Cross-validate several feature subsets obtained by various machine learning algorithms to obtain feature subsets that meet the performance requirements of various machine learning algorithms, and perform weighted fusion to obtain a common feature set;
[0011] Step S4: Use the improved Transformer model to perform nonlinear fitting on the common feature set, predict the content of sulfides and nitrogen oxides in the flue gas purified by catalytic cracking, and output the results.
[0012] In one embodiment, the various machine learning algorithms in step S2 further include an extreme gradient boosting algorithm.
[0013] The evaluation function expression for the extreme gradient boosting algorithm is:
[0014]
[0015] Among them, G X G is the sum of the first-order partial derivatives of the leaf node X. Y H is the sum of the first-order partial derivatives of the leaf node Y. X H is the sum of the second-order partial derivatives of the leaf node X. Y Let y be the sum of the second-order partial derivatives of the leaf node Y, and γ and λ represent the L1 penalty term and the L2 penalty term, respectively.
[0016] In one embodiment, the various machine learning algorithms in step S2 further include the random forest algorithm.
[0017] The evaluation function expression for the random forest algorithm is as follows:
[0018]
[0019] in, This represents the average change in node splitting impurity for the i-th variable across all trees in the random forest; N is the number of categories in the random forest.
[0020] The corresponding expression is:
[0021]
[0022]
[0023] Among them, GI l and GI rThese represent the Gini indices of the two child nodes that split at node m, respectively, when variable X i It appears M times in the k-th tree.
[0024] In one embodiment, the cross-validation method includes the following steps:
[0025] The feature subset is divided into several subsets of equal size;
[0026] Choose one subset as the validation set and the remaining subset as the training set;
[0027] Train the model on the training set and evaluate the model on the validation set, recording the evaluation metric values;
[0028] Repeat the above steps until all subsets have been evaluated;
[0029] The final evaluation result of the feature subset is obtained by summing all the evaluation index values.
[0030] In one embodiment, step S3 further includes:
[0031] Based on the final evaluation results of the feature subset obtained from cross-validation, select the feature subset whose final evaluation results meet the requirements as the final feature subset;
[0032] For each machine learning algorithm, the final feature subset is weighted according to the final evaluation result.
[0033] The final feature subsets are weighted and fused according to their weights to obtain a common feature set.
[0034] In one embodiment, step S4 further includes:
[0035] The product quantization method is used to calculate the similarity between feature vectors in a set of common features.
[0036] Weights are generated between feature vectors based on similarity;
[0037] The feature vectors are weighted and fused using weights to obtain the fused feature vectors.
[0038] The prediction result is generated based on the fused feature vector.
[0039] In one embodiment, the step of calculating the similarity between feature vectors in the common feature set using the product quantization method further includes:
[0040] Divide the feature vector into several sub-vectors.
[0041] Generate an independent quantization table for each sub-vector;
[0042] Quantize each subvector using the generated quantization table;
[0043] The original feature vector is recovered from the quantized sub-vectors;
[0044] An optimized distance calculation method is used to calculate the similarity between feature vectors.
[0045] In one embodiment, the optimized distance calculation method further includes Euclidean distance calculation.
[0046] In one embodiment, step S4 further includes:
[0047] An improved Transformer model is used as a nonlinear fitting method to establish the mapping relationship between the input feature vector and the output label, which is the predicted value of the sulfide and nitrogen content in the flue gas purified by catalytic cracking.
[0048] In one embodiment, the improved Transformer model in step S4 further includes an encoding component and a decoding component:
[0049] The encoding component consists of a multi-layer encoder, and the decoding component consists of a decoder with the same number of layers;
[0050] Each encoder consists of a self-attention layer and a feedforward neural network layer.
[0051] To achieve the above objectives, the present invention provides a system for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, comprising:
[0052] Memory is used to store instructions that can be executed by the processor;
[0053] Processor, configured to execute the instructions to implement the method as described in any of the preceding embodiments.
[0054] To achieve the above objectives, the present invention provides a computer-readable medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the method described in any of the preceding claims is performed.
[0055] The present invention provides a method and system for predicting the content of sulfides and nitrides in flue gas purified by catalytic cracking. It has the advantages of high prediction accuracy, high calculation efficiency and wide applicability. It can be applied to various scenarios that require prediction of the component content of flue gas purified by catalytic cracking, and has broad application value. Attached Figure Description
[0056] The above and other features, properties and advantages of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings and embodiments, in which the same reference numerals always denote the same features, wherein:
[0057] Figure 1 A flowchart of a method for predicting the content of sulfides and nitrides in flue gas purified by catalytic cracking according to an embodiment of the present invention is disclosed.
[0058] Figure 2 A histogram showing the distribution of output variables according to an embodiment of the present invention is disclosed;
[0059] Figure 3 A schematic diagram illustrating the process of a cross-validation evaluation algorithm according to an embodiment of the present invention is disclosed;
[0060] Figure 4 A schematic diagram illustrating variable names and correlation rankings according to an embodiment of the present invention is disclosed;
[0061] Figure 5 The internal structure diagram of the encoder and decoder of the improved Transformer model according to an embodiment of the present invention is disclosed;
[0062] Figure 6 A matrix operation diagram of the multi-head attention mechanism of the improved Transformer model according to an embodiment of the present invention is disclosed;
[0063] Figure 7a An output distribution diagram showing the relationship between the predicted and actual values of nitrogen oxide content in flue gas according to an embodiment of the present invention is disclosed.
[0064] Figure 7b An output distribution diagram showing the relationship between the predicted and actual values of sulfur dioxide content in flue gas according to an embodiment of the present invention is disclosed. Detailed Implementation
[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.
[0066] Data modeling in catalytic cracking reactors involves collecting, organizing, and analyzing reactor operating data, and then using statistical, mathematical, and machine learning methods for modeling and prediction. This modeling technique helps process engineers better understand the operating status and changing patterns of the catalytic cracking reactor, enabling optimization and improvement. Through data modeling, precise process control can be achieved, improving product quality and yield, and optimizing the process. Furthermore, data modeling can improve production efficiency, reduce energy consumption and resource waste, and lower production costs.
[0067] Therefore, data modeling in catalytic cracking reactors has significant advantages and promising applications.
[0068] The data modeling techniques in the present technology have the following drawbacks:
[0069] Previous methods typically employed simple feature selection and nonlinear fitting techniques, which may not fully utilize the information in the dataset;
[0070] Simple model structures, such as linear or rule-based models, are used, which may not be able to fully capture the complex relationships in the dataset.
[0071] In real-world datasets, the number of samples in different categories can vary significantly. If left unchecked, these factors can negatively impact the accuracy of the modeling, resulting in low modeling accuracy.
[0072] The present invention proposes a method and system for predicting the content of sulfides and nitrides in flue gas purified by catalytic cracking. The modeling method used for FCC riser reactors is a data modeling method.
[0073] Figure 1 A flowchart of a method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas according to an embodiment of the present invention is disclosed, such as... Figure 1 As shown, the method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas proposed in this invention includes the following steps:
[0074] Step S1: Obtain catalytic cracking process data;
[0075] Step S2: Use various machine learning algorithms to perform feature selection on the catalytic cracking process data, evaluate the importance score of each feature, select features with the required importance scores, and generate several feature subsets.
[0076] Step S3: Cross-validate several feature subsets obtained by various machine learning algorithms to obtain feature subsets that meet the performance requirements of various machine learning algorithms, and perform weighted fusion to obtain a common feature set;
[0077] Step S4: Use the improved Transformer model to perform nonlinear fitting on the common feature set, predict the content of sulfides and nitrogen oxides in the flue gas purified by catalytic cracking, and output the results.
[0078] This paper proposes a method and system for predicting the content of sulfides and nitrogen oxides in flue gas from catalytic cracking. In the feature selection stage, multiple machine learning algorithms, such as Random Forest and XGBoost, are used to effectively evaluate the importance score of each feature and select a high-quality feature subset. This maintains or improves modeling accuracy while using fewer features. Cross-validation combined with Random Forest and XGBoost algorithms more accurately evaluates the effects of feature selection and nonlinear fitting, selecting the best-performing feature subset. Optimal feature combinations can be selected, thus improving modeling accuracy. An improved Transformer model is used to perform nonlinear fitting by calculating the similarity between features. An improved product quantization method optimizes the computational efficiency and accuracy of similarity calculation, allowing for better integration of multiple features to enhance modeling accuracy.
[0079] The following describes in detail the steps of the method for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking proposed in this invention. It should be understood that, within the scope of this invention, the above-described technical features of this invention and the technical features specifically described below (such as in the embodiments) can be combined and correlated with each other to constitute preferred technical solutions.
[0080] Step S1: Obtain catalytic cracking process data.
[0081] Furthermore, data on the catalytic cracking process are obtained, and the original data is filled in, outliers are removed, and standardized preprocessing is performed to obtain training samples and test samples, which are used to train the improved Transformer model and test the model performance, respectively.
[0082] Standardization preprocessing transforms raw data to conform to specific standards or ranges, better adapting it to the needs of the model. In this invention, the raw data of the catalytic cracking process undergoes standardization preprocessing. The dataset is standardized to have zero mean and unit variance. The standardization method used is Z-score standardization, which transforms the data into a distribution with zero mean and unit variance by subtracting the mean and dividing by the standard deviation. Standardization preprocessing aims to eliminate dimensional differences between data points, making them more similar in scale. This helps improve the convergence speed and performance of the model and reduces the varying degrees of influence of different features on the model.
[0083] After standardization preprocessing, the original dataset underwent steps such as imputation, outlier removal, and standardization to obtain processed training and testing samples. These samples will be used to train the improved Transformer model and evaluate its performance, enabling the prediction of sulfide and nitrogen oxide content in catalytic cracking flue gas.
[0084] In this embodiment, the data source used is an actual oil refining catalytic cracking unit in production, with a collection cycle of 1 year, and the sampling time interval is set to 10 minutes according to the actual operating characteristics of the unit.
[0085] More specifically, the data source used comes from the stable production data of the catalytic cracking unit of a petrochemical company's refinery A during three time periods from November 2020 to November 2021.
[0086] The dataset contains 52 operating variables and properties, including overflow hopper fluidizing steam flow rate, hot wax oil inlet flow rate, raw material atomizing steam flow rate, feed flow rate, flue gas temperature, reactor pressure, reaction temperature, and regenerator pressure, as well as the content of nitrogen oxides and sulfides in the flue gas.
[0087] Each dataset has corresponding upper and lower limits for its input variables, which are determined by the device operator.
[0088] If an input variable exceeds its upper or lower limit, it is set to the corresponding upper or lower limit value.
[0089] Since the data collected by this method covers almost a complete operating cycle of the catalytic cracking unit, the collected data can well represent the operating characteristics of the unit.
[0090] In this embodiment, training samples and test samples are divided according to time sequence.
[0091] The first 70% of the samples are used as training samples, and the last 30% of the samples are used as test samples (training samples + test samples = total samples).
[0092] Since the model predicts the sulfide and nitrogen oxide content in the flue gas purified by catalytic cracking, the emission content of the purified flue gas is calculated as the sulfide and nitrogen oxide content in the purified flue gas using the following formula:
[0093]
[0094] In the formula: y product F represents the product yield. product This indicates the product flow rate, and this indicates the total feed flow rate.
[0095] In this embodiment, the three output variables of the device—the percentage of sulfur dioxide, nitrogen oxides, and carbon dioxide in the flue gas—are calculated using the above formula and taken as target variables.
[0096] Figure 2 A histogram showing the distribution of output variables according to an embodiment of the present invention is disclosed, such as... Figure 2As shown, the distribution and range of output variables, namely nitrogen oxides and carbon dioxide, are displayed more intuitively.
[0097] Step S2: Use various machine learning algorithms to perform feature selection on the catalytic cracking process data, evaluate the importance score of each feature, select features with the required importance scores, and generate several feature subsets.
[0098] In this embodiment, the machine learning algorithms include extreme gradient boosting (XGBoost) and random forest algorithms.
[0099] For the dataset after preprocessing in step S1, the Random Forest and XGBoost algorithms are used for feature selection. Random Forest and XGBoost algorithms can automatically identify the most relevant features from the original data, improving the prediction accuracy of subsequent models.
[0100] By using the Random Forest algorithm and the XGBoost algorithm to select features from the data, the importance score of each feature is obtained. Features that meet the importance score requirements are selected to generate several high-quality feature subsets for use in subsequent models, thereby improving the prediction accuracy of subsequent models.
[0101] Extreme Gradient Boosting (XGBoost) is a commonly used tree-based machine learning algorithm based on gradient decision trees, which has been widely applied to data classification and fitting in various fields.
[0102] In this embodiment, the XGBoost algorithm selects features, and further includes the following steps:
[0103] First, define the most basic unit decision tree for the XGBoost algorithm:
[0104] f t (x)=ω k(x) ;
[0105] In the formula, ω∈R T The weight vector representing the leaf node, k:R d →1,2,...,Tk represent mapping relationships.
[0106] Number complexity It can be defined as:
[0107]
[0108] In the formula, N represents the number of leaf nodes. γ represents the L2 norm of the weight vector, and γ and λ represent the L1 and L2 penalty terms, respectively.
[0109] Furthermore, let the loss function formula for the algorithm be:
[0110]
[0111] In the formula, For loss function, For predicted values, The regularization term represents the complexity of the number.
[0112] Model prediction formula:
[0113]
[0114] Substituting the formula for the model prediction into the formula for the loss function, we get:
[0115]
[0116] Expanding the above equation using Taylor's formula yields:
[0117]
[0118] In the formula g i and h i Representing the loss function pairs respectively The first and second derivatives.
[0119] We can obtain:
[0120]
[0121] Furthermore, expanding the regularization term yields:
[0122]
[0123] Since the first t-1 trees of the current tree are fixed and do not participate in subsequent calculations, substituting the basic unit decision tree formula into the loss function formula yields:
[0124]
[0125] Define the sum of the first-order partial derivatives and the sum of the second-order partial derivatives of leaf node j as... Substitute into the above equation and adjust ω j Find the partial derivative, and when it equals 0, we get... and then:
[0126] G j +(H j +λ)ω j =0
[0127]
[0128] Finally, the evaluation function of the extreme gradient boosting algorithm is obtained, and its corresponding expression is as follows:
[0129]
[0130] Among them, G X G is the sum of the first-order partial derivatives of the leaf node X. Y H is the sum of the first-order partial derivatives of the leaf node Y. X H is the sum of the second-order partial derivatives of the leaf node X. Y Let y be the sum of the second-order partial derivatives of the leaf node Y, and γ and λ represent the L1 penalty term and the L2 penalty term, respectively.
[0131] After obtaining the evaluation function of the XGBoost algorithm, features are selected using the evaluation function, the data is trained, the importance score of each feature is obtained, and features that meet the importance score requirements are selected to generate a high-quality feature subset for subsequent model use.
[0132] In this embodiment, the random forest algorithm selects features, and further includes the following steps:
[0133] Each tree in a random forest is a binary tree for classification, with the root node containing all training samples. Assume the variables are X1, X2, ..., X... M We need to calculate the importance scores (VIM) of M variables.
[0134] Statistic The Gini exponent represents the average change in node splitting impurity across all trees in a random forest for the i-th variable. The formula for calculating the Gini exponent is:
[0135]
[0136] K is the number of features in the dataset. This is a probability estimate of node m belonging to the k-th feature.
[0137] VARIABLE X i The importance of node m or the change in the Gini index before and after node m forks is as follows:
[0138]
[0139] Among them, GI m GI represents the Gini index at node m. l and GI r These represent the Gini exponents of child nodes l and r, which split at node m, respectively, when variable X iIf a variable appears M times in the k-th tree, then the importance of that variable in that tree can be expressed as:
[0140]
[0141] That is, variable X i The Gini importance in the entire random forest is represented as:
[0142]
[0143] Where N is the number of random forest classifications.
[0144] After obtaining the evaluation function of the random forest algorithm, features are selected using the evaluation function, the data is trained, the importance score of each feature is obtained, and features that meet the importance score requirements are selected to generate a high-quality feature subset for subsequent model use.
[0145] Step S3: Cross-validate several feature subsets obtained by various machine learning algorithms to obtain feature subsets that meet the performance requirements of various machine learning algorithms, and perform weighted fusion to obtain a common feature set.
[0146] To evaluate the performance of the feature subsets obtained by the Random Forest and XGBoost algorithms, cross-validation was used in this embodiment. Cross-validation can effectively estimate the model's performance on unknown data.
[0147] Figure 3 A schematic diagram illustrating the process of a cross-validation evaluation algorithm according to an embodiment of the present invention is shown, such as... Figure 3 In the illustrated embodiment, the performance of the feature subsets obtained by the Random Forest algorithm and the XGBoost algorithm is evaluated using the cross-validation method. The best-performing feature subsets are selected from the feature subsets obtained by the Random Forest algorithm and the XGBoost algorithm, respectively. The two sets of optimal feature subsets evaluated by the cross-validation method are then fused using linear weighting to obtain a common feature set (optimal training set), which can be used as features in nonlinear fitting.
[0148] Furthermore, the cross-validation method includes the following steps:
[0149] The feature subset is divided into several subsets of equal size;
[0150] Choose one subset as the validation set and the remaining subset as the training set;
[0151] Train the model on the training set and evaluate the model on the validation set, recording the evaluation metric values;
[0152] Repeat the above steps until all subsets have been evaluated;
[0153] The final evaluation result of the feature subset is obtained by summing all the evaluation index values.
[0154] For example, first, the feature subset is randomly divided into k subsets of equal size, of which k-1 subsets are used as the training set and the remaining 1 subset is used as the validation set.
[0155] For each subset, repeat the following steps:
[0156] a. Use the current subset as the validation set and the remaining k-1 subsets as the training set;
[0157] b. Train the model on the training set and evaluate the model on the validation set, recording the evaluation metric values.
[0158] Repeat the above steps until all k subsets have been evaluated, and obtain the evaluation index values for the k subsets.
[0159] Then, the evaluation metrics of all k subsets are summed to obtain the final evaluation result of the feature subset.
[0160] Based on the final evaluation results of the feature subsets obtained from cross-validation, select the feature subsets whose performance meets the requirements.
[0161] Furthermore, based on the final evaluation results of the feature subsets obtained from cross-validation, the feature subset with the best final evaluation results is selected as the final feature subset;
[0162] For each machine learning algorithm, the final feature subset is weighted according to the final evaluation result.
[0163] The final feature subsets are weighted and fused according to their weights to obtain a common feature set.
[0164] In this embodiment, two feature subsets with the best final evaluation results are obtained by cross-selection from the feature subsets obtained by the Random Forest algorithm and the XGBoost algorithm, respectively. A weight is assigned to each feature subset with the best performance, and each feature subset is multiplied by its corresponding weight. Then, they are weighted and summed to obtain the final feature fusion result, i.e., the common feature set.
[0165] Furthermore, the weights can be determined based on the evaluation performance of the optimal feature subset. Feature subsets with better performance can be assigned higher weights to enhance their influence in the final feature fusion. Linear weighted fusion is then performed on each feature subset.
[0166] Furthermore, cross-selection obtains feature subsets that meet performance requirements. Performance doesn't necessarily have to be optimal, but rather satisfy certain conditions. In this case, there can be more than one corresponding feature subset. Linear weighted fusion is then performed on each feature subset, further including the following steps:
[0167] Suppose we have k feature subsets, and their performance in cross-validation is {p1, p2, ..., p...} k}, p2, ..., p k Then a weight {w1, w2, ..., w} can be assigned to each feature subset. s},w2,...,w s}
[0168] The weights can be calculated using the following formula:
[0169]
[0170] Among them, w i This represents the weight of the i-th feature subset;
[0171] p i This represents the performance of the i-th feature subset in cross-validation;
[0172] This represents the sum of the performance of all feature subsets in cross-validation;
[0173] The fused features are normalized to ensure that the numerical ranges of different feature subsets are consistent.
[0174] The set of common features after fusion is the feature fusion result obtained through linear weighted combination. Figure 4 A schematic diagram illustrating variable names and correlation rankings according to an embodiment of the present invention is shown, such as... Figure 4 In the illustrated embodiment, the common feature set includes two best-performing feature subsets obtained from the Random Forest and XGBoost algorithms. The variable names and relevance rankings for feature selection are as follows: Figure 4 As shown.
[0175] Cross-validation can balance the contributions of different feature subsets to the final model based on their performance and importance. The weights can be adjusted according to the actual situation and the needs of the problem to achieve better prediction performance. Then, a set of common features is obtained, which can be used as features in nonlinear fitting.
[0176] Using a linearly weighted fusion strategy can comprehensively consider the contributions of different feature subsets while avoiding over-reliance on a particular feature subset.
[0177] Meanwhile, the weights can be automatically determined based on the performance of the feature subsets in cross-validation, without the need for manual specification, which can reduce human intervention and improve the automation of feature selection.
[0178] In addition, using cross-validation to evaluate each feature subset before feature fusion can better estimate the model's performance on unknown data, thereby selecting the best-performing feature subset and improving the prediction accuracy of the entire feature set.
[0179] Step S4: Use the improved Transformer model to perform nonlinear fitting on the common feature set, predict the content of sulfides and nitrogen oxides in the flue gas purified by catalytic cracking, and output the results.
[0180] The Transformer model is a type of neural network that learns context and thus meaning by tracking relationships in sequential data.
[0181] An improved Transformer model is used to receive feature vector input, calculate the similarity between features, and use a product quantization method to optimize computational efficiency and improve the accuracy of similarity calculation between features.
[0182] To address the issue that while traditional recurrent neural network models perform well in sequence modeling, their recursive structure makes parallel computation difficult, resulting in slow training speeds, this invention employs an improved Transformer model for nonlinear fitting and uses product quantization to quantize high-dimensional feature vectors to optimize computational efficiency and improve the accuracy of similarity calculation between features. Parallel computation is used to avoid the temporal dependency problem in recursive computation, thereby accelerating the training speed.
[0183] In this embodiment, an improved Transformer model is used for nonlinear fitting. Furthermore, a product quantization method is introduced to quantize the high-dimensional feature vector to optimize computational efficiency and improve the accuracy of similarity calculation between dimensional features. This allows for better capture of the dependencies between different positions in the input sequence to obtain prediction results.
[0184] Furthermore, the nonlinear fitting of the common feature set using the improved Transformer model includes the following steps:
[0185] The product quantization method is used to calculate the similarity between feature vectors in a set of common features.
[0186] Weights are generated between feature vectors based on the calculated similarity.
[0187] The generated weights are used to perform weighted fusion of the input feature vectors to obtain the fused feature vector;
[0188] The prediction result is generated based on the fused feature vector.
[0189] Furthermore, the step of calculating the similarity between feature vectors in the common feature set using the product quantization method further includes:
[0190] The partitioning operation divides the input feature vector into several sub-vectors;
[0191] The generation operation generates an independent quantization table for each subvector;
[0192] The quantization operation uses the generated quantization table to quantize each subvector;
[0193] The reconstruction operation recovers the original feature vector by combining the quantized sub-vectors.
[0194] An optimized distance calculation method is adopted to calculate the similarity between feature vectors, thereby improving the accuracy and efficiency of similarity calculation between feature vectors.
[0195] The product quantization method is used to calculate similarity. The main idea is to decompose a high-dimensional vector into multiple low-dimensional sub-vectors, then cluster each sub-vector, and finally concatenate the clustering results to form a new low-dimensional vector. This new low-dimensional vector effectively reduces computation while preserving information from the original vector, thus achieving efficient similarity calculation.
[0196] Optimized distance calculation methods, including Euclidean distance calculation methods.
[0197] Furthermore, the similarity is calculated using the product quantization method, with the following specific steps:
[0198] Divide the original high-dimensional vector into multiple sub-vectors: Divide the original vector into m sub-vectors, each sub-vector having a dimension of d / m.
[0199] Cluster each subvector: Each subvector is clustered into k clusters, and each cluster corresponds to a cluster center.
[0200] Map each subvector to the cluster center it belongs to: for each subvector, map it to the cluster center closest to it;
[0201] The cluster center numbers of each sub-vector are concatenated to form a new low-dimensional vector with dimensions m*log2k.
[0202] Calculate the distance between new low-dimensional vectors. For two new low-dimensional vectors, use the Euclidean distance method to calculate the distance between them, thereby obtaining their similarity.
[0203] The formula for Euclidean distance is as follows:
[0204]
[0205] Among them, A i and B i Let represent the i-th component of vectors A and B respectively, and n represent the dimension of the vectors.
[0206] The improved Transformer model further includes encoding and decoding components:
[0207] The encoding component consists of a multi-layer encoder;
[0208] The decoding component consists of decoders with the same number of layers.
[0209] The encoder is used to transform the input sequence into a hidden representation, and the decoder transforms the hidden representation into an output sequence.
[0210] Each encoder consists of a self-attention layer and a feedforward neural network layer.
[0211] In the self-attention layer, each vector in the input sequence is compared with other vectors to calculate a similarity score. Vectors with higher scores have a greater impact on the representation of the current vector, thereby modeling the dependencies between different positions and better capturing the dependencies between different positions in the input sequence to obtain prediction results.
[0212] Fully connected feedforward neural networks are used to process the output of the attention layer to obtain a better representation.
[0213] The input data is processed through the self-attention mechanism of the self-attention layer to calculate the autocorrelation of the input feature vector.
[0214] In addition, the improved Transformer model uses positional encoding to add positional information to the input feature vector.
[0215] The multi-head attention mechanism of the self-attention layer is used to process information from different feature spaces simultaneously.
[0216] In this embodiment, an improved Transformer model is used as a nonlinear fitting method to establish a mapping relationship between input features and output labels, wherein the output labels are the predicted values of sulfide and nitrogen oxide content in the flue gas purified by catalytic cracking.
[0217] Specifically, the input features are represented as a sequence of vectors, each vector representing a feature, and the output label is a continuous numerical value or vector.
[0218] In the process of establishing the mapping relationship, the improved Transformer model transforms the input feature sequence into a set of feature vectors, and then feeds these feature vectors into a multi-layer neural network for nonlinear transformation and feature extraction.
[0219] In each layer of the neural network, a self-attention mechanism and a forward propagation network are used to weight and transform the features in order to better capture the relationships and nonlinear changes between features;
[0220] Through multi-layered nonlinear transformations and feature extraction, the Transformer model can map input features to output labels and has good prediction accuracy and generalization ability.
[0221] Figure 5 A diagram illustrating the internal structure of the encoder-decoder of an improved Transformer model according to an embodiment of the present invention is disclosed, such as... Figure 5 As shown, each encoder consists of two sub-layers: a Self-Attention layer and a Position-wise Feed Forward Network (FFN).
[0222] For self-attention, the three matrices Q (Query), K (Key), and V (Value) all come from the same input. First, the dot product between Q and K is calculated; to prevent the result from being too large, it is divided by... Where d k Let V be the dimension of the key vector. Then, the softmax operation is used to normalize the result into a probability distribution, and multiplying it by matrix V yields the weighted summation representation.
[0223] Figure 6 A matrix operation graph of the multi-head attention mechanism of the improved Transformer model according to an embodiment of the present invention is disclosed, such as... Figure 6 As shown, the improved Transformer multi-head attention mechanism used in this invention can be represented as follows:
[0224]
[0225] In this embodiment, a multi-head attention mechanism is added, which maps Q, K, and V through h different linear transformations;
[0226] Then, the different attentions are spliced together and subjected to another linear transformation.
[0227] The entire calculation process can be represented as:
[0228] MultiHead(Q,K,V)=Concat(head1,...,headh W O
[0229]
[0230] in, and
[0231] In multi-head attention, different Query, Key, and Value weight matrices are maintained separately for each group of attention, thus obtaining different Query, Key, and Value matrices.
[0232] As mentioned earlier, multiply X by W Q W K and W V The matrix yields the Query, Key, and Value matrices.
[0233] By performing multiple self-attention calculations using different weight matrices, multiple different Z matrices can be obtained.
[0234] like Figure 6 As shown, the multi-head attention mechanism mainly consists of three parts: query, key and value, and attention score calculation and weighted summation.
[0235] In a multi-head attention mechanism, the input features are first divided into multiple sub-features, and then each sub-feature is used to compute multiple attention heads.
[0236] Each attention head has its own query, key, and value, as well as an attention score calculation and weighted summation process.
[0237] Since the feedforward neural network layer receives a single matrix instead of the multiple matrices mentioned above, a method is needed to integrate these multiple matrices into a single matrix.
[0238] The specific method for feedforward neural network layers is as follows:
[0239] Given multiple matrices {Z0, Z1, ..., Z... n splicing;
[0240] The concatenated matrix and a weight matrix W o Multiply;
[0241] The final matrix Z is obtained, and this matrix will be input into the feedforward neural network layer.
[0242] A positional feedforward network is a fully connected feedforward network in which each word at each position passes through the same identical feedforward neural network.
[0243] The position feedforward network consists of two linear transformations, i.e., two fully connected layers. The activation function of the first fully connected layer is the ReLU activation function.
[0244] A location feedforward network can be represented as:
[0245] FFN(x) = max(0, xW1+b1)W2+b2
[0246] Although the fully connected feedforward network structure is the same in each encoder and decoder, the parameters are not shared.
[0247] There is one detail in the encoder structure that needs attention:
[0248] Each sublayer of each encoder has a residual connection, and then a layer normalization operation is performed. The entire calculation process can be represented as:
[0249] sub_layer_output=LayerNorm(x+SubLayer(x))
[0250] Furthermore, the improved Transformer model information is as follows:
[0251] The input dimension of the encoder and decoder of the Transformer model is set to 512;
[0252] The bullish attention level is set to 8.
[0253] The network is configured with 6 layers;
[0254] The feedforward network has a dimension of 2048;
[0255] The model learning rate is set to 0.001;
[0256] The maximum number of training iterations is 100.
[0257] Use ReLU as the activation function;
[0258] The Dropout parameter is set to 0.2.
[0259] Based on the above-described improved Transformer model implementation process, the prediction results of the catalytic cracking target variable using the improved Transformer model on the test set dataset are shown in Table 1:
[0260] Table 1 Prediction results of target variables for catalytic cracking
[0261] Catalytic cracking output variables MAE RMSE Carbon dioxide percentage content in flue gas 2.233 2.610 Sulfur dioxide percentage content in flue gas 18.098 18.535 Nitrogen oxides percentage content in flue gas 18.566 21.685
[0262] Table 1 shows the prediction errors for different output variables, where MAE (mean absolute error) and RMSE (root mean square error) are two commonly used metrics for evaluating model performance.
[0263] As can be seen, the MAE and RMSE values of each variable are relatively small, indicating that the model's prediction error is small.
[0264] However, it should be noted that the MAE and RMSE values are different for different variables, indicating that the model's predictive performance varies for different variables.
[0265] Secondly, MAE and RMSE have different application scenarios. MAE is not sensitive to outliers, while RMSE is more sensitive to outliers.
[0266] In Table 1, since there are no obvious outliers, the two indicators perform similarly, and it can be seen that the relative prediction error of the model is small, which also indicates that the model's prediction performance is relatively reliable.
[0267] To provide a more intuitive and clearer view of the model's predictive performance, an output distribution diagram of the improved Transformer model was drawn based on this experiment, showing the relationship between the predicted and actual product content in the flue gas purified by the two catalytic cracking processes.
[0268] Figure 7a and Figure 7b Output distribution diagrams showing the relationship between predicted and actual values of nitrogen oxide and sulfur dioxide content in flue gas according to an embodiment of the present invention are presented, such as... Figure 7a and Figure 7b As shown, the predicted curves of the products in the flue gas purified by the two catalytic cracking processes are in good agreement with the actual curves, with a high degree of overlap and trend, indicating that the model has good predictive performance and relatively small prediction error.
[0269] In summary, the data provided in the table demonstrates that the improved Transformer model's prediction method for sulfide and nitrogen oxide content in catalytic cracking flue gas exhibits good predictive performance, relatively small prediction error, and a certain degree of reliability.
[0270] This invention proposes a device for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking. It can be used to predict the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, as well as the content of nitrogen oxides and sulfides in flue gas purified by catalytic cracking.
[0271] This invention proposes a device for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, comprising:
[0272] The preprocessing module is used to acquire data information from the catalytic cracking process, preprocess the data information, and obtain training samples and test samples.
[0273] The machine learning algorithm module is used to employ machine learning algorithms such as random forest and XGBoost to obtain the importance score of each feature and select a high-quality feature subset.
[0274] The cross-validation method module is used to evaluate the performance of the two sets of features through cross-validation, select the best-performing feature subset, and fuse the feature subsets obtained by the random forest and XGBoost algorithms to obtain a common feature set, which can be used as features in nonlinear fitting.
[0275] The similarity calculation module is used to calculate the similarity between features. It adopts an improved product quantization method to optimize the calculation efficiency and improve the accuracy of similarity calculation between features.
[0276] The output module uses an improved Transformer model for nonlinear fitting to predict the content of sulfides and nitrogen oxides in the purified flue gas and process the output data.
[0277] In this embodiment, the training samples and test samples are used to train the model and test the model performance, respectively.
[0278] This invention proposes a prediction system for the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking. The prediction system may include an internal communication bus, a processor, a read-only memory (ROM), a random access memory (RAM), a communication port, and a hard disk. The internal communication bus enables data communication between the components of the prediction system. The processor can perform judgments and issue prompts. In some embodiments, the processor may consist of one or more processors.
[0279] The communication port enables data transmission and communication between the sulfide and nitrogen oxide content prediction system in catalytic cracking flue gas and external input / output devices. In some embodiments, the sulfide and nitrogen oxide content prediction system can send and receive information and data from a network via the communication port. In some embodiments, the sulfide and nitrogen oxide content prediction system can also transmit and communicate with external input / output devices via wired connections through the input / output terminals.
[0280] A system for predicting the sulfide and nitrogen oxide content in flue gas purified by catalytic cracking can also include different types of program storage units and data storage units, such as hard disks, read-only memory (ROM), and random access memory (RAM), capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor. The processor executes these instructions to implement the main part of the method. The results of the processor processing are transmitted to an external output device via a communication port and displayed on the user interface of the output device.
[0281] For example, the implementation process file of the above-mentioned prediction system for the content of sulfides and nitrogen oxides in the flue gas purified by catalytic cracking can be a computer program, stored on a hard disk, and recorded in a processor for execution to implement the method of the present invention.
[0282] When the implementation process document of the method for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking is a computer program, it can also be stored as an article of manufacture in a computer-readable storage medium. For example, computer-readable storage media can include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical discs (e.g., compact discs (CDs), digital multifunction discs (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memory (EPROM), cards, sticks, key drives). Furthermore, the various storage media described herein can represent one or more devices and / or other machine-readable media used for storing information. The term "machine-readable medium" can include, but is not limited to, wireless channels and various other media (and / or storage media) capable of storing, containing, and / or carrying code and / or instructions and / or data.
[0283] The method, apparatus, system, and medium for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas proposed in this invention can effectively predict the emission content of sulfides and nitrogen oxides in the flue gas, playing an important role in reducing environmental pollution and protecting human health.
[0284] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.
[0285] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.
[0286] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and arts. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.
[0287] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.
[0288] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.
[0289] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.
[0290] The above embodiments are provided for those skilled in the art to implement or use the present invention. Those skilled in the art can make various modifications or changes to the above embodiments without departing from the inventive concept of the present invention. Therefore, the protection scope of the present invention is not limited to the above embodiments, but should be the maximum scope that conforms to the innovative features mentioned in the claims.
Claims
1. A method for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, characterized in that, Includes the following steps: Step S1: Obtain catalytic cracking process data; Step S2: Use various machine learning algorithms to perform feature selection on the catalytic cracking process data, evaluate the importance score of each feature, select features with the required importance scores, and generate several feature subsets. Step S3: Cross-validate several feature subsets obtained by various machine learning algorithms to obtain feature subsets that meet the performance requirements of various machine learning algorithms, and perform weighted fusion to obtain a common feature set; Step S4: Use the improved Transformer model to perform nonlinear fitting on the common feature set, predict the content of sulfides and nitrides in the flue gas purified by catalytic cracking, and output the results. Step S3 further includes: Based on the final evaluation results of the feature subsets obtained from cross-validation, select the feature subsets whose final evaluation results meet the requirements as the final feature subsets; For each machine learning algorithm, the final feature subset is assigned weights according to the final evaluation result; The final feature subsets are weighted and fused according to their weights to obtain a common feature set; Step S4 further includes: The product quantization method is used to calculate the similarity between feature vectors in a set of common features. Weights are generated between feature vectors based on similarity; The feature vectors are weighted and fused using weights to obtain the fused feature vectors. The prediction result is generated based on the fused feature vector; The step of calculating the similarity between feature vectors in the common feature set using the product quantization method further includes: Divide the feature vector into several sub-vectors; Generate an independent quantization table for each sub-vector; Quantize each subvector using the generated quantization table; The original feature vector is recovered from the quantized sub-vectors; An optimized distance calculation method is used to calculate the similarity between feature vectors.
2. The method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas according to claim 1, characterized in that, The various machine learning algorithms in step S2 further include extreme gradient boosting algorithms. The evaluation function expression for the extreme gradient boosting algorithm is: ； in, G X Let X be the sum of the first-order partial derivatives of the leaf node X. G Y Let Y be the sum of the first-order partial derivatives of the leaf node Y. H X Let X be the sum of the second-order partial derivatives of the leaf node X. H Y Let Y be the sum of the second-order partial derivatives of the leaf node Y. and Represent Penalty items and Penalties.
3. The method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas according to claim 1, characterized in that, The various machine learning algorithms in step S2 further include the random forest algorithm. The evaluation function expression for the random forest algorithm is as follows: ； in, Indicates the first i The average change in node splitting impurity across all trees in a random forest; N The number of categories in the random forest; The corresponding expression is: ； ； in, M For variables In the k The number of times it appears on a tree; Represents at the node m of Gini index; and They represent the nodes respectively m Split child nodes l and child nodes r of Gini index.
4. The method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas according to claim 1, characterized in that, The cross-validation method includes the following steps: The feature subset is divided into several subsets of equal size; Choose one subset as the validation set and the remaining subset as the training set; Train the model on the training set and evaluate the model on the validation set, recording the evaluation metric values; Repeat the above steps until all subsets have been evaluated; The final evaluation result of the feature subset is obtained by summing all the evaluation index values.
5. The method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas according to claim 1, characterized in that, Step S4 further includes: An improved Transformer model is used as a nonlinear fitting method to establish the mapping relationship between the input feature vector and the output label, which is the predicted value of the sulfide and nitrogen content in the flue gas purified by catalytic cracking.
6. The method for predicting the content of sulfides and nitrogen oxides in catalytic cracking flue gas according to claim 1, characterized in that, The improved Transformer model in step S4 further includes an encoding component and a decoding component: The encoding component consists of a multi-layer encoder, and the decoding component consists of a decoder with the same number of layers; Each encoder consists of a self-attention layer and a feedforward neural network layer.
7. A system for predicting the content of sulfides and nitrogen oxides in flue gas purified by catalytic cracking, comprising: Memory is used to store instructions that can be executed by the processor; A processor for executing the instructions to implement the method as described in any one of claims 1-6.