A method for optimizing a dynamic model based on a timing-enhanced MLP

Abstract

The application discloses a kind of kinetics model optimization methods based on timing enhancement MLP, belong to kinetics model construction technical field, the method includes: obtaining the kinetics experimental data of sample, and construct five kinds of time series features based on experimental data;The integral form of time series features and mechanism model is combined, to build enhanced feature matrix and timing enhancement MLP prediction model;The optimization objective function based on Huber loss and L2 regularization is constructed, and non-negative constraint is applied to mechanism model weight;L-BFGS-B algorithm is used to quickly solve the optimization objective function, output optimal weight vector, obtain final dynamics prediction model.The kinetics model after optimization of the application, by introducing first derivative, second derivative and sliding window statistical characteristics, the representation ability of model to local kinetics evolution characteristics is enhanced, and optimization algorithm is also used to realize the rapid convergence of algorithm, significantly improve the prediction accuracy, numerical stability and generalization performance of model.

Description

Technical Field

[0001] This invention relates to the field of dynamic model construction technology, and in particular to a dynamic model optimization method based on time-enhanced MLP. Background Technology

[0002] In the field of dynamics modeling, accurately fitting and predicting dynamic evolution processes is crucial. Traditional machine learning models (such as multilayer perceptrons, MLPs) typically use only the transformation rate α as a single input variable when approximating dynamics models. (See attached reference.) Figure 1 In a traditional multilayer perceptron neural network structure, the input layer typically has one neuron to input the temperature at different heating rates; the number of neurons in the hidden layer is consistent with the number of dynamic models used in the master curve method; and the output layer has one neuron, with the output being the conversion rate.

[0003] However, in real-world physics, the conversion rate α is a continuous function of temperature T, and its derivative contains important dynamic evolution characteristics. Existing MLP models only utilize the zero-order information α(T), neglecting these higher-order evolutionary information, which limits their function approximation capabilities.

[0004] Furthermore, dynamic experimental data are often inevitably accompanied by experimental noise or local outliers during the acquisition process. If the traditional mean squared error (MSE) is directly used as the loss function, the model is highly susceptible to interference from outliers, leading to fitting bias. At the same time, without effective regularization constraints, the condition number of the model is prone to drastic fluctuations during numerical solution, which in turn leads to problems such as slow algorithm convergence and overfitting, seriously affecting the model's generalization ability and numerical stability.

[0005] Therefore, how to improve the fitting accuracy, noise robustness, and numerical stability of dynamic models is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a dynamic model optimization method based on time-enhanced MLP, which aims to solve the problems of existing dynamic models being easily disturbed by outliers, lacking effective regularization constraints, and the algorithms being prone to slow convergence and overfitting.

[0007] To achieve the above objectives, the present invention provides the following solution: A dynamic model optimization method based on time-enhanced MLP includes the following steps: Step 1: Obtain experimental data of the sample during the heating oxidation process. The experimental data includes temperature sequences and corresponding conversion rate sequences at different heating rates. Step 2: Construct time series features based on the experimental data. The time series features include first derivative features, second derivative features, moving average features, standard deviation features, and reaction rate features based on the time scale. Step 3: Perform feature fusion between the time series features and the integral form of the dynamic mechanism model to construct an enhanced feature matrix, and construct a prediction model based on the enhanced feature matrix; Step 4: Construct an optimization objective function, which includes a Huber loss function term and an L2 regularization term, and apply non-negative constraints to the weights of the time series features in the prediction model; Step 5: The L-BFGS-B algorithm is used to iteratively solve the optimization objective function, output the optimal time series feature weights, and obtain the final dynamic prediction model.

[0008] Furthermore, in step 2, the first derivative characteristic is used to characterize the reaction rate, and its calculation formula is as follows: ; in, T i Indicates temperature. α i Indicates conversion rate. T j This represents the temperature of data point j.

[0009] Furthermore, the second derivative is used to characterize the reaction acceleration rate, and its calculation formula is as follows: .

[0010] Furthermore, the specific method for calculating the moving average characteristic is as follows: define the moving window size as... N w The moving average is For each data point j, calculate the statistics within the window: .

[0011] Furthermore, the standard deviation feature is used to evaluate the local volatility and dispersion of data, and its specific calculation formula is as follows: ; In the formula, This represents the average conversion rate.

[0012] Furthermore, the specific formula for calculating the reaction rate is as follows: ; in, β This indicates the rate of temperature increase.

[0013] Furthermore, the enhanced feature matrix constructed in step 3 is as follows: ; in, , which is the integral form of the dynamic model; , represents the set of 5 types of time series features calculated in step 2; The prediction model is represented as follows: ; Where w is the weight of the time feature.

[0014] Furthermore, the optimization objective function constructed in step 4 is: ; Where λ is the regularization coefficient, and the function... The Huber loss function is as follows: ; In the formula, δ Let r represent the hyperparameter, and r = Yexp - Ypred, which represents the residual between the experimental and predicted values.

[0015] Furthermore, in step 5, the L-BFGS-B algorithm is used to iteratively solve the optimization objective function, specifically including: Initialization phase: Set the initial weight vector, upper and lower bound constraints on the weights, convergence accuracy tolerance, and maximum number of iterations; Iterative solution loop: Calculate the gradient of the current objective function; use Cauchy points to determine the active set that touches the boundary and the free set that does not touch the boundary; update the L-BFGS matrix in the free set subspace to obtain candidate update directions; perform line search to ensure that the updated parameters satisfy the constraint boundary. Convergence criterion: When the gradient projection norm is less than the tolerance or the relative decrease in the objective function value is less than the threshold, the iteration stops and the optimal time series feature weights are output.

[0016] The present invention discloses the following technical effects: The dynamic model optimization method of this invention breaks through the limitation of traditional neural networks that rely solely on zero-order input information. By introducing first-order and second-order derivatives and sliding window statistical features, it greatly enhances the model's ability to represent local dynamic evolution features. Simultaneously, the time-series features constructed in this invention are linearly independent of the integral form G, resulting in small fluctuations in the Hessian matrix condition number of the objective function, which, combined with the optimization algorithm, enables rapid convergence. Furthermore, the introduction of sliding window statistical features reduces experimental noise, and the use of the Huber loss function to shield outlier interference comprehensively enhances the numerical stability and noise robustness of the dynamic model optimization process, effectively improving the accuracy of sample reaction rate prediction. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a diagram of the neural network structure of a common multilayer perceptron in existing technologies; Figure 2 A schematic diagram of the dynamic model optimization method based on time-enhanced MLP provided in an embodiment of the present invention; Figure 3 A comparison chart of predicted and experimental values ​​of time series feature-enhanced MLP provided in an embodiment of the present invention; wherein, (a) represents a heating rate of 10 ℃ / min, (b) represents a heating rate of 15 ℃ / min, and (c) represents a heating rate of 20 ℃ / min; Figure 4 The bar chart shows the contribution of different kinetic models provided in the embodiments of the present invention; where (a) represents a heating rate of 10 ℃ / min, (b) represents a heating rate of 15 ℃ / min, and (c) represents a heating rate of 20 ℃ / min. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The purpose of this invention is to provide a dynamic model optimization method based on time-enhanced MLP, which solves the problems of existing dynamic models being easily disturbed by outliers, lacking effective regularization constraints, and the algorithms being prone to slow convergence and overfitting.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 2 As shown, this invention provides a dynamic model optimization method based on time-enhanced MLP, comprising: Step 1: Conduct heating oxidation experiments on the samples using thermogravimetric analysis to obtain heating oxidation curves at different heating rates, and plot the conversion rate change curve based on the heating weight loss curve. Step 2: Based on experimental data ( T i , α i Five types of time series features were constructed. The numerical derivative was calculated using the central difference method, and a sliding window was introduced to extract statistical features to reduce noise. 1) The first derivative is used to characterize the reaction rate, and its calculation formula is: (1) in, T i Indicates temperature. α i Indicates conversion rate. T j Representing data points j The temperature.

[0023] 2) The second derivative is used to characterize the reaction acceleration rate, and its calculation formula is: (2) 3) Moving Average: The specific method for calculating the moving average feature is as follows: Define the moving window size as... N w The moving average is For each data point j, calculate the statistics within the window: (3) 4) Standard deviation is used to evaluate the local volatility and dispersion of data. Its specific calculation formula is as follows: (4) In the formula, This represents the average conversion rate.

[0024] 5) The specific formula for calculating the reaction rate (based on the time scale) is as follows: (5) in, β This indicates the rate of temperature increase.

[0025] Step 3: Perform feature fusion between the time series features and the integral form of the dynamic mechanism model to construct an enhanced feature matrix, and construct a prediction model based on the enhanced feature matrix; The enhanced feature matrix is ​​defined as follows: (6) in, , which is the integral form of the dynamic model; , represents the set of 5 types of time series features calculated in step 2; The prediction model is represented as follows: (7) in, w The weights are for time features.

[0026] Step 4: Construct an optimization objective function based on Huber loss and L2 regularization. Define the residual between the experimental and predicted values ​​as... r = Y exp - Y pred The Huber loss function is used for optimization: (8) The optimization objective is: (9) In the formula, λ is the regularization coefficient used to suppress overfitting; Huber loss, as a mixture of mean squared error (MSE) and absolute error (MAE), utilizes hyperparameters. δ The conversion range between the two is controlled. Meanwhile, in this embodiment, to preserve physical meaning, a non-negativity constraint w is applied to the weights of the mechanistic model. i ≥0 indicates that the time series feature weights mainly provide compensation and are not constrained.

[0027] Step 5: Rapid Model Convergence Solution. The L-BFGS-B algorithm is used to iteratively solve the optimization objective function, outputting the optimal time series feature weights to obtain the final dynamic prediction model.

[0028] The prediction model achieves fast convergence based on the following formula: (10) In the formula, α σ is the conversion rate; Bias is the bias; Var is the variance; σ 2 This represents the noise variance.

[0029] Specifically, the L-BFGS-B algorithm is used to iteratively solve the optimization objective function, including: 1. Initialization phase.

[0030] ① Set the initial weight vector w (0) =[w 机理 (0) ,w 时序 (0) ], which can usually be set to all zeros or a small random integer.

[0031] ② Define the upper and lower bound constraint vectors for the weights, 1 and μ.

[0032] For the weights of the mechanistic model, a lower bound l is set. i =0, upper bound μ i = ; For time series feature weights, a lower bound l is set. j = Upper bound μ j = .

[0033] ③ Set the convergence accuracy tolerance ε and the maximum number of iterations M.

[0034] 2. Iterative solution loop.

[0035] In each iteration, the algorithm mainly performs the following core steps: ① Calculate the gradient and the effective gradient space.

[0036] Calculate the current parameter w (k) Below, the gradient of the objective function is calculated using formula (9). .

[0037] ②Use Cauchy points to determine the active set.

[0038] The algorithm performs a projection search along the negative gradient direction to find the boundary (w). i The set of variables whose boundaries are equal to 0 (active set) and the set of free variables whose boundaries are not reached (free set) are defined. This achieves a dynamic trade-off between mechanistic constraints and temporal compensation.

[0039] ③ Two-stage alternating / approximate fitting within the subspace (L-BFGS matrix update).

[0040] Joint optimization of mechanistic and temporal features: Using the displacement difference and gradient difference of m iterations, the Hessian matrix inverse of the objective function in the free variable subspace is approximately calculated.

[0041] Constraint Solution Correction: Solve the quadratic minimization problem within the free variable subspace to obtain candidate update directions d. (k) .

[0042] ④ Line search.

[0043] Along d (k) Perform a line search in the direction to ensure step size α k Satisfy the Armijo criterion, while ensuring the updated parameter w (k +1) = w (k) + α k ·d (k) Within the constraint boundary (i.e., satisfying w)(0) ≥0).

[0044] 3. Convergence judgment.

[0045] Check if the model meets one of the following stopping conditions in the current iteration: ① The gradient projection norm is less than the tolerance (indicating that the extreme point has been reached): (11) ② The relative decrease in the objective function value is less than the threshold (indicating that the model is difficult to optimize further and has entered the convergence plateau): (12) In the formula, g (k) is the gradient vector of the current step; L(w) is the objective function; fact and eps are built-in hyperparameters for controlling accuracy in the L-BFGS-B algorithm.

[0046] If any of the above conditions are met, the iteration stops and the optimal weight vector is output. Otherwise, let k = k + 1. Return to step 2 and continue iterating.

[0047] This invention also utilizes an optimized kinetic model to predict the kerogen oxidation process at different heating rates (10, 15, 20 °C / min), such as... Figure 3 As shown in the figure, the optimized kinetic model reduces the fitting error. Taking a heating rate of 15℃ / min as an example, the contribution of different kinetic models calculated based on weight W2 is as follows: Figure 4 As shown in the figure (D represents the diffusion model, R represents the interfacial reaction model, A represents the nucleation model, D1, D2, D3, and D4 represent one-dimensional, two-dimensional, three-dimensional, and four-dimensional diffusion, respectively; R2 and R3 represent two-dimensional and three-dimensional diffusion, respectively; and A2, A3, and A4 represent two-dimensional, three-dimensional, and four-dimensional diffusion, respectively), the results indicate that the kerogen oxidation process is significantly influenced by D1, R2, R3, and A4, with D1 and R2 being the dominant reaction mechanisms. Overall, the kerogen oxidation process is a dynamic process that transitions from diffusion control to nucleation / interfacial reaction control, and then back to diffusion control. Its evolution can be divided into three stages: 1) In the initial stage of the reaction, oxygen diffuses to the kerogen surface and undergoes oxidation, forming discretely distributed three-dimensional reaction nuclei; 2) As the reaction progresses, the reaction interface advances towards the kerogen core, and the geometric contraction characteristics begin to intensify. At this point, the reaction rate is mainly controlled by interfacial reaction kinetics. 3) In the later stages of the reaction, the local aggregation of solid products (such as char-like substances) or the formation of a dense oxide layer due to particle shrinkage together increase the diffusion resistance of gas along the reaction direction, thus gradually enhancing the influence of the diffusion process on the overall reaction rate.

[0048] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dynamic model optimization method based on time-enhanced MLP, characterized in that, Includes the following steps: Step 1: Obtain experimental data of the sample during the heating oxidation process. The experimental data includes temperature sequences and corresponding conversion rate sequences at different heating rates. Step 2: Construct time series features based on the experimental data. The time series features include first derivative features, second derivative features, moving average features, standard deviation features, and reaction rate features based on the time scale. Step 3: Perform feature fusion between the time series features and the integral form of the dynamic mechanism model to construct an enhanced feature matrix, and construct a prediction model based on the enhanced feature matrix; Step 4: Construct an optimization objective function, which includes a Huber loss function term and an L2 regularization term, and apply non-negative constraints to the weights of the time series features in the prediction model; Step 5: The L-BFGS-B algorithm is used to iteratively solve the optimization objective function, output the optimal time series feature weights, and obtain the final dynamic prediction model.

2. The dynamic model optimization method based on time-enhanced MLP according to claim 1, characterized in that, In step 2, the first derivative characteristic is used to characterize the reaction rate, and its calculation formula is as follows: ； in, T i Indicates temperature. α i Indicates conversion rate. T j Representing data points j The temperature.

3. The dynamic model optimization method based on time-enhanced MLP according to claim 2, characterized in that, The second derivative is used to characterize the reaction acceleration rate, and its calculation formula is as follows: 。 4. The dynamic model optimization method based on time-enhanced MLP according to claim 3, characterized in that, The specific method for calculating the moving average feature is as follows: Define the moving window size as... N w The moving average is For each data point j Calculate the statistics within the window: 。 5. The dynamic model optimization method based on time-enhanced MLP according to claim 4, characterized in that, The standard deviation feature is used to evaluate the local volatility and dispersion of data. Its specific calculation formula is as follows: ； In the formula, This represents the average conversion rate.

6. The dynamic model optimization method based on time-enhanced MLP according to claim 4, characterized in that, The specific formula for calculating the reaction rate is: ； in, β This indicates the rate of temperature increase.

7. The dynamic model optimization method based on time-enhanced MLP according to claim 1, characterized in that, The enhanced feature matrix constructed in step 3 is: ； in, , which is the integral form of the dynamic model; , represents the set of 5 types of time series features calculated in step 2; The prediction model is represented as follows: ； in, w The weights are for time features.

8. The dynamic model optimization method based on time-enhanced MLP according to claim 1, characterized in that, The optimization objective function constructed in step 4 is: ； Where λ is the regularization coefficient, and the function... The Huber loss function is as follows: ； In the formula, δ Indicates hyperparameters, r = Y exp - Y pred , representing the residual between the experimental value and the predicted value.

9. The dynamic model optimization method based on time-enhanced MLP according to claim 1, characterized in that, In step 5, the L-BFGS-B algorithm is used to iteratively solve the optimization objective function, specifically including: Initialization phase: Set the initial weight vector, upper and lower bound constraints on the weights, convergence accuracy tolerance, and maximum number of iterations; Iterative solution loop: Calculate the gradient of the current objective function; use Cauchy points to determine the active set that touches the boundary and the free set that does not touch the boundary; update the L-BFGS matrix in the free set subspace to obtain candidate update directions; perform line search to ensure that the updated parameters satisfy the constraint boundary. Convergence criterion: When the gradient projection norm is less than the tolerance or the relative decrease in the objective function value is less than the threshold, the iteration stops and the optimal time series feature weights are output.

Images