A Semi-Supervised Soft Sensor Modeling Method for Extreme Learning Machine Based on Variable Weighted Adaptive Local Structure Composition

Abstract

The present invention relates to a semi-supervised soft sensor modeling method based on a variable-weighted adaptive local graph construction extreme learning machine. This method comprehensively utilizes the weighted Euclidean distance information in the data input space and the prediction output space to adaptively construct a nearest neighbor graph to accurately approximate the potential structure information of the data. At the same time, considering that different auxiliary variables have different degrees of contribution to the accurate estimation of the dominant variable, different weights are assigned to different auxiliary variables through variable-weighted learning to reduce the adverse effects of redundant variables and noise on graph construction and regression learning. Finally, variable weighting, adaptive graph construction, and extreme learning machine modeling are integrated into a unified learning framework, and the overall optimal solution of the modeling learning is obtained by using alternating iterative optimization. Therefore, the semi-supervised learning framework provided by the present invention can make full use of the supervision information contained in the labeled data, assisted by the structure information contained in the unlabeled data, improve the performance of the extreme learning machine model, and achieve the purpose of enhancing the generalization ability and reliability of the soft sensor model.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial process detection, and relates to industrial process soft sensing technology. Specifically, it relates to a semi-supervised soft sensing modeling method for extreme learning machines based on variable weighted adaptive local graph construction. Background Art

[0002] Modern industrial production processes are developing rapidly towards digitalization and intelligentization. At the same time, the pursuit of product quality control is also getting higher and higher. For this reason, actual production devices are usually equipped with a large number of industrial sensors to measure in real time the operating parameters reflecting the process operating state, providing the necessary feedback information for realizing closed-loop optimal control of product quality. However, there are always some important parameters in industrial processes that are closely related to product quality but are difficult to directly measure, such as product concentration, components, and various physical property parameters. These key quality-related parameters can currently only be obtained by sending offline samples to the laboratory for chemical analysis, which has problems such as long measurement cycles, large feedback lags, and high human and material costs. Therefore, when the process conditions change, it is difficult for operators to give correct adjustment countermeasures because they cannot timely grasp the true operating state of the process, resulting in reduced production efficiency and even endangering operation safety. Soft sensing technology has developed under such a background. It realizes the indirect estimation of key quality-related parameters by establishing a mathematical model between easily measurable process variables (also called auxiliary variables) and difficult-to-directly-measure quality-related variables (also called dominant variables). Compared with laboratory chemical analysis or online component meters, it has significant advantages such as low application and maintenance costs and timely response, so it has been widely used in many industrial fields such as oil refining, chemical engineering, metallurgy, and pharmaceuticals.

[0003] The key to soft sensing technology lies in establishing a mathematical model that can accurately describe the potential functional relationship between auxiliary variables and dominant variables. If there is a deep understanding of the production process and rich knowledge in related fields, the mechanism modeling method can be used to establish a soft sensing model. However, the complexity and uncertainty of modern industrial processes have greatly restricted the application scope of the mechanism modeling method. Therefore, the data-driven regression modeling method, due to its independence from specific field expertise and greater advantages in universality and flexibility, has been widely used in the field of soft sensing in recent decades. Representative technologies include principal component regression, partial least squares, artificial neural networks, and support vector machines. In particular, with the advent of the big data era, artificial neural network algorithms based on deep learning, such as convolutional neural networks, recurrent neural networks, and autoencoders, have become research hotspots in the field of soft sensing modeling in recent years and have achieved a series of remarkable results.

[0004] The performance of data-driven soft sensor modeling methods depends largely on the quantity and quality of training data. Specifically, in order to obtain a soft sensor model with strong generalization ability, it is necessary to use a large number of input-output data sets covering the main operating conditions of the process for training. In particular, this is especially true for machine learning models with complex structures and many adjustable parameters such as artificial neural networks. However, for actual soft sensor modeling problems, the sampling rate of the dominant variable (corresponding to the output variable of the soft sensor model) is generally much lower than the sampling frequency of the auxiliary variable (corresponding to the input variable of the soft sensor model). This results in only a small part of the actual collected training data having both input and output values, while the vast majority of the data only has input values, and the corresponding output values ​​are missing. In the field of machine learning, data with values ​​at both the input and output ends are called labeled data, while data with values ​​only at the input end are called unlabeled data. At present, data-driven soft sensor modeling methods mostly use supervised learning, that is, only using labeled data for modeling and ignoring the role of unlabeled data. In the case of scarce labeled samples, it is easy to overfit the model, and the generalization ability and reliability of the model cannot be guaranteed, which cannot meet the needs of practical applications. In fact, unlabeled data contains rich data structure information. A series of studies have shown that the rational use of the information contained in unlabeled data can significantly improve the performance of the regression model. Therefore, the use of semi-supervised learning, that is, using a small amount of labeled data and a large amount of unlabeled data at the same time, to establish a soft sensor mathematical model has received more and more attention.

[0005] According to the different utilization methods of unlabeled data, existing semi-supervised soft sensor modeling methods can be divided into several types such as Probabilistic Generative, Self-Training, Co-Training, and Manifold Regularization (MR). Among them, MR is based on the local smoothness assumption that "similar inputs correspond to similar outputs", establishes the connection between unlabeled data and labeled data by constructing a nearest neighbor graph, and provides an effective mechanism to generalize the supervised learning model to the semi-supervised learning scenario. For example, Huang et al. (2014) constructed a graph regularization constraint term using the k-nearest neighbor method based on labeled data and unlabeled data and added this constraint term to the optimization objective function of the Extreme Learning Machine (ELM) to obtain a semi-supervised ELM algorithm. Similarly, Yan et al. (2016) and Zhao et al. (2020) introduced the MR term into the Gaussian process regression model and the width learning neural network model respectively to obtain the corresponding semi-supervised learning algorithms. The above research shows that the semi-supervised learning framework based on MR can simultaneously utilize the supervised / discriminative information provided by labeled data and the structural information contained in unlabeled data to improve the generalization ability and reliability of the model, has a concise description form and a solid theoretical foundation, and thus has been successfully applied in many fields of soft sensor modeling.

[0006] It should be noted that an important prerequisite for the semi-supervised learning method based on MR to achieve performance improvement is that the constructed nearest neighbor graph can accurately approximate the potential local manifold structure of the data. Considering that the data manifold structure is unknown in advance and problem-related, many methods for constructing the nearest neighbor graph have been proposed, such as the k-nearest neighbor method, local linear representation, sparse self-representation, and low-rank self-representation, etc., and have been successfully applied in many different research fields. However, most of the existing methods construct the nearest neighbor graph offline using unsupervised learning methods in the original high-dimensional input space of the data. This may cause the following two problems: (1) There are inevitably redundant auxiliary variables and noises in the actual modeling data. During the graph construction process, these redundant information may seriously affect the calculation of the similarity between data, resulting in incorrect connections between nodes in the constructed nearest neighbor graph. (2) The existing methods generally adopt the offline graph construction method, and the graph construction and subsequent regression modeling learning are completed separately as two independent learning tasks, ignoring the internal connection between graph construction and regression learning. Therefore, the supervised information provided by labeled samples cannot be effectively utilized during graph construction, resulting in the problem that the constructed nearest neighbor graph is not compatible with the subsequent regression modeling task.

[0007] In summary, when using the existing MR-based semi-supervised learning method to solve the actual soft sensor modeling problem, prominent problems such as weak model generalization ability and poor reliability are likely to occur. This is mainly because the existing methods ignore the inevitable internal connection between graph construction and regression modeling learning, resulting in the structure and parameters of the constructed graph being unable to accurately describe the potential structure information of the data, and failing to achieve the purpose of improving the model performance by using unlabeled samples. Summary of the Invention

[0008] Aiming at the key problem of the disconnection between graph construction and regression modeling existing in the existing semi-supervised soft sensor modeling technology based on popular regularization, the present invention provides a semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction, which closely combines variable weighting, adaptive graph construction and extreme learning machine modeling, and forms a unified optimization learning framework for joint solution. Specifically, the present invention comprehensively uses the weighted Euclidean distance information in the data input space and the prediction output space to adaptively construct a neighbor graph to accurately approximate the potential structure information of the data; at the same time, considering that different auxiliary variables have different contribution degrees to the accurate estimation of the dominant variable, different weights are assigned to different auxiliary variables through variable weighted learning to reduce the adverse effects of redundant variables and noise on graph construction and regression modeling; finally, variable weighting, adaptive graph construction and extreme learning machine modeling are integrated into a unified optimization framework, and alternating iteration is used to solve to achieve the overall optimum of modeling learning. Therefore, the semi-supervised learning framework provided by the present invention can make full use of the supervision information contained in the labeled data, supplemented by the structure information contained in the unlabeled data, to achieve the purpose of improving the generalization ability and reliability of the soft sensor model.

[0009] In order to achieve the above object, the present invention provides a semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction, comprising the following steps:

[0010] (1) Offline modeling stage: Collect the chemical analysis values y of the dominant variable i and the corresponding measured values of the auxiliary variables where i = 1, 2,..., n l , n l is the number of the collected dominant variable values, d is the dimension of the auxiliary variables; additionally collect n u measured values of the auxiliary variables j = n l +1, n l +2,..., n l +n u , define n = n l +n u as the number of the collected auxiliary variable values; sort the collected auxiliary variable values by row to obtain the auxiliary variable data matrix The superscript T represents the matrix transposition operation. Accordingly, the collected dominant variable values ​​are sorted by row to obtain the dominant variable data row vector Furthermore, define n u Row vector with all zeros y l and u Merge into row vectors Utilizing X 0 The mean value of mean(X 0 ) and mean square error std(X 0 ) to X 0 Standardized processing Take advantage of y 0 The mean value of mean(y 0 ) and mean square error std(y 0 ) for y 0 Standardized processing Get the offline training data X, y of the extreme learning machine;

[0011] (ii) Specify the number of neurons in the hidden layer of the extreme learning machine as n h , regularization parameters β, λ, μ, θ and maximum number of iterations max_iterate, initialize variable weight matrix Calculate the distance between each sample and select the one with the i-th sample x i The most recent k samples construct the initial Laplacian matrix

[0012] (III) Randomly generate the weight matrix between the input layer and the hidden layer of the extreme learning machine and the bias matrix Use activation function to calculate hidden layer output

[0013] (IV) Update the weights between the hidden layer and the output layer Bias b and model predicted label value

[0014] (V) Using Similarity Matrix Update variable weight matrix

[0015] (VI) Update the similarity matrix

[0016] (VII) Repeat steps (IV), (V), (VI) until the maximum number of iterations max_iterate is reached;

[0017] (VIII) Online testing phase: collecting test data X new , using the training data X 0 The mean value of mean(X 0) and the mean square error std(X 0 ) perform standardization processing on the test data X new to obtain the standardized test data Perform inverse standardization on the ELM prediction result to obtain the estimated value corresponding to X new

[0018] Furthermore, in the step (1), first use the training data mean value mean(X 0 ) and the mean square error std(X 0 ) perform standardization processing on the training data X through formula (1). The expression of formula (1) is: 0

[0019]

[0020] In the formula, mean(·) represents calculating the mean value of each column of the matrix, and std(·) represents calculating the mean square error of each column of the matrix, to obtain the standardized training data For a similar standardization process is also required as formula (2):

[0021]

[0022] Furthermore, in the step (2), for the initialized variable weighted matrix calculate the distance between each sample and select the k samples closest to the i-th sample x i to construct the initial Laplacian matrix The specific steps are as follows;

[0023] First, initialize the variable weighted matrix M through formula (3). Formula (3) is expressed as:

[0024]

[0025] Among them, M is a diagonal matrix with elements on the diagonal all being and the remaining elements being 0,

[0026] Secondly, calculate the initial Laplacian matrix L through formulas (4) - (7). The process is as follows:

[0027]

[0028]

[0029]

[0030] ​​

[0031] In the formula, D ii is the i-th element on the diagonal of the diagonal matrix D, and L is the Laplacian matrix corresponding to the data set X. Then, solve formula (4) according to formulas (8) - (12):

[0032]

[0033]

[0034] Write formula (9) as the Lagrangian formula (10) by defining two Lagrange multipliers. Take the partial derivative of formula (10) with respect to s i and set it to 0, then we can get According to the KTT condition, the optimal solution is as shown in formula (11),

[0035]

[0036]

[0037] where where (·) + means taking the value itself when the value inside the parentheses is greater than 0, and taking 0 when it is less than or equal to 0. Use formula (12) to obtain s with a sparse representation having only k non-zero elements i ,

[0038]

[0039] Further simplify formula (12) to formula (13) as:

[0040]

[0041] Since γ is related to k, k is an integer and 0 ≤ k ≤ n, the parameter γ can be expressed as formula (14):

[0042]

[0043] Substitute η and γ into formula (11) to get:

[0044]

[0045] Furthermore, in the step (iii), randomly generate the weight matrix between the input layer and the hidden layer of the extreme learning machine and the bias term matrix Map the data X through the sigmod function using formula (16) to form the output matrix of the hidden layer of the extreme learning machine

[0046]

[0047] Among them, W in is an input weight matrix randomly generated within (-1, 1), and B in represents a randomly generated bias term matrix. The hidden layer output H 0 and the input data X are merged by rows to obtain an augmented data matrix H = [H 0 , X];

[0048] Furthermore, in the step (iv), update the weights bias b and the model prediction label value

[0049] First, the extreme learning machine, variable weighting, and adaptive local graph construction are organically integrated into a unified optimization objective function, and the minimized objective function is shown in formula (17):

[0050]

[0051]

[0052] Among them, Tr(·) represents the matrix trace operation, represents the square of the l 2 norm, 1 represents a column vector with all elements being 1, and the diagonal matrix U = diag(β, β, … β, 0, 0, …, 0) ∈ R n×n , that is, a certain weight β is assigned to the first n l label values, and λ, μ, β, θ are given regularization parameters, and w, b, and f are the weights, bias, and model prediction label values between the hidden layer and the output layer respectively;

[0053] Then, fix the similarity matrix variable weighting matrix to obtain an optimization problem description about w, b, and f as shown in formula (18), and thus the analytical expressions of w and b can be obtained as shown in formula (19):

[0054]

[0055]

[0056]

[0057] In formula (19), A = λ(λHH C H T + I q×q ) -1 H TH C where \(q = d + n\) h , Let \(I\) denote the identity matrix and \(1\) denote the column vector with all elements equal to \(1\). Substitute formula (19) into the objective function \(Hw + 1\) in formula (18) n×1 for \(b\) to obtain formula (20):

[0058]

[0059] where

[0060] Finally, according to formula (19) and formula (20), the optimization problem in formula (18) can be transformed into formula (21):

[0061]

[0062] Taking the partial derivative of \(f\) and setting it to \(0\), the analytical expression of \(f\) can be obtained as shown in formula (22),

[0063] \(f=(U + L+\mu\lambda H\) C -\mu\lambda 2 N) -1 Uy (22)

[0064] where X C =HH C ;

[0065] Furthermore, in step (v), the specific steps of updating the variable weighted matrix using the similarity matrix are given by formulas (23) - (25):

[0066] First, fix the output similarity matrix \(S\), and the objective function is simplified from formula (17) to formula (23):

[0067]

[0068] Then, solve formula (23) to obtain the update formula (24) of the variable weighted matrix \(M\):

[0069]

[0070] where \(t\) i =z ii , \(Z = X\) T LX,z ii is the element on the \(i\)-th main diagonal of matrix \(Z\);

[0071] Furthermore, in step (vi), update the similarity matrix The steps are given by Formula (25) - Formula (27):

[0072] First, fix the output Variable weighting matrix The objective function is simplified from Formula (17) to Formula (25):

[0073]

[0074] Then, Formula (25) can be further simplified to solve the optimization problem shown in Formula (26):

[0075]

[0076] Finally, similarly, according to the principle in Step (ii), the update formula (27) of the similarity matrix S is obtained

[0077]

[0078] In Formula (27),

[0079] Furthermore, in Step (vii), repeat Steps (iv) (v) (vi) until the maximum number of iterations max_iterate is reached.

[0080] Furthermore, in Step (viii), the specific steps in the online test phase are as follows:

[0081] First, for the n t test data samples collected Use the mean mean(X ) and standard deviation std(X 0 ) of the training data 0 ) to standardize the test data X new through Formula (28). The expression of Formula (28) is:

[0082]

[0083] Then, according to the standardized test data Calculate the output value of the test data through Formula (29) and Formula (30) Formulas (29) and (30) are respectively expressed as:

[0084] H t0 = X test W in + B in (29)

[0085]

[0086] Among them, is the output of the hidden layer. The hidden layer output H t0 and the input data X test are merged row by row to obtain the augmented data matrix

[0087] Finally, the ELM prediction result y test is de-normalized to obtain the estimated value corresponding to X new as follows: As shown in formula (31):

[0088] y new = y test × std(y 0 ) + mean(y 0 ) (31)

[0089] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0090] The semi-supervised soft sensor modeling method based on variable weighted adaptive local graph construction of the extreme learning machine provided by the present invention closely combines variable weighting, adaptive graph construction and extreme learning machine modeling, and forms a unified optimization learning framework for joint solution. Specifically, on the one hand, the present invention comprehensively utilizes the weighted Euclidean distance information in the data input space and the prediction output space to adaptively construct a neighbor graph to accurately approximate the potential structure information of the data. On the other hand, considering that different auxiliary variables have different contribution degrees to the accurate estimation of the dominant variable, different weights are assigned to different auxiliary variables through variable weighted learning to reduce the adverse effects of redundant variables and noise on graph construction and regression modeling. Compared with other existing algorithms, the present invention integrates variable weighting, adaptive graph construction and extreme learning machine modeling in a unified optimization framework, and adopts alternating iteration to solve to achieve the overall optimum of modeling learning. This method can make full use of the supervision information contained in the labeled data and be assisted by the structure information contained in the unlabeled data to improve the generalization ability and reliability of the soft sensor model. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 is a flowchart of the semi-supervised soft sensor modeling method based on variable weighted adaptive local graph construction of the extreme learning machine according to the present invention;

[0092] Figure 2 is a schematic diagram of the de-butane tower process according to an embodiment of the present invention;

[0093] Figure 3 is a diagram showing the influence of different regularization parameters on the determination coefficients of the training set and the test set under the semi-supervised extreme learning machine model;

[0094] Figure 4 shows the influence diagram of different regularization parameters on the determination coefficients of the training set and the test set under the semi-supervised extreme learning machine model with adaptive local composition;

[0095] Figure 5 shows the influence diagram of different regularization parameters on the determination coefficients of the training set and the test set under the semi-supervised extreme learning machine model with variable-weighted adaptive local composition;

[0096] Figure 6 It is the change diagram of the determination coefficients of the test sets of the three models under the optimal parameters; Specific Embodiments

[0097] Next, the present invention will be specifically described through exemplary embodiments. However, it should be understood that without further description, the elements, structures, and features in one embodiment can also be beneficially combined into other embodiments.

[0098] See Figure 1 , the present invention discloses a semi-supervised soft sensor modeling method based on extreme learning machine with variable-weighted adaptive local composition, which includes the following steps:

[0099] (I) Offline modeling stage: Collect the laboratory analysis values y i of the dominant variables and the corresponding measured values of the auxiliary variables where i = 1, 2,..., n l , n l is the number of the collected dominant variable values, d is the dimension of the auxiliary variables; additionally collect n u measured values of the auxiliary variables j = n l + 1, n l + 2,..., n l + n u , define n = n l + n u as the number of the collected auxiliary variable values; sort the collected auxiliary variable values by row to obtain the auxiliary variable data matrix The superscript T represents the matrix transpose operation. Correspondingly, sort the collected dominant variable values by row to obtain the dominant variable data row vector Further, define the n u row all-zero row vector Combine y l and y u into a row vector Use the mean mean(X 0 ) and the standard deviation std(X 0 ) of X 0 to perform standardization processing on X 0 to obtain Use the mean mean(y 0 of y0 ) and the standard deviation std(y 0 ) for y 0 Perform standardization processing to obtain Obtain the offline training data X and y of the extreme learning machine. The specific steps are as follows:

[0100] Utilize the training data The mean value mean(X 0 ) and the standard deviation std(X 0 ) Through formula (1), perform standardization processing on the training data X 0 The expression of formula (1) is:

[0101]

[0102] In formula (1), mean(·) represents calculating the mean value of each column of the matrix, and std(·) represents calculating the standard deviation of each column of the matrix, obtaining the standardized training data For A similar standardization process is also required, such as formula (2):

[0103]

[0104] (2) Specify the number of hidden layer neurons of the extreme learning machine as n h , regularization parameters β, λ, μ, θ, and the maximum number of iterations max_iterate, and initialize the variable weight matrix Calculate the distance between each sample and select the k samples closest to the i-th sample x i Construct the initial Laplacian matrix The specific process is as follows:

[0105] First, initialize the variable weight matrix M through formula (3), and formula (3) is expressed as:

[0106]

[0107] Among them, M is a diagonal matrix with all elements on the diagonal being The remaining elements are 0 diagonal matrices,

[0108] Secondly, calculate the initial Laplacian matrix L through formula (4) - formula (7), and the process is as follows:

[0109]

[0110]

[0111]

[0112]

[0113] In the formula, D ii is the i-th element on the diagonal of the diagonal matrix D, L is the Laplacian matrix corresponding to the data set X, and then solve formula (4) according to formulas (8)-(12):

[0114]

[0115]

[0116] Write formula (9) as the Lagrangian formula (10) by defining two Lagrange multipliers. Take the partial derivative of formula (10) with respect to s i and set it to 0, then we can get According to the KTT conditions, the optimal solution is as shown in formula (11),

[0117]

[0118]

[0119] where where (·) + means taking the value itself when the value in the parentheses is greater than 0, and taking 0 when it is less than or equal to 0. Use formula (12) to get s with only k non-zero elements for sparse representation i ,

[0120]

[0121] Further simplify formula (12) to formula (13) as:

[0122]

[0123] Because γ is related to k, k is an integer and 0 ≤ k ≤ n, the parameter γ can be expressed as formula (14):

[0124]

[0125] Substitute η and γ into formula (11) to get:

[0126]

[0127] (III) Randomly generate the weight matrix between the input layer and the hidden layer of the extreme learning machine and the bias term matrix and calculate the output of the hidden layer using the activation function The specific steps are as follows:

[0128] First, within the range of (-1, 1), randomly generate the weight matrix W between the input layer and the hidden layer of the extreme learning machine in and the bias term matrix B in ;

[0129] Then, use the sigmoid function and the data X to calculate the output matrix of the hidden layer of the extreme learning machine The expression of the sigmoid function is shown in formula (16):

[0130]

[0131] Finally, merge the hidden layer output H 0 and the input data X row by row to obtain the augmented data matrix H = [H 0 , X];

[0132] (IV) Update the weights between the hidden layer and the output layer bias b and the predicted label values of the model The specific process is as follows:

[0133] First, organically integrate the extreme learning machine, variable weighting, and adaptive local graph construction into a unified optimization objective function to minimize the optimization problem shown in formula (17):

[0134]

[0135]

[0136] where Tr(·) represents the matrix trace operation, represents the square of the l 2 norm, 1 represents the column vector with all elements being 1, and the diagonal matrix U = diag(β, β, …, β, 0, 0, …, 0) ∈ R n×n , that is, assign a certain weight β to the first n l label values, and λ, μ, β, θ are given regularization parameters, and w, b, and f are the weights, biases, and predicted label values between the hidden layer and the output layer respectively;

[0137] Then, fix the similarity matrix variable weighting matrix to obtain the optimization problem description about w, b, and f as shown in formula (18), and thus the analytical expressions of w and b can be obtained as shown in formula (19):

[0138]

[0139]

[0140]

[0141] In Equation (19), A = λ(λHH C H T + I q×q ) -1 H T H C , where q = d + n h , I represents the identity matrix, 1 represents the column vector with all elements being 1. Substituting Equation (19) into the objective function Hw + 1 n×1 b in Equation (18) gives Equation (20):

[0142]

[0143] where

[0144] Finally, according to Equations (19) and (20), the optimization problem in Equation (18) can be transformed into Equation (21):

[0145]

[0146] Taking the partial derivative of f and setting it to 0, the analytical expression of f can be obtained as shown in Equation (22),

[0147] f = (U + L + μλH C - μλ 2 N) -1 Uy (22)

[0148] where

[0149] (V) Using similar matrices to update the variable weighted matrix The specific process is as follows:

[0150] First, fix the output similar matrix S, and the objective function is simplified from Equation (17) to Equation (23):

[0151]

[0152] Then, solving Equation (23), the update formula (24) of the variable weighted matrix M can be obtained:

[0153]

[0154] where t i = z ii , Z = X T LX,z ii is the element on the i-th main diagonal of matrix Z;

[0155] (6) Update the similarity matrix The specific process is as follows:

[0156] First, fix the output Variable weighted matrix The objective function is simplified from formula (17) to formula (25):

[0157]

[0158] Then, formula (25) can be further simplified to solve the optimization problem as shown in formula (26):

[0159]

[0160] Finally, according to the principle in step (2), the update formula (27) of the similarity matrix S is obtained.

[0161]

[0162] In formula (27),

[0163] (7) Repeat steps (4), (5), and (6) until the maximum number of iterations max_iterate is reached. The specific process is as follows:

[0164] Repeat steps (4), (5), and (6) until the maximum number of iterations max_iterate is reached.

[0165] (8) Online testing phase: For the n t test data collected Use the mean mean(X ) and standard deviation std(X 0 ) of the training data to standardize the test data X 0 ) through formula (28). Formula (28) is expressed as: new The standardized test data is obtained

[0166]

[0167] The steps to obtain the prediction result using the given model are formulas (29) - (30): Use the given model to obtain the prediction result The steps are formulas (29) - (30):

[0168] H t0 = X test W in + B in (29)

[0169]

[0170] Among them, is the output of the hidden layer. The hidden layer output H t0 and the input data X test are merged row by row to obtain the augmented data matrix

[0171] Finally, the ELM prediction result y test is de-normalized to obtain the estimated value corresponding to X new as shown in the following formula (31): As shown in formula (31):

[0172] y new = y test × std(y 0 ) + mean(y 0 ) (31)

[0173] In the above method of the embodiment of the present invention, the model integrates variable weighting, adaptive graph construction, and extreme learning machine modeling within a unified learning framework, and uses alternating iterative optimization to obtain the overall optimal solution of the modeling learning. This method comprehensively utilizes the weighted Euclidean distance information in the data input space and the prediction output space to adaptively construct a nearest neighbor graph to accurately approximate the potential structure information of the data. The extreme learning machine with variable-weighted adaptive local graph construction performs variable-weighted learning on the input samples based on the extreme learning machine with adaptive local graph construction, and assigns different weights to different auxiliary variables through variable-weighted learning to reduce the adverse effects of redundant variables and noise on graph construction and regression learning. This method improves the performance of the extreme learning machine model by using the supervised information contained in the labeled data and supplementing it with the structure information contained in the unlabeled data.

[0174] To illustrate the effect of the above extreme learning machine soft sensor modeling method based on variable-weighted adaptive local graph construction of the present invention, the following further illustrates the present invention in combination with specific embodiments.

[0175] Embodiment: Taking the process data of a debutanizer as an example for illustration.

[0176] The debutanizer distillation column is part of the desulfurization and naphtha separator unit. Its main task is to maximize the C5 (stable gasoline) content in the top of the debutanizer (feed to the liquefied petroleum gas separator) and minimize the C4 (butane) content in the bottom of the debutanizer (feed to the naphtha separator). Its block diagram is as Figure 2As shown. Except for the distillation column (T102), the debutanizer also includes equipment such as heat exchanger (E105B), top condenser (E107AB), bottom reboiler (E108AB), top reflux pump (P102AB), and feed water pump (P103AB) of the LPG separator. The C5 content in the top of the debutanizer is indirectly measured by an analyzer located at the bottom of the liquefied petroleum gas fractionation column in Unit 900. The measurement period of this device is 10 minutes. In addition, the position of the measuring device causes a delay, which is unknown but constant and may be in the range of 20 - 60 minutes. Similarly, the C4 content in the bottom of the debutanizer cannot be directly detected at the bottom, but is detected by installing a gas chromatograph at the top. The measurement period of this device is generally 15 minutes. Similarly, due to the installation position of the analytical instrument, there will be a large delay when obtaining the concentration value. This delay is not well-known, but constant and may be in the range of 30 - 75 minutes. Therefore, in order to achieve real-time measurement of butane concentration and improve the control quality of the debutanizer, it is necessary to establish a soft sensor model to estimate the bottom butane concentration in real time. In addition, considering the problems of low sampling efficiency and large time delay of quality variables in the actual production process, it is assumed that only one-fifth of all historical samples have labels (including both input data and output data), and the other historical samples are unlabeled samples (only including input data).

[0177] Next, the specific steps of the present invention will be described in combination with the production process of the debutanizer:

[0178] 1. Offline modeling stage: The collected data is used as the training data set and preprocessed.

[0179] First, all samples are preprocessed to delete the abnormal samples; then, considering the dynamic characteristics of the process, all samples are dimensionally expanded, and the number of sample features after expansion is 30; finally, standardization processing is performed to obtain the final offline training data. The collected quality variable values are sorted by row to obtain the quality variable data row vector. Furthermore, a 1440-row all-zero row vector is defined. Take y l and y u and merge them into a row vector. Use the mean mean(y 0 ) and standard deviation std(y 0 ) of y 0 ) to perform standardization processing on y 0 ) to obtain Obtain the offline training data X, y of the extreme learning machine.

[0180] 2. Construct an initial Laplacian matrix according to the training data set.

[0181] Specify that the number of hidden layer neurons of the extreme learning machine is 5, and the regularization parameters β, λ, μ, θ are 5, 5, 10 -2 , 10 3 and the maximum number of iterations is 15, and initialize the variable weighted matrix Calculate the distance between each sample and select the 3 samples closest to the i-th sample x i to construct the initial Laplacian matrix

[0182] 3. Calculate the output of the hidden layer using the activation function

[0183] Randomly generate the weights and biases between the input layer and the hidden layer, and calculate the output of the hidden layer using the activation function

[0184] 4. Update the model parameters Variable weighted matrix Similarity matrix

[0185] 5. Repeat step 4 until the maximum number of iterations 15 is reached;

[0186] 6. Online testing phase: Collect test data The number of dominant variable values collected in the test set is 400. Use the mean mean(X 0 ) and standard deviation std(X 0 ) of the training data X 0 ) to standardize the test data X new to obtain the standardized test data Inverse standardize the ELM prediction result to obtain the estimated value corresponding to X new

[0187] Adopt three evaluation indexes of root mean square error (RMSE), coefficient of determination (R 2 ), and mean absolute error (MAE) to comprehensively evaluate the prediction performance of the soft sensor model. The expressions of the three evaluation indexes are shown in formulas (32)-(34):

[0188]

[0189]

[0190]

[0191] In the formula, y i and are the true value and predicted value of the target variable of the i-th sample respectively,​ is the average value of the target variables for all samples. The coefficient of determination R 2 can measure the reliability of the prediction results. The closer the calculated result is to 1, the better the prediction effect of the soft sensor model. The root mean square error (RMSE) and mean absolute error (MAE) are used to calculate the prediction error of the soft sensor model. The smaller the error value, the higher the prediction accuracy of the soft sensor model.

[0192] Table 1 shows the fitting situations of the traditional semi-supervised extreme learning machine model, the extreme learning machine model with adaptive local graph construction, and the extreme learning machine model with variable weighted adaptive local graph construction of the present invention for the deisobutanizer data in 15 simulation experiments under the optimal parameters.

[0193] Table 1

[0194]

[0195] As can be seen from Table 1, the method provided by the present invention generally achieves the best results, and there is a certain improvement in the MAE and R 2 , RMSE of the test set.

[0196] Based on the above analysis, the extreme learning machine model with variable weighted adaptive local graph construction provided by the present invention can not only adaptively construct a neighbor graph by comprehensively using the weighted Euclidean distance information in the data input space and the prediction output space to accurately approximate the potential structure information of the data, but also assign different weights to different auxiliary variables through variable weighted learning, thereby improving the generalization ability and reliability of the model.

[0197] The influence diagrams of the semi-supervised extreme learning machine model, the semi-supervised extreme learning machine model with adaptive local graph construction, and the method of the present invention on the coefficient of determination of the predicted values and the true values of the deisobutanizer data with different regularization parameters are shown in Figures 3, 4, and 5. Figure 6 is the change diagram of the coefficient of determination of the test set for the three models under the optimal parameters. As Figure 6 can be seen, compared with the traditional method, the method of the present invention has higher prediction accuracy.

[0198] The above embodiments are used to explain the present invention, rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction, characterized in that, the specific steps are as follows: (1) Offline modeling stage: collect the laboratory analysis values y of the dominant variables i and the corresponding measured values of the auxiliary variables where y and x are respectively the butane concentration values that are difficult to directly measure and the process variable values that are easy to measure during the production process of the debutanizer tower, and the subscript i = 1, 2,..., n l , n l is the number of the collected y i , d is the dimension of the auxiliary variables; additionally collect n u measured values of the auxiliary variables n u is the number of the collected x j , and additionally define n = n l + n u as the total number of the collected auxiliary variable values x i and x j ; sort the collected n auxiliary variable values row by row to obtain the training data The superscript T represents the transpose operation. Correspondingly, sort the collected n l dominant variable values row by row to obtain the dominant variable data row vector Furthermore, define an n u row all-zero row vector Combine y l and y u into a row vector Use the mean mean(X 0 ) and the standard deviation std(X 0 ) of X 0 to perform standardization processing on X 0 to obtain Use the mean mean(y 0 ) and the standard deviation std(y 0 ) of y 0 to perform standardization processing on y 0 to obtain Obtain the extreme learning machine offline training data X, y; (2) Set the number of hidden layer neurons n of the extreme learning machine h , regularization parameters β, λ, μ, θ and the maximum number of iterations max_iterate, and initialize the variable weighted matrix Calculate the distance between each sample and select the k samples closest to the i-th sample x i Construct the initial Laplacian matrix (3) Randomly generate the weight matrix between the input layer and the hidden layer of the extreme learning machine and the bias term matrix Then calculate the output of the hidden layer (4) Update the weights between the hidden layer and the output layer The bias b and the model predicted label value (5) Updating the variable weighting matrix using a similarity matrix (6) Update the similarity matrix (VII) Repeat steps (IV), (V), and (VI) until the maximum number of iterations max_iterate is reached; (8) Testing phase: Collect test data n t The number of dominant variable values collected for the test set, using the training data X 0 's mean mean(X 0 ) and standard deviation std(X 0 ) to standardize the test data X new to obtain the standardized test data and perform inverse standardization on the prediction results to obtain the estimated values corresponding to X new ​ 2. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 1, characterized in that, In the said step (1), first, the mean value mean(X ) and the mean square deviation std(X 0 ) of the training data 0 are used to perform standardization processing on the training data X 0 through formula (1). The expression of formula (1) is: In formula (1), mean(·) represents calculating the mean of each column of the matrix, and std(·) represents calculating the standard deviation of each column of the matrix, obtaining the standardized training data For Standardization is performed using formula (2):

3. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 2, characterized in that, In the step (2), for the initialized variable weighted matrix calculate the distance between each pair of samples and select the k samples closest to the i-th sample x i to construct the initial Laplacian matrix The specific steps are as follows: First, initialize the variable weighted matrix M through formula (3), and formula (3) is expressed as: In formula (3), M is a diagonal matrix with elements on the diagonal being , diag(·) is the operation for constructing a diagonal matrix, is a column vector composed of the initial values m i , i = 1, 2, … d; then, the initial Laplacian matrix L is calculated according to formulas (4) - (7), and the process is as follows: The s in formula (4) ij is the similarity coefficient between the i-th sample x i and the j-th sample x j in the standardized training data X. is the similarity column vector composed of the similarity coefficients between the i-th sample x i in X and other samples in X. is the transpose of s i , and the superscript T represents the transpose operation. is a column vector with all elements being 1, and γ is the similarity regularization coefficient; in formula (5), S is the adjacency matrix corresponding to X. is the similarity matrix composed of the similarity column vectors s i , i = 1, 2, …, n. is the transpose of S 1 ; in formula (6), L is the Laplacian matrix corresponding to X, and the diagonal matrix D is the degree matrix corresponding to X; in formula (7), D ii is the i-th element on the diagonal of the diagonal matrix D; then solve formula (4) according to formulas (8) - (12): Introduce the Lagrange multipliers η and ζ i , where i = 1, 2, 2n, and further transform formula (9) into formula (10): In formula (10), η is the Lagrange multiplier corresponding to the constraint condition , ζ i is the Lagrange multiplier corresponding to the non - negative constraint of the similarity column vector s i ; is the transpose of ζ i ; Taking the partial derivative of formula (10) with respect to s i and setting it to 0, we can obtain According to the KTT condition, the optimal solution is shown in formula (11). In formula (11) where (·) + means taking the value itself when the value inside the parentheses is greater than 0, and taking 0 when it is less than or equal to 0; after expanding formula (11), it can be written in the form of formula (12): To obtain a sparse similarity column vector s consisting of only k non-zero elements i , formula (12) is further simplified to formula (13) as follows: Since γ is related to k, k is an integer and 0 ≤ k ≤ n, the parameter γ can be calculated by formula (14): Substitute η and γ into formula (11), and finally the similarity coefficient s can be obtained. ij :

4. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 3, characterized in that, in the said step (III), randomly generate the weight matrix between the input layer and the hidden layer of the extreme learning machine and the bias term matrix and calculating the hidden layer output using an activation function The specific steps are as follows: First, within the range of (-1, 1), randomly generate the weight matrix W between the input layer and the hidden layer of the extreme learning machine in and the bias term matrix B in ; Then, the output matrix of the hidden layer of the extreme learning machine is calculated using the sigmoid function and the data X The expression of the sigmoid function is shown in formula (16): Finally, merge the hidden layer output H 0 and the input data X row by row to obtain the augmented data matrix H = [H 0 , X].

5. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 4, characterized in that, In step (iv), update the weights between the hidden layer and the output layer bias b and the model prediction label value The specific steps are as follows: First, organically integrate the extreme learning machine, variable weighting, and adaptive local graph construction into a unified optimization objective function, and minimize the optimization problem as shown in formula (17): In formula (17), Tr(·) represents the matrix trace operation, represents the square of the l 2 norm, 1 represents the column vector with all elements being 1, and the diagonal matrix U = diag(β, β, …, β, 0, 0, …, 0) ∈ R n×n , that is, assign a certain weight β to the first n l tag values, λ, μ, β, θ are given regularization parameters, and w, b, f are the weights, biases, and model predicted tag values between the hidden layer and the output layer respectively; Then, fix the similarity matrix Variable weighted matrix The optimization problem description for w, b, and f is obtained as shown in Equation (18), and thus the analytical expressions for w and b can be obtained as shown in Equation (19): In Equation (19), A = λ(λHH C H T + I q×q ) -1 H T H C , where let q = d + n h , I represents the identity matrix, 1 represents the column vector with all elements being 1. Substituting Equation (19) into the objective function Hw + 1 n×1 b in Equation (18) gives Equation (20): In formula (20) Finally, according to formula (19) and formula (20), transform the optimization problem formula (18) into formula (21): Take the partial derivative of f and set it to 0 to obtain its optimal solution as shown in formula (22), f = (U + L + μλH C - μλ 2 N) -1 Uy (22) Among them X C = HH C .

6. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 5, characterized in that, In the step (v), a similarity matrix is used to update the variable weighted matrix The specific steps are as follows: First, fix the output f and the similarity matrix S, and the objective function is simplified from formula (17) to formula (23): Then, solve formula (23) to obtain the update formula (24) of the variable weighted matrix M: where t i = z ii , Z = X T LX,z ii is the element on the i-th main diagonal of matrix Z.

7. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 6, characterized in that, In step (vi), update the similarity matrix The specific steps are as follows: First, fix the output Variable weighting matrix The objective function is simplified from formula (17) to formula (25): Then, formula (25) can be further simplified to solve the optimization problem as shown in formula (26): Finally, similarly obtain the update formula (27) of the similarity matrix S according to the principle in step (II), In Formula (27), 8. The semi-supervised soft sensor modeling method for extreme learning machine based on variable weighted adaptive local graph construction according to claim 7, characterized in that, in the said step (VIII), the specific steps in the test phase are: First, for the n t collected test data using the training data mean value mean(X 0 ) and standard deviation std(X 0 ), the test data X new is standardized through formula (28), and formula (28) is expressed as: Then, according to the standardized test data Calculate the output value of the test data through Formula (29) and Formula (30) Formula (29) and Formula (30) are respectively expressed as: H t0 = X test W in + B in (29) Among them, is the output of the hidden layer. The output H of the hidden layer t0 and the input data X test are merged row by row to obtain the augmented data matrix Finally, the prediction result y test is inverse-normalized to obtain X new the corresponding estimated value as shown in formula (31): y new = y test × std(y 0 ) + mean(y 0 ) (31).

Images