A semi-supervised soft-sensing modeling method for width learning by integrating attention mechanism and adaptive composition

Abstract

This invention relates to a width-learning semi-supervised soft sensor modeling method that integrates attention mechanisms and adaptive graphing. This method comprehensively utilizes weighted Euclidean distance information from the data input space and the prediction output space to adaptively construct a nearest neighbor graph, achieving accurate approximation of the latent structural information of the data. Simultaneously, considering that different auxiliary variables contribute differently to the accurate estimation of the dominant variable, an attention mechanism is incorporated to weight the variables. Important information receives increased weight, while information that does not improve model performance is weighted less, thus reducing the adverse effects of redundant variables and noise on graphing and regression learning. Finally, the attention mechanism, adaptive graphing, and width-learning modeling are integrated into a unified learning framework, and an alternating iterative optimization method is used to obtain the overall optimal solution for modeling. Therefore, the semi-supervised learning framework provided by this invention can fully utilize the supervisory information contained in labeled data, supplemented by the structural information inherent in unlabeled data, to improve the performance of the width-learning model, thereby enhancing the generalization ability and reliability of the soft sensor model.

Description

Technical Field

[0001] This invention belongs to the field of industrial process detection technology, and relates to industrial process soft measurement technology. Specifically, it relates to a width learning semi-supervised soft measurement modeling method that integrates attention mechanism and adaptive graphing. Background Technology

[0002] Modern industrial production processes are rapidly evolving towards digitalization and intelligence, while simultaneously demanding increasingly stringent product quality control. To this end, actual production facilities are typically equipped with numerous industrial sensors to measure operational parameters reflecting the process's operating status in real time, providing essential feedback information for achieving closed-loop optimization control of product quality. However, industrial processes always contain some crucial parameters closely related to product quality that are difficult to measure directly, such as product concentration, composition, and various physical properties. Currently, these critical quality-related parameters can only be obtained through offline sampling followed by laboratory analysis, which suffers from long measurement cycles, significant feedback lag, and high costs in terms of manpower and resources. Therefore, when process conditions change, operators cannot promptly grasp the true operating status of the process, making it difficult to provide correct adjustment strategies, leading to reduced production efficiency and even jeopardizing operational safety. Soft measurement technology has developed against this backdrop. It achieves indirect estimation of key quality-related parameters by establishing mathematical models between easily measurable process variables (also known as auxiliary variables) and difficult-to-measure quality-related variables (also known as dominant variables). Compared with laboratory analysis or online component instruments, it has significant advantages such as low application and maintenance costs and timely response, and therefore has a wide range of applications in many industrial fields such as oil refining, chemical industry, metallurgy and pharmaceuticals.

[0003] The key to soft measurement technology lies in establishing a mathematical model that can accurately describe the latent functional relationship between auxiliary and dominant variables. If there is a deep understanding of the production process and rich knowledge in related fields, mechanistic modeling methods can be used to build a soft measurement model. However, the complexity and uncertainty of modern industrial processes greatly limit the application scope of mechanistic modeling methods. Therefore, data-driven regression modeling methods, due to their independence from specific domain expertise and their greater universality and flexibility, have been widely used in the field of soft measurement in recent decades. Representative techniques include principal component regression, partial least squares, artificial neural networks, and support vector machines. In particular, with the advent of the big data era, deep learning-based artificial neural network algorithms, such as convolutional neural networks, recurrent neural networks, and autoencoders, have become a research hotspot in the field of soft measurement modeling in recent years, achieving a series of remarkable results.

[0004] The performance of data-driven soft measurement modeling methods largely depends on the quantity and quality of training data. Specifically, to obtain a soft measurement model with strong generalization ability, training with a large input-output dataset covering the main operating conditions of the process is required. This is especially true for machine learning models with complex structures and numerous adjustable parameters, such as artificial neural networks. However, for practical soft measurement modeling problems, the sampling rate of the dominant variable (corresponding to the output variable of the soft measurement model) is generally much lower than the sampling frequency of the auxiliary variables (corresponding to the input variables of the soft measurement model). This results in only a small portion of the actual collected training data having both input and output values, while the vast majority of the data only has input values, with the corresponding output values ​​missing. In the field of machine learning, data with values ​​at both the input and output ends is called labeled data, while data with values ​​only at the input end is called unlabeled data. Currently, most data-driven soft measurement modeling methods adopt supervised learning, that is, using only labeled data for modeling while ignoring the role of unlabeled data. When labeled samples are scarce, overfitting is easily observed, and the model's generalization ability and reliability cannot be guaranteed, failing to meet the needs of practical applications. In fact, unlabeled data contains rich information about data structure. A series of studies have shown that properly utilizing the information contained in unlabeled data can significantly improve the performance of regression models. Therefore, the use of semi-supervised learning methods, that is, simultaneously utilizing a small amount of labeled data and a large amount of unlabeled data to build soft measurement mathematical models, is receiving increasing attention.

[0005] However, even in mainstream deep learning networks where the network model structure is quite stable, the time consumption during network training remains a significant drawback. This is mainly due to the complexity of deep learning network structures and the use of many hyperparameters, making further analysis of the network structure more difficult. Pao proposed the Stochastic Vector Function Chain Neural Network, which can effectively solve the problem of long model training time to some extent. Based on the former, Chen et al. (2018) proposed the width learning model, which not only greatly shortens the time required for model training but also uses mapped features as input during data feature extraction, allowing for more efficient model updates after the addition of massive amounts of new data. To improve the generalization performance of width learning, Feng et al. (2018) proposed a fuzzy width system. Addressing the challenges of small sample sizes and difficult labeling in specific datasets, Kong et al. (2019) proposed unsupervised width learning for hyperspectral image clustering. They employed a sparse encoder to fine-tune random weights and optimized the connection weights using graph regularization and the l2 norm of the output weights as the cost function. Based on the Manifold Regularization (MR) framework, Zhao et al. (2020) proposed semi-supervised width learning, further enriching the theoretical framework of width learning. Research shows that the MR-based semi-supervised learning framework can simultaneously utilize the supervisory / discriminative information provided by labeled data and the structural information inherent in unlabeled data to improve the model's generalization ability and reliability. It possesses a concise descriptive form and a solid theoretical foundation, thus finding many successful applications in the field of soft sensor modeling.

[0006] It is worth noting that an important prerequisite for the performance improvement of semi-supervised learning methods based on MR is that the constructed nearest neighbor graph can accurately approximate the potential local popular structure of the data. Considering that the popular structure of the data is unknown in advance and is problem-related, many methods for constructing nearest neighbor graphs have been proposed, such as k-nearest neighbor method, local linear representation, sparse self-representation and low-rank self-representation, and have been successfully applied in many different research fields. However, most existing methods construct the nearest neighbor graph offline in the original high-dimensional input space of the data using unsupervised learning. This may cause the following two problems: (1) In the actual modeling data, there are inevitably redundant auxiliary variables and noise. During the graph construction process, this redundant information may seriously affect the calculation of similarity between data, resulting in incorrect connections between nodes in the constructed nearest neighbor graph. (2) Existing methods generally adopt the offline graph construction method. Graph construction and subsequent regression modeling learning are completed as two independent learning tasks, ignoring the inherent connection between graph construction and regression learning. Therefore, the supervision information provided by the label samples cannot be effectively utilized during graph construction, resulting in the problem that the constructed nearest neighbor graph is not compatible with the subsequent regression modeling task.

[0007] In summary, existing semi-supervised learning methods based on mapping (MR) are prone to problems such as weak model generalization ability and poor reliability when used to solve practical soft sensor modeling problems. This is mainly because existing methods ignore the inherent connection between graph construction and regression modeling learning, resulting in the structure and parameters of the constructed graph failing to accurately describe the potential structural information of the data, and thus failing to achieve the goal of improving model performance using unlabeled samples. Summary of the Invention

[0008] This invention addresses the critical issue of the disconnect between graph construction and regression modeling in existing semi-supervised soft sensor modeling techniques based on popular regularization. It provides a width-learning semi-supervised soft sensor modeling method that integrates attention mechanisms and adaptive graph construction. This method tightly links attention mechanisms, adaptive graph construction, and width-learning modeling into a unified optimization learning framework for joint solution. Specifically, this invention comprehensively utilizes weighted Euclidean distance information from the data input space and the prediction output space to adaptively construct a nearest neighbor graph, achieving accurate approximation of the latent structural information of the data. Simultaneously, considering that different auxiliary variables contribute differently to the accurate estimation of the dominant variable, an attention mechanism is introduced to weight the variables; that is, a certain proportion of weight is increased for important information, while the weight of information that does not improve model performance is reduced. Finally, the attention mechanism, adaptive graph construction, and width-learning modeling are integrated into a unified optimization framework, and alternating iterative solutions are used to achieve overall optimal modeling learning. Therefore, the semi-supervised learning framework provided by this invention can fully utilize the supervision information contained in labeled data, supplemented by the structural information contained in unlabeled data, to improve the generalization ability and reliability of soft sensor models.

[0009] To achieve the above objectives, this invention provides a width-learning semi-supervised soft measurement modeling method that integrates attention mechanisms and adaptive graph construction, comprising the following steps:

[0010] (I) Offline Modeling Phase: Collecting Laboratory Analysis Values ​​of the Dominant Variable y i and the corresponding auxiliary variable measurement values Where i = 1, 2, ..., n l n l d represents the number of values ​​for the dominant variable collected, and d is the dimension of the auxiliary variables; additionally, n values ​​are collected. u Measurement values ​​of auxiliary variables Define n = n l +n u The number of auxiliary variable values ​​collected; sort the collected auxiliary variable values ​​by row to obtain the auxiliary variable data matrix. The superscript T indicates the matrix transpose operation. Accordingly, sorting the collected dominant variable values ​​by row yields the dominant variable data row vector. Furthermore, define nu Row vector with all zeros y l and y u Merge into row vectors Standardizing X0 using its mean (mean(X0)) and standard deviation (std(X0)) yields the following results: Standardizing y0 using its mean (mean(y0)) and standard deviation (std(y0)) yields... Obtain the offline training data X, y for width learning;

[0011] (ii) Specify the width and the number of feature nodes n in each group for learning c The number of augmented nodes is n s Regularization parameters β, λ, μ, θ and maximum number of iterations max_iterate, initializing the weighted matrix of variables. Calculate the distance between each sample and select the sample x that is closest to the i-th sample. i The initial Laplacian matrix is ​​constructed from the k most recent samples.

[0012] (III) Randomly generate N sets of weight matrices between the input layer and the feature layer for width learning. and bias term matrix Calculate feature layer output using linear mapping Randomly generate M sets of weight matrices between feature layers and enhancement layers. and bias term matrix The enhancement layer output is calculated using a nonlinear activation function. This leads to the output augmented data matrix.

[0013] (iv) Update the weights between the hidden layer and the output layer Bias b and model predicted label values

[0014] (V) Utilizing Similarity Matrices Update the variable weight matrix

[0015] (vi) Updating the similarity matrix

[0016] (vii) Repeat steps (iv), (v), and (vi) until the maximum number of iterations max_iterate is reached;

[0017] (VIII) Online Testing Phase: Collecting Test Data X new Using the mean (x0) and standard deviation (std(x0)) of the training data X0, the test data X is analyzed. new Standardization is performed to obtain standardized test data. Learning prediction results for width Perform destandardization to obtain X new Corresponding estimated value

[0018] Furthermore, in step (i), the training data is first utilized. The mean (X0) and standard deviation (X0) of the training data X0 are standardized using formula (1), which is expressed as follows:

[0019]

[0020] In the formula, mean(·) calculates the mean of each column of the matrix, and std(·) calculates the standard deviation of each column of the matrix, thus obtaining the standardized training data. right A similar standardization process is also needed, such as in formula (2):

[0021]

[0022] Furthermore, in step (ii), the initialization variable weighting matrix is... Calculate the distance between each sample and select the sample x that is closest to the i-th sample. i The initial Laplacian matrix is ​​constructed from the k most recent samples. The specific steps are as follows;

[0023] First, the variable weighting matrix M is initialized using formula (3), which is expressed as:

[0024]

[0025] Where M represents all elements on the diagonal. A diagonal matrix with all other elements being 0.

[0026] Secondly, the initial Laplace matrix L is calculated using formulas (4)-(7), and the process is as follows:

[0027]

[0028]

[0029]

[0030]

[0031] In the formula, D ii S is the i-th element on the diagonal of the diagonal matrix D. ijLet be the element in the i-th row and j-th column of the similarity matrix S, and let L be the Laplacian matrix corresponding to the dataset X. Then, solve formula (4) according to formulas (8)-(11):

[0032]

[0033]

[0034] Formula (9) can be rewritten as the Lagrange formula (10) by defining two Lagrange multipliers. Formula (10) with respect to s... i Taking the partial derivative and setting it to 0, we can obtain According to the KTT conditions, the optimal solution is shown in formula (11).

[0035]

[0036]

[0037] in in(·) + The expression within the parentheses indicates that the value is itself when it is greater than 0, and 0 when it is less than or equal to 0. Using formula (12), we can obtain the sparse representation of only k non-zero elements. i ,

[0038]

[0039] Equation (12) can be further simplified to Equation (13):

[0040]

[0041] Since γ is related to k, where k is an integer and 0≤k≤n, the parameter γ can be expressed as formula (14):

[0042]

[0043] Substituting η and γ into formula (11), we get:

[0044]

[0045] Furthermore, in step (iii), N sets of weight matrices between the width learning input layer and the feature layer are randomly generated. and bias term matrix Calculate feature layer output using linear mapping Randomly generate M sets of weight matrices between feature layers and enhancement layers. and bias term matrix The enhancement layer output is calculated using a nonlinear activation function. This leads to the output augmented data matrix. The specific steps are as follows:

[0046] First, within the range (-1, 1), the weight matrix W between the width-learning input layer and the feature layer is randomly generated. ei and bias term matrix B ei The feature layer Z is obtained through linear mapping;

[0047] Z i =φ i (XW ei +B ei ), i = 1, 2, ..., N (16)

[0048] In the formula, Let n represent the i-th group of mapping features. c φ represents the number of feature nodes in each group. i (·) denotes a linear mapping; each mapping contains n c There are N feature nodes, thus the N sets of mappings contain a total of Nn c The feature nodes, through N sets of mappings, can partially cancel out the errors caused by the random generation of weights and biases, thereby improving the accuracy of the model. The output Z of all feature nodes is denoted by formula (17):

[0049]

[0050] Then, the output matrix H of the width learning enhancement layer is calculated using a nonlinear function and the data Z, as shown in Equations (18)-(19):

[0051]

[0052]

[0053] In the formula, Let W represent the mapping feature of the j-th group. hj B hj Indicates its weights and biases, n s This indicates the number of augmentation nodes in each group. Represents a nonlinear activation function;

[0054] Finally, the feature layer output Z and the enhancement layer output matrix H are merged row by row to obtain the augmented data matrix A as shown in formula (20);

[0055] A = [Z1, Z2, ..., Z] N ,H1,H2,…,H M (20)

[0056] In the formula, q = Nn c +Mn s ;

[0057] Furthermore, in step (iv), the weights between the hidden layer and the output layer are updated. Bias b and model predicted label values

[0058] First, width learning, variable weighting, and adaptive local graph construction are organically integrated into a unified optimization objective function, which is minimized as shown in formula (21):

[0059]

[0060]

[0061] Where Tr(·) represents the trace operation of a matrix. Let l represent the square of the l2 norm, 1 represent a column vector with all elements equal to 1, and a diagonal matrix. That is, give the first n l Each label value is assigned a certain weight β, where λ, μ, β, θ are given regularization parameters, and w, b, and f represent the weights, biases, and model-predicted label values ​​between the hidden and output layers, respectively.

[0062] Then, fix the similarity matrix. Variable weighted matrix The optimization problem description for w, b, and f is given by equation (22), and the analytical expressions for w and b are given by equation (23).

[0063]

[0064]

[0065] In formula (23), D = λ(λA) T H C A+I q×q ) -1 A T H C ,in, I represents the identity matrix, and 1 represents a column vector with all elements equal to 1. Substituting formula (23) into the objective function Aw+1 in formula (22) n×1 b yields formula (24):

[0066]

[0067] in,

[0068] Finally, based on formulas (23) and (24), the optimization problem in formula (22) can be transformed into formula (25):

[0069]

[0070] Taking the partial derivative of f and setting it to 0, we can obtain the analytical expression of f as shown in formula (26).

[0071] f=(U+L+μλH C -μλ 2 N) -1 Uy (26)

[0072] in X C =AH C ;

[0073] Furthermore, in step (v), a similarity matrix is ​​used. Update the variable weight matrix The specific steps are as follows: Formula (27) - Formula (28):

[0074] First, fixed output The similarity matrix S, and the objective function are simplified from formula (21) to formula (27):

[0075]

[0076] Then, by solving formula (27), we can obtain the update formula (28) for the variable weighting matrix M:

[0077]

[0078] Among them, t i =z ii Z n =X T LX,z ii For matrix Z n The element on the i-th main diagonal;

[0079] Furthermore, in step (vi), the similarity matrix is ​​updated. The steps are formula (25) - formula (27):

[0080] First, fixed output Variable weighted matrix The objective function is simplified from formula (21) to formula (29):

[0081]

[0082] Then, equation (29) can be further simplified to solving the optimization problem shown in equation (30):

[0083]

[0084] Finally, based on the principle in step (ii), the update formula (31) for the similarity matrix S is obtained.

[0085]

[0086] In formula (31),

[0087] Furthermore, in step (vii), steps (iv), (v), and (vi) are repeated until the maximum number of iterations, max_iterate, is reached, and the model parameters are output. b.

[0088] Furthermore, in step (eight), the specific steps of the online testing phase are as follows:

[0089] First, for the collected n t Test data Using training data The mean (X0) and standard deviation (X0) of the test data are expressed by formula (32). new After standardization, formula (32) is expressed as:

[0090]

[0091] Then, based on the standardized test data The output value of the test data is calculated using formulas (33)-(35). Formulas (33), (34), and (35) are expressed as follows:

[0092] Z ti =φ i (X test W ei +B ei ), i = 1, 2, ..., N (33)

[0093]

[0094]

[0095] Among them, the feature layer output Z t and enhancement layer output data H t The augmented data matrix is ​​obtained by merging rows.

[0096] Finally, regarding the predicted result y test Perform destandardization to obtain X new Corresponding estimated value As shown in formula (36):

[0097] y new =y test ×std(y0)+mean(y0) (36)

[0098] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0099] This invention provides a width-learning semi-supervised soft sensor modeling method that integrates attention mechanisms and adaptive graph construction. It tightly links attention mechanisms, adaptive graph construction, and width-learning modeling, forming a unified optimization learning framework for joint solution. Specifically, on one hand, this invention comprehensively utilizes weighted Euclidean distance information from the data input space and the prediction output space to adaptively construct a nearest neighbor graph, achieving accurate approximation of the latent structural information of the data. On the other hand, considering that different auxiliary variables contribute differently to the accurate estimation of the dominant variable, the variable weights are adjusted by incorporating an attention mechanism. This means increasing the weight of important information and decreasing the weight of information that cannot improve model performance. Through continuous model learning and weight adjustment, the adverse effects of redundant variables and noise on graph construction and regression modeling are reduced. Compared to other existing algorithms, this invention integrates attention mechanisms, adaptive graph construction, and width-learning modeling into a unified optimization framework and uses alternating iterative solutions to achieve overall optimal modeling learning. This method can fully utilize the supervisory information contained in labeled data, supplemented by the structural information inherent in unlabeled data, to improve the generalization ability and reliability of soft sensor models. Attached Figure Description

[0100] Figure 1 This is a flowchart of the width learning semi-supervised soft measurement modeling method (AA-SS-BLS) that integrates attention mechanism and adaptive graph construction as described in this invention;

[0101] Figure 2 This is a schematic diagram of the butane removal tower process described in an embodiment of the present invention;

[0102] Figure 3 The image shows the prediction results of the Adaptive Local Graph Semi-Supervised Width Learning Model (A-SS-BLS) on the test set.

[0103] Figure 4 This is a graph showing the prediction results of the method on the test set described in this invention.

[0104] Figure 5 Histogram of error distribution on the test set of the semi-supervised width learning model (SS-BLS) based on this model;

[0105] Figure 6 This is a histogram of the error distribution of the test set for the method described in this invention.

[0106] Figure 7 A scatter plot of the test set between the true and predicted values ​​of a semi-supervised width learning model based on this model.

[0107] Figure 8 This is a scatter plot showing the relationship between the actual and predicted values ​​in the test set of the method described in this invention.

[0108] Figure 9 Box plots showing the coefficients of determination for the three models on the test set under optimal parameters;

[0109] Figure 10 A graph showing the impact of different iterations on the coefficients of determination of the training and test sets of the butanizer under an adaptive local graph semi-supervised width learning model;

[0110] Figure 11 This is a graph showing the influence of different iteration numbers on the coefficient of determination of the training and test sets of the butanizer under the method described in this invention; Detailed Implementation

[0111] The present invention will now be described in detail through exemplary embodiments. However, it should be understood that, without further description, elements, structures, and features in one embodiment may be advantageously incorporated into other embodiments.

[0112] See Figure 1 This invention discloses a width-learning semi-supervised soft measurement modeling method that integrates attention mechanisms and adaptive graph construction, comprising the following steps:

[0113] (I) Offline Modeling Phase: Collecting Laboratory Analysis Values ​​of the Dominant Variable y i and the corresponding auxiliary variable measurement values Where i = 1, 2, ..., n l n l d represents the number of values ​​for the dominant variable collected, and d is the dimension of the auxiliary variables; additionally, n values ​​are collected. u Measurement values ​​of auxiliary variables Define n = n l +n u The number of auxiliary variable values ​​collected; the collected auxiliary variable values ​​are sorted by row to obtain the auxiliary variable data matrix. The superscript T indicates the matrix transpose operation. Accordingly, the collected dominant variable values ​​are sorted by row to obtain the dominant variable data row vector. Furthermore, define n u Row vector with all zeros y l and y u Merge into row vectors Standardizing X0 using its mean (mean(X0)) and standard deviation (std(X0)) yields the following results: Standardizing y0 using its mean (mean(y0)) and standard deviation (std(y0)) yields... To obtain the offline training data X, y for width learning, the specific steps are as follows:

[0114] Using training data The mean (X0) and standard deviation (X0) of the training data X0 are standardized using formula (1), which is expressed as follows:

[0115]

[0116] In equation (1), mean(·) represents calculating the mean of each column of the matrix, and std(·) represents calculating the standard deviation of each column of the matrix to obtain the standardized training data. right A similar standardization process is also needed, such as in formula (2):

[0117]

[0118] (ii) Specify the width and the number of feature nodes n in each group for learning c The number of augmented nodes is n s Regularization parameters β, λ, μ, θ and maximum number of iterations max_iterate, initializing the weighted matrix of variables. Calculate the distance between each sample and select the sample x that is closest to the i-th sample. i The initial Laplacian matrix is ​​constructed from the k most recent samples. The specific process is as follows:

[0119] First, the variable weighting matrix M is initialized using formula (3), which is expressed as:

[0120]

[0121] Where M represents all elements on the diagonal. A diagonal matrix with all other elements being 0.

[0122] Secondly, the initial Laplace matrix L is calculated using formulas (4)-(7), and the process is as follows:

[0123]

[0124]

[0125]

[0126]

[0127] In the formula, D ii S is the i-th element on the diagonal of the diagonal matrix D. ij Let be the element in the i-th row and j-th column of the similarity matrix S, and let L be the Laplacian matrix corresponding to the dataset X. Then, solve formula (4) according to formulas (8)-(11):

[0128]

[0129]

[0130] Formula (9) can be rewritten as the Lagrange formula (10) by defining two Lagrange multipliers. Formula (10) with respect to s... i Taking the partial derivative and setting it to 0, we can obtain According to the KTT conditions, the optimal solution is shown in formula (11).

[0131]

[0132]

[0133] in in(·) + The expression within the parentheses indicates that the value is itself when it is greater than 0, and 0 when it is less than or equal to 0. Using formula (12), we can obtain the sparse representation of only k non-zero elements. i ,

[0134]

[0135] Equation (12) can be further simplified to Equation (13):

[0136]

[0137] Since γ is related to k, where k is an integer and 0≤k≤n, the parameter γ can be expressed as formula (14):

[0138]

[0139] Substituting η and γ into formula (11), we get:

[0140]

[0141] (III) Randomly generate N sets of weight matrices between the input layer and the feature layer for learning width. and bias term matrix Calculate feature layer output using linear mapping Randomly generate M sets of weight matrices between feature layers and enhancement layers. and bias term matrix The enhancement layer output is calculated using a nonlinear activation function. This leads to the output augmented data matrix. The specific steps are as follows:

[0142] First, within the range (-1, 1), the weight matrix W between the width-learning input layer and the feature layer is randomly generated. ei and bias term matrix B ei The feature layer Z is obtained through linear mapping;

[0143] Z i =φ i (XW ei +B ei ), i = 1, 2, ..., N (16)

[0144] In the formula, Let n represent the i-th group of mapping features. c φ represents the number of feature nodes in each group. i (·) denotes a linear mapping; each mapping contains n c There are N feature nodes, thus the N sets of mappings contain a total of Nn c The feature nodes, through N sets of mappings, can partially cancel out the errors caused by the random generation of weights and biases, thereby improving the accuracy of the model. The output Z of all feature nodes is denoted by formula (17):

[0145]

[0146] Then, the output matrix H of the width learning enhancement layer is calculated using a nonlinear function and the data Z, as shown in Equations (18)-(19):

[0147]

[0148]

[0149] In the formula, Let W represent the mapping feature of the j-th group. hj B hj Indicates its weights and biases, n s This indicates the number of augmentation nodes in each group. Represents a non-linear activation function;

[0150] Finally, the feature layer output Z and the enhancement layer output matrix H are merged row by row to obtain the augmented data matrix. As shown in formula (20);

[0151] A = [Z1, Z2, ..., Z] N ,H1,H2,…,H M (20)

[0152] In the formula, q = Nn c +Mn s ;

[0153] (iv) Update the weights between the hidden layer and the output layer Bias b and model predicted label values The specific process is as follows:

[0154] First, the width learning, attention mechanism, and adaptive local graph construction are organically integrated into a unified optimization objective function, which is minimized as shown in Equation (21):

[0155]

[0156]

[0157] Where Tr(·) represents the trace operation of a matrix. Let l represent the square of the l2 norm, 1 represent a column vector with all elements equal to 1, and a diagonal matrix. That is, give the first n l Each label value is assigned a certain weight β, where λ, μ, β, θ are given regularization parameters, and w, b, and f represent the weights, biases, and model-predicted label values ​​between the hidden and output layers, respectively.

[0158] Then, fix the similarity matrix. Variable weighted matrix The optimization problem description for w, b, and f is given by equation (22), and the analytical expressions for w and b are given by equation (23).

[0159]

[0160]

[0161] In formula (23), D = λ(λA) T H C A+I q×q ) -1 A T H C , I represents the identity matrix, and 1 represents a column vector with all elements equal to 1. Substituting formula (23) into the objective function Aw+1 in formula (22) n×1 b yields formula (24):

[0162]

[0163] in, Finally, based on formulas (23) and (24), the optimization problem in formula (22) can be transformed into formula (25):

[0164]

[0165] Taking the partial derivative of f and setting it to 0, we can obtain the analytical expression of f as shown in formula (26).

[0166] f=(U+L+μλH C -μλ 2 N) -1 Uy (26)

[0167] in X C =AH C ;

[0168] (V) Utilizing Similarity Matrices Update the variable weight matrix The specific process is as follows:

[0169] First, fixed output The similarity matrix S, and the objective function are simplified from formula (21) to formula (27):

[0170]

[0171] Then, by solving formula (27), we can obtain the update formula (28) for the variable weighting matrix M:

[0172]

[0173] Among them, t i =z ii Z n =X T LX,z ii For matrix Z n The element on the i-th main diagonal;

[0174] (vi) Updating the similarity matrix The specific process is as follows:

[0175] First, fixed output Variable weighted matrix The objective function is simplified from formula (21) to formula (29):

[0176]

[0177] Then, equation (29) can be further simplified to solving the optimization problem shown in equation (30):

[0178]

[0179] Finally, based on the principle in step (ii), the update formula (31) for the similarity matrix S is obtained.

[0180]

[0181] In formula (31),

[0182] (vii) Repeat steps (iv), (v), and (vi) until the maximum number of iterations, max_iterate, is reached. The specific process is as follows:

[0183] Repeat steps (iv), (v), and (vi) until the maximum number of iterations, max_iterate, is reached, and then output the model parameters. b.

[0184] (VIII) Online Testing Phase: For the collected n t Test data First, using training data The mean (X0) and standard deviation (X0) of the test data are expressed by formula (32). new After standardization, formula (32) is expressed as:

[0185]

[0186] Then, based on the standardized test data The output value of the test data is calculated using formulas (33)-(35). Formulas (33), (34), and (35) are expressed as follows:

[0187] Z ti =φ i (X test W ei +B ei ), i = 1, 2, ..., N (33)

[0188]

[0189]

[0190] Among them, the feature layer output Z t and enhancement layer output data H t The augmented data matrix is ​​obtained by merging rows.

[0191] Finally, regarding the predicted result y test Perform destandardization to obtain X new Corresponding estimated value As shown in formula (36):

[0192] y new =y test ×std(y0)+mean(y0) (36)

[0193] The method described in this invention integrates attention mechanisms, adaptive graphing, and width learning modeling into a unified learning framework, and employs alternating iterative optimization to obtain the overall optimal solution for modeling and learning. This method comprehensively utilizes weighted Euclidean distance information from the data input space and the prediction output space to adaptively construct a nearest neighbor graph, achieving accurate approximation of the data's latent structural information. The width learning, which integrates attention mechanisms and adaptive local graphing, performs variable weighted learning of the input samples based on the width learning of adaptive local graphing. By incorporating an attention mechanism, different auxiliary variables are assigned different weights to reduce the adverse effects of redundant variables and noise on graphing and regression learning. This method improves the performance of the width learning model by utilizing the supervisory information contained in labeled data and supplementing it with the structural information inherent in unlabeled data.

[0194] To illustrate the effectiveness of the width learning soft measurement modeling method that integrates attention mechanism and adaptive graph composition described above, the present invention will be further explained below with reference to specific embodiments.

[0195] Example: The process data of the butane removal tower is used as an example for illustration.

[0196] The butane removal distillation column is part of the desulfurization and naphtha separator unit. Its main task is to maximize the C5 (stabilized gasoline) content at the top of the butane removal column (LPG separator feed) and minimize the C4 (butane) content at the bottom of the butane removal column (Naptha separator feed). Its block diagram is shown below. Figure 2As shown. Besides the distillation column (T102), the butanizer also includes a heat exchanger (E105B), a top condenser (E107AB), a bottom reboiler (E108AB), a top reflux pump (P102AB), and a feed pump for the LPG separator (P103AB). The C5 content at the top of the butanizer is indirectly measured by an analyzer located at the bottom of the LPG fractionation column in unit 900. The measurement cycle of this analyzer is 10 minutes. Additionally, the location of the measuring device causes a delay, which is unknown but constant and likely ranges from 20 to 60 minutes. Similarly, the C4 content at the bottom of the butanizer cannot be directly detected at the bottom of the column; instead, it is detected by a gas chromatograph installed at the top. The measurement cycle of this analyzer is generally 15 minutes. Again, due to the installation location of the analyzer, there is a significant delay in obtaining the concentration value; this delay is not well-known but constant and likely ranges from 30 to 75 minutes. Therefore, to achieve real-time measurement of butane concentration and improve the control quality of the butane removal tower, it is necessary to establish a soft-sensor model to estimate the bottom butane concentration in real time. Furthermore, considering the low sampling efficiency and long time delay of quality variables in actual production, it is assumed that only one-fifth of all historical samples have labels (containing both input and output data), while the remaining historical samples are unlabeled samples (containing only input data).

[0197] The specific steps of this invention will now be described in conjunction with the butane removal tower production process:

[0198] 1. Offline modeling stage: The collected data is used as the training dataset and preprocessed.

[0199] First, all samples are preprocessed to remove outliers. Then, considering the dynamic nature of the process, the dimensions of all samples are expanded, resulting in 30 features per sample. Finally, standardization is performed to obtain the final offline training data. The collected quality variable values ​​are sorted by row to obtain the quality variable data row vector. Furthermore, define a vector with 1440 rows of all zeros. y l and y u Merge into row vectors Standardizing y0 using its mean (mean(y0)) and standard deviation (std(y0)) yields... Obtain the offline training data X, y for width learning;

[0200] 2. Construct the initial Laplacian matrix based on the training dataset.

[0201] The learning width is specified to include 130 feature nodes and 5 augmentation nodes per group, with regularization parameters β, λ, μ, and θ each having a value of 10. -1 10 -1 10 -3 0.05 and maximum iteration count 15, initialization variable weighting matrix Calculate the distance between each sample and select the sample x that is closest to the i-th sample. i The three most recent samples are used to construct the initial Laplacian matrix.

[0202] 3. Calculate the output of the feature layer and enhancement layer to obtain the final output.

[0203] 4. Update model parameters Variable weighted matrix Similarity matrix

[0204] 5. Repeat step 4 until the maximum number of iterations, 15, is reached;

[0205] 6. Online testing phase: Collect test data The test set contains 400 dominant variable values. The mean (mean(X0)) and standard deviation (std(X0)) of the training data X0 are used to evaluate the test data X. new Standardization is performed to obtain standardized test data. Learning prediction results for width Perform destandardization to obtain X new Corresponding estimated value

[0206] Using root mean square error (RMSE) and coefficient of determination (R²), 2 The predictive performance of the soft sensor model is comprehensively evaluated using three evaluation metrics: mean absolute error (MAE). The expressions for the three evaluation metrics are shown in formulas (37) to (39).

[0207]

[0208]

[0209]

[0210] In the formula, y i and Let be the true value and the predicted value of the target variable for the i-th sample, respectively. The target variable is the mean of all samples. The coefficient of determination R0 2The reliability of the prediction results can be measured; the closer the calculated result is to 1, the better the prediction performance of the soft sensor model. The prediction error of the soft sensor model is calculated using RMSE and MAE; the smaller the error value, the higher the prediction accuracy of the soft sensor model.

[0211] Table 1 shows the fitting results of the traditional semi-supervised width learning model (SS-BLS), the width learning model with adaptive local graphing (A-SS-BLS), and the width learning model of this invention that integrates attention mechanism and adaptive graphing (AA-SS-BLS) for the butanizer tower data in 10 simulation experiments under optimal parameters.

[0212] Table 1

[0213]

[0214] As shown in Table 1, the method provided by this invention achieved the best overall results, with the test set MAE,R... 2 Both RMSE and RMSE have improved to some extent.

[0215] Based on the above analysis, the width learning model that integrates attention mechanism and adaptive graph construction provided by this invention can not only comprehensively utilize the weighted Euclidean distance information of the data input space and the prediction output space to adaptively construct the nearest neighbor graph to accurately approximate the potential structural information of the data, but also assign different weights to different auxiliary variables through the integration of attention mechanism, thereby improving the generalization ability and reliability of the model.

[0216] Figure 3 , Figure 4 The image shows the prediction results on the test set of the semi-supervised width learning model (SS-BLS) and the method described in this invention (AA-SS-BLS). Figure 5 , Figure 6 Based on the semi-supervised width learning model and the error distribution histogram of the test set of the method described in this invention, Figure 7 , Figure 8 Based on the semi-supervised width learning model and the scatter plot of the test set between the true and predicted values ​​of the method described in this invention, Figure 9 Box plots of the coefficients of determination for the test set of the three models under optimal parameters in ten simulations are shown. The influence of the Adaptive Local Graph Semi-Supervised Width Learning Model (A-SS-BLS) and the method described in this invention on the coefficients of determination for the butanizer data at different iteration numbers is also shown in the figure. Figure 10 and Figure 11 As shown in the figure, the determination coefficients of the test set under both models show an upward trend and stabilize around the 8th iteration. In summary, the method described in this invention has higher prediction accuracy compared to traditional methods.

[0217] The above embodiments are used to explain the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A width-learning semi-supervised soft measurement modeling method that integrates attention mechanisms and adaptive graph construction, characterized in that, Includes the following steps: (I) Offline Modeling Phase: Collecting Laboratory Analysis Values ​​of the Dominant Variable y i and the corresponding auxiliary variable measurement values Where i = 1, 2, ..., n l n l d represents the number of values ​​for the dominant variable collected, and d is the dimension of the auxiliary variables; additionally, n values ​​are collected. u Measurement values ​​of auxiliary variables Define n = n l +n u The number of auxiliary variable values ​​collected; sort the collected auxiliary variable values ​​by row to obtain the auxiliary variable data matrix. The superscript T indicates the matrix transpose operation. Accordingly, sorting the collected dominant variable values ​​by row yields the dominant variable data row vector. Furthermore, define n u Row vector with all zeros y l and y u Merge into row vectors Standardizing X0 using its mean (mean(X0)) and standard deviation (std(X0)) yields the following results: Standardizing y0 using its mean (mean(y0)) and standard deviation (std(y0)) yields... Obtain the offline training data X, y for width learning; (ii) Specify the width and the number of feature nodes n in each group for learning c The number of augmented nodes is n s Regularization parameters β, λ, μ, θ and maximum number of iterations max_iterate, initializing the weighted matrix of variables. Calculate the distance between each sample and select the sample x that is closest to the i-th sample. i The initial Laplacian matrix is ​​constructed from the k most recent samples. (III) Randomly generate N sets of weight matrices between the input layer and the feature layer for width learning. and bias term matrix Calculate feature layer output using linear mapping Randomly generate M sets of weight matrices between feature layers and enhancement layers. and bias term matrix The enhancement layer output is calculated using a nonlinear activation function. This leads to the output augmented data matrix. (iv) Update the weights between the hidden layer and the output layer b and model predicted label values (v) Updating the variable weighting matrix using similarity matrices (vi) Updating the similarity matrix (vii) Repeat steps (iv), (v) and (vi) until the maximum number of iterations max_iterate is reached, and output the model parameters w and b; (VIII) Online Testing Phase: Data Collection n t The number of dominant variable values ​​collected for the test set is determined by using the mean (X0) and standard deviation (std(X0)) of the training data X0 to evaluate the test data X. new Standardization is performed to obtain standardized test data. Learning prediction results for width Perform destandardization to obtain X new Corresponding estimated value .

2. The width-learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 1, characterized in that, In step (i), the training data is first used. The mean (X0) and standard deviation (X0) of the training data X0 are standardized using formula (1), which is expressed as follows: In the formula, mean(·) represents calculating the mean of each column of the matrix, and std(·) represents calculating the standard deviation of each column of the matrix to obtain the standardized training data. right A similar standardization process is also needed, such as in formula (2): 。 3. The width-learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 2, characterized in that, In step (ii), the initialization variable weighting matrix is... Calculate the distance between each sample and select the sample x that is closest to the i-th sample. i The initial Laplacian matrix is ​​constructed from the k most recent samples. The specific steps are as follows; First, the variable weighting matrix M is initialized using formula (3), which is expressed as: Where M represents all elements on the diagonal. A diagonal matrix with all other elements being 0. Secondly, the initial Laplace matrix L is calculated using formulas (4)-(7), and the process is as follows: In the above formula, D ii S is the i-th element on the diagonal of the diagonal matrix D. ij Let be the element in the i-th row and j-th column of the similarity matrix S, and let L be the Laplacian matrix corresponding to the dataset X. Then, solve formula (4) according to formulas (8)-(11): By defining two Lagrange multipliers, formula (9) can be rewritten as a Lagrange formula (10). Then, formula (10) is interpreted with respect to s. i Taking the partial derivative and setting it to 0, we can obtain According to the KTT conditions, the optimal solution is shown in formula (11). in in(·) + The expression within the parentheses indicates that the value is itself when it is greater than 0, and 0 when it is less than or equal to 0. Using formula (12), we can obtain the sparse representation of only k non-zero elements. i , Equation (12) can be further simplified to Equation (13): Since γ is related to k, where k is an integer and 0≤k≤n, the parameter γ can be expressed as formula (14): Substituting η and γ into formula (11), we get: 。 4. The width-learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 3, characterized in that, In step (iii), N sets of weight matrices between the width learning input layer and the feature layer are randomly generated. and bias term matrix Calculate feature layer output using linear mapping Randomly generate M sets of weight matrices between feature layers and enhancement layers. and bias term matrix The enhancement layer output is calculated using a nonlinear activation function. This leads to the output augmented data matrix. The specific steps are as follows: First, within the range (-1, 1), N sets of weight matrices W between the width learning input layer and the feature layer are randomly generated. ei and bias term matrix B ei The feature layer Z is obtained through linear mapping; Z i =φ i (XW ei +B ei ),i=1,2,…,N (16) In the formula, Let n represent the i-th group of mapping features. c φ represents the number of feature nodes in each group. i (·) denotes a linear mapping; each mapping contains n c There are N feature nodes, thus the N sets of mappings contain a total of Nn c The feature nodes, through N sets of mappings, can partially cancel out the errors caused by the random generation of weights and biases, thereby improving the accuracy of the model. The output Z of all feature nodes is denoted by formula (17): Then, the output matrix H of the width learning enhancement layer is calculated using a nonlinear activation function and the data Z, as shown in Equations (18)-(19): In the formula, Let W represent the mapping feature of the j-th group. hj B hj Indicates its weights and biases, n s This indicates the number of augmentation nodes in each group. Represents a nonlinear activation function; Finally, the feature layer output Z and the enhancement layer output matrix H are merged row by row to obtain the augmented data matrix A as shown in formula (20); A=[Z1,Z2,…,Z N ,H1,H2,…,H M ] (20) In the formula, q = Nn c +Mn s .

5. The width-learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 4, characterized in that, In step (iv), the weights between the hidden layer and the output layer are updated. Bias b and model predicted label values The specific steps are as follows: First, the width learning, attention mechanism, and adaptive local graph construction are organically integrated into a unified optimization objective function, minimizing the optimization problem shown in Equation (21): Where Tr(·) represents the trace operation of a matrix. Let l represent the square of the l2 norm, 1 represent a column vector with all elements equal to 1, and a diagonal matrix. That is, give the first n l Each label value is assigned a certain weight β, where λ, μ, β, θ are given regularization parameters, and w, b, and f represent the weights, biases, and model-predicted label values ​​between the hidden and output layers, respectively. Then, fix the similarity matrix. Variable weighted matrix The optimization problem description for w, b, and f is given by equation (22), and the analytical expressions for w and b are given by equation (23). In formula (23), D = λ(λA) T H C A+I q×q ) -1 A T H C , I represents the identity matrix, and 1 represents a column vector with all elements equal to 1. Substituting formula (23) into formula (22) for Aw+1 n×1 b yields formula (24): In formula (24) Finally, based on formulas (23) and (24), the optimization problem formula (22) is transformed into formula (25): Taking the partial derivative of f and setting it to 0, we obtain the optimal solution as shown in formula (26). f=(U+L+μλH C -ml 2 N) -1 Uy (26) in X C =AH C .

6. The width-learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 5, characterized in that, In step (v), the similarity matrix is ​​used. Update the variable weight matrix The specific steps are as follows: First, fixed output The similarity matrix S, and the objective function are simplified from formula (21) to formula (27): Then, by solving formula (27), we can obtain the update formula (28) for the variable weighting matrix M: Among them, t i =z ii Z n =X T LX,z ii For matrix Z n The element on the i-th main diagonal.

7. The width learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 6, characterized in that, In step (vi), the similarity matrix is ​​updated. The specific steps are as follows: First, fixed output Variable weighted matrix The objective function is simplified from formula (21) to formula (29): Then, equation (29) can be further simplified to solving the optimization problem shown in equation (30): Finally, based on the principle in step (ii), the update formula (31) for the similarity matrix S is obtained. Formula (31) .

8. The width learning semi-supervised soft measurement modeling method integrating attention mechanism and adaptive graph construction as described in claim 7, characterized in that, In step (viii), the specific steps of the online testing phase are as follows: First, for the collected n t Test data Using training data The mean (X0) and standard deviation (X0) of the test data are expressed by formula (32). new After standardization, formula (32) is expressed as: Then, based on the standardized test data The output value of the test data is calculated using formulas (33)-(35). Formulas (33), (34), and (35) are expressed as follows: Z ti =φ i (X test W ei +B ei ),i=1,2,…,N (33) y test =A t w+1 nt×1 b (35) Among them, the feature layer output Z t and enhancement layer output data H t The augmented data matrix is ​​obtained by merging rows. Finally, regarding the predicted result y test Perform destandardization to obtain X new Corresponding estimated value As shown in formula (36): y new =y test ×std(y0)+mean(y0) (36) 。

Images