Hydrogen load prediction method for green ammonia production based on sparse attention variational bayes

Abstract

The method comprises the following steps: obtaining green ammonia electrolytic hydrogen production industrial historical data collected by a sensor, and dividing the historical data into a training set, a validation set and a test set; performing standardization processing on the training set, and constructing an SVAE soft measurement training model; determining a final target function suitable for SVAE; using the trained SVAE to perform real-time online prediction on the output target variable by using the current input feature variable. SVAE combines CNNs, PSSAM and cross-attention mechanism, and captures long-term dependence and complex mutual relationship in industrial data. An optimized target function is formulated by combining variational inference and Monte Carlo method, and is used for offline training. SVAE has significant advantages in accurately predicting hydrogen flow and optimizing green ammonia production process, improves production efficiency, and reduces energy consumption.

Description

Technical Field

[0001] This invention relates to the application of artificial intelligence technology in the field of automated measurement technology. Specifically, it relates to a method for predicting hydrogen load in the green ammonia production process using a sparse attention mechanism and variational Bayesian SVAE network model. Background Technology

[0002] Ammonia is a crucial raw material for fertilizer production, and hydrogen is one of the raw materials in the ammonia synthesis process. Traditional ammonia synthesis methods, such as the Haber process, typically rely on natural gas reforming technology to produce hydrogen. However, this process generates significant carbon dioxide emissions, which is inconsistent with the environmental protection and energy conservation requirements of global energy transition and sustainable development. With the development of renewable energy technologies, water electrolysis for hydrogen production, as a green and efficient method, is gradually becoming a major source of hydrogen. Water electrolysis for hydrogen production utilizes renewable energy sources such as wind and solar power (referred to as "green electricity") to drive an electrolyzer, decomposing water into hydrogen (referred to as "green hydrogen") and oxygen. Green hydrogen is used in ammonia production (referred to as "green ammonia"), offering the advantage of low carbon emissions. Therefore, the use of clean energy such as hydrogen as a raw material for green ammonia production is receiving increasing attention.

[0003] In the production of green ammonia, hydrogen is a core raw material with a huge demand. The flow rate of hydrogen directly affects the efficiency and reaction rate of ammonia synthesis. However, as the importance of hydrogen flow rate in green ammonia production continues to increase, accurately predicting the hydrogen flow rate during water electrolysis has become a major challenge in the field of chemical engineering. Therefore, accurately controlling the hydrogen flow rate and modeling and predicting the process is of great significance for improving production efficiency, reducing energy consumption, and optimizing resource allocation.

[0004] Traditional process modeling methods can generally be divided into two main categories: first-principles models (FPMs) and data-driven models (DDMs). First-principles models, based on physical and chemical laws, attempt to describe the relationships between process variables by establishing precise equations, and are suitable for simple systems with well-defined physical laws. However, when faced with complex multi-reaction chemical processes, first-principles models often struggle to solve the problem effectively. In contrast, thanks to advancements in big data storage and distributed control systems, data-driven models aim to develop accurate predictive models by utilizing the relationships between AVs (Ambient Variables) and QVs (Query Variables), demonstrating greater adaptability and flexibility when dealing with complex systems.

[0005] Among various data-driven modeling approaches, latent variable models (LVMs) are used to extract statistical information, but traditionally they rely on a linear mapping between observed and latent variables, limiting their utility when dealing with nonlinear data. With the development of deep learning techniques, especially multilayer neural networks (DNNs), the capabilities of LVMs have been expanded by transforming nonlinear phenomena into interpretable latent spaces. For example, variational autoencoders (VAEs) combine variational inference with the nonlinear approximation capabilities of deep neural networks, enhancing Bayesian interpretability and effectively capturing complex nonlinear patterns.

[0006] However, because VAEs assume that latent variables are time-independent, they often struggle with sequential data, relying primarily on feedforward neural networks for data reconstruction. This limitation restricts VAE applications to single-step prediction. To address the challenges of long-term sequence prediction, more complex deep learning models are needed, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and Transformers. Transformers, in particular, utilize self-attention mechanisms to synthesize global information, enhancing interpretability and focusing on selective features. The high computational demands of traditional self-attention mechanisms spurred the development of probabilistic sparse self-attention mechanisms, which reduce complexity and memory requirements, significantly improving the efficiency of analytical workflows. Summary of the Invention

[0007] To address the complex nonlinear problem of difficulty in modeling and predicting hydrogen load in the green ammonia electrolysis hydrogen production process, this invention proposes a soft sensor modeling method based on sparse attention variational Bayes and applies it to soft sensor modeling of chemical processes, achieving accurate prediction of key parameter components of chemical processes.

[0008] Based on the above research in existing technologies, this invention introduces a Sequence Variational Autoencoder (SVAE) within a deep Bayesian learning framework. SVAE combines Convolutional Neural Networks (CNNs), Probabilistic Sparse Self-Attention Mechanism (PSSAM), and cross-attention mechanisms to cleverly capture long-term dependencies and complex interrelationships in industrial data. By combining variational inference and Monte Carlo methods, SVAE formulates an optimized objective function for offline training, thereby significantly enhancing its ability to model nonlinear dynamics in chemical processes. Compared to traditional methods, the SVAE model demonstrates stronger modeling capabilities and prediction accuracy when handling prediction tasks in complex industrial processes. SVAE has significant advantages in accurately predicting hydrogen flow rates and optimizing green ammonia production, thereby improving production efficiency, reducing energy consumption, and promoting the development of green energy technologies.

[0009] The technical solution adopted in this invention is: a method for predicting hydrogen load in green ammonia production based on sparse attention variational Bayes, which uses a sequence variational autoencoder (SVAE) as the prediction model and utilizes the input feature variables in the green hydrogen ammonia synthesis process to output the target variable.

[0010] The input characteristic variables are auxiliary variables for the green hydrogen ammonia synthesis process, which generally include the DC voltage of the electrolyzer, the outlet pressure of the hydrogen purification system, the mass flow rate of the nitrogen inlet, the circulating gas pressure of the synthesis tower, the outlet temperature of the compressor, the liquid level of the ammonia separator, the real-time temperature of the catalyst bed, the temperature difference monitoring value of the cooling water system, the hydrogen / nitrogen molar ratio of the gas mixer, the flow rate of the waste gas circulation loop, and the buffer pressure of the hydrogen storage tank.

[0011] The target variable for output is the online predicted green hydrogen flow rate, such as the real-time green hydrogen flow rate at the inlet of the synthesis tower.

[0012] This method includes the following steps:

[0013] S1: Historical data of the green ammonia electrolysis hydrogen production industry is collected by sensors. The historical data includes input feature variables and output target variables. The historical data is divided into training set, validation set and test set. The training data is used to train the SVAE, and the test set data is used to verify the accuracy of the trained SVAE and realize online prediction.

[0014] S2: Standardize the training set obtained in S1 to obtain standardized training features and standardized parameters. Use the standardized parameters obtained from the training set standardization to standardize the validation set and test set, and build the SVAE soft measurement training model based on sparse attention and variational Bayes.

[0015] S3: Determining the final objective function applicable to SVAE can effectively iterate and update the model parameters to a local optimum, thereby achieving online estimation of quality variables and providing a strong foundation for subsequent process monitoring, optimization and control operations.

[0016] S4: Employ a trained SVAE soft measurement model to perform real-time online prediction of the output target variable using the current input feature variables.

[0017] The soft measurement model is a soft measurement model for green ammonia electrolysis hydrogen production based on a probabilistic sparse attention mechanism and a variational autoencoder; Reference Figure 1 The network structure of the soft measurement model SVAE is a network composed of an encoder module, a reparameterization module, and a decoder module; wherein,

[0018] A. The encoder module includes: a sparse attention encoding module and a cross attention module;

[0019] The A1 sparse attention coding module includes: a location embedding module and a multi-head sparse self-attention module;

[0020] The location embedding module is based on location information encoding and includes a convolutional layer, a location information encoder layer, and an activation layer. The location embedding module is used to improve the model's ability to perceive location information.

[0021] The multi-head sparse self-attention module is based on multi-head probability coefficient attention and includes a multi-head attention computation layer, a convolutional layer, a residual connection layer, and an activation layer.

[0022] The A2 cross-attention module, similar to the multi-head sparse self-attention module, includes a multi-head attention computation layer, a convolutional layer, a residual connection layer, and an activation layer; the sparse attention layer and the cross-attention layer are used to handle information interaction at different positions in the input sequence.

[0023] B. The reparameterization module makes the sampling process differentiable, thus enabling effective gradient updates through backpropagation.

[0024] C. The decoder module is a feedforward neural network used to obtain reconstructed samples and predict label values.

[0025] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0026] 1. The prediction method of this invention adopts an encoder-decoder framework based on variational Bayes, which improves the accuracy and computational efficiency of the prediction method for key parameters of chemical processes.

[0027] 2. This invention incorporates a positional encoding module, sparse attention, and cross-attention mechanisms into the latent variable generation stage of Variational Bayes, enabling the independent latent variables of Variational Bayes to be time-series-based. This enhances the reliability and accuracy of the decoder in generating predictive time-series variables.

[0028] 3. Based on the proposed neural network model, this invention derives an applicable objective function within a Bayesian framework. Attached Figure Description

[0029] Figure 1 Here is the overall algorithm flowchart for the SVAE model.

[0030] Figure 2 A simplified flow chart of the green ammonia synthesis process.

[0031] Figure 3 This embodiment describes the offline modeling and online prediction process for the SVAE model. Detailed Implementation

[0032] The present invention will be described below with reference to the accompanying drawings and predictions of key process parameters for the green hydrogen ammonia synthesis process.

[0033] The auxiliary variables in the green hydrogen ammonia synthesis process include: DC voltage of the electrolyzer, outlet pressure of the hydrogen purification system, mass flow rate of nitrogen inlet, circulating gas pressure of the synthesis tower, outlet temperature of the compressor, liquid level of the ammonia separator, real-time temperature of the catalyst bed, temperature difference monitoring value of the cooling water system, hydrogen / nitrogen molar ratio of the gas mixer, flow rate of the waste gas circulation loop, buffer pressure of the hydrogen storage tank, and current density of the electrolyzer.

[0034] The key process parameter is the real-time flow rate of green hydrogen gas at the inlet of the synthesis tower.

[0035] The sparse self-attention variational Bayesian neural network SVAE prediction model based on this technical solution is used to estimate the hydrogen flow rate in the reactor during the green hydrogen to ammonia synthesis process online. The specific steps are as follows:

[0036] S1: Historical industrial data on hydrogen production via ammonia electrolysis is acquired through sensors. The historical data includes input feature variables and output target variables, and the historical data is divided into training sets. Validation set and test set The validation dataset is used to train the hyperparameters of the SVAE, and the training data is used to train the SVAE.

[0037] S2: The training set obtained from S1 Standardization is performed to obtain standardized training features and standardized parameters. The standardized parameters obtained from the training set are used to standardize the validation set and test set. The SVAE soft measurement model is constructed based on sparse attention and variational Bayes.

[0038] S2-1, Given dataset , eigenvalues , tag value , This represents the total number of samples. Indicates the dimension of each data point. and Let x and y represent the dimensions, respectively. The dataset is processed using a sliding window mechanism. This provides information about the input sequence for the Probabilistic Sparse Attention Encoder (PSAE) module. The first PSAE module 1 and the second PSAE module 2 share the same input. The input to the third sparse attention encoding module 3 is... . The first table shows the lengths of the input sequences for sparse attention coding module 1 and sparse attention coding module 2. and These are the lengths of the label sequence and the prediction sequence of the third sparse attention coding module, respectively. For initial input To predict the input, ,in This provides convenience for generating sequences from model inputs.

[0039] S2-2. To capture the global dependencies between elements in the input sequence, SVAE uses a positional encoding module (i.e., a positional embedding module) in its sparse attention encoding module to explicitly introduce positional information into the sequence, enabling the model to understand the sequential relationships between elements. The positional encoding module consists of a convolutional activation operation followed by the addition of contextual positional information, which is encoded as follows:

[0040]

[0041] in and The corresponding row and column entries of the matrix are specified. .

[0042] S2-3. In order to handle the information interaction at different positions in the input sequence, SVAE adopts sparse self-attention to enhance the model's ability to capture long-range dependencies within the sequence, thereby enhancing its expressive power. This module is referred to as the multi-head sparse self-attention module.

[0043] The t-th input matrix after encoding by the position embedding module is denoted as... The input to the multi-head sparse self-attention module first passes through three linear layers to obtain... :

[0044]

[0045] in Represents a linear transformation function. This represents the relevant learning parameters.

[0046] Will The matrix is ​​divided into h separate heads, each head focusing on a unique portion of the input space. Then... That is, the first The matrix corresponding to each head is The same record and The The matrix corresponding to each head is , By employing a multi-head partitioning self-attention mechanism, information is processed in parallel across different representation subspaces, thereby enhancing the model's ability to capture diverse features of the input data.

[0047] S2-4. Introduce a sparse self-attention mechanism. Compared to the traditional attention mechanism, sparse attention requires first identifying important query vectors, and then only calculating the attention values ​​of these query vectors to optimize computational efficiency, enabling the model to better capture long-range dependencies and contextual information in the sequence.

[0048] 1) To identify specific query vectors The significance of the model is determined by the key matrix. A strategy of random sampling from row vectors to generate vectors containing A subset of samples This subset facilitates the calculation of the attention probability distribution, as shown below:

[0049]

[0050] in for No. A vector formed by rows for The The vector formed by the rows for The The vector formed by the rows , , .

[0051] 2) Define uniform distribution for calculation:

[0052]

[0053] 3) Introduce KL divergence calculation to measure and The similarity is calculated as follows:

[0054]

[0055] 4) Discarding the constant term Approximate assessment

[0056]

[0057]

[0058] After simplification, the formula for the sparsity criterion can be obtained:

[0059]

[0060] 5) Calculate each Find the sparsity of the highest sparsity. indivual Composition matrix . for The matrix after sparse transformation A matrix consisting of rows of vectors.

[0061] The ones found and Calculate the attention score and pass it through a softmax layer to obtain the attention weight matrix. Multiplying them together gives the attention score. :

[0062]

[0063] For the remaining use The mean is used instead. for The matrix formed after averaging.

[0064] Will and By splicing them together, we can obtain the first one. Complete attention score matrix under size At this point, the attention score for a single head has been obtained. The output results of each head are concatenated to obtain the final multi-head output. , For the first A sliding window, The multi-head output is obtained by concatenating the results of a single head. For the first The first sliding window, the... The attention score matrix for each single head is now complete. Calculation of multi-head sparse attention scores under a sliding window.

[0065] S2-5, Output after sparse attention computation The output is first obtained through a linear layer. Two more residual connections are needed to address the vanishing gradient and network degradation issues in deep neural networks. First... go through After and starting input The first residual connection output is obtained by performing residual connection on the common input. After that, The output can be obtained after two more one-dimensional convolutional layers. .

[0066] after and Summation and passing through By performing residual connections on the first layer, a second residual connection output can be obtained. . The first sparse attention encoding module 1 input data is at the... The output of each sliding window, since the second sparse attention coding module 2 has the same structure as the first sparse attention coding module 1, is denoted as... This is the output of the second sparse attention encoding module 2.

[0067] S2-6, the third sparse attention coding module 3 is structurally similar to the first sparse attention coding module 1 and the second sparse attention coding module 2. First, the position information is encoded using the same position embedding module 3, and then processed by the third multi-head sparse self-attention module 3. The embedding output undergoes similar operations to those in the first multi-head sparse self-attention module 1 of the first sparse attention encoding module 1. However, there are three main differences in the third multi-head sparse self-attention module 3: First, similar to the attention score calculation in the first sparse attention encoding module 1, in the third sparse attention encoding module 3, to avoid future information leakage, elements pre-correlated with the future are set to zero before the softmax operation to ensure that their softmax output is zero. The formula for calculating the attention score here is:

[0068]

[0069] Secondly, unlike the first sparse attention encoding module 1, the third sparse attention encoding module 3 uses cumulative summation instead of mean to fill subsequent elements. Finally, after calculating the sparse attention, the third sparse attention encoding module 3 only needs one additional residual connection to achieve the final output. .

[0070] S2-7. Establish cross-attention calculation to enhance the predictive power of the model. First, calculate the three outputs of the sparse attention encoding module. Perform a linear transformation to obtain The corresponding rules are as follows:

[0071] , , , These are the learning parameters for each linear layer in the cross-attention module. Similarly, the attention score for the cross-attention layer is calculated using the following formula:

[0072]

[0073] Similar to the first sparse attention encoding module 1, the calculated cross-attention scores are... After performing one residual connection, followed by two one-dimensional convolutions, and finally one more residual connection, the output of the Cross-attenion 1 layer can be calculated. .enter After the same operation as above, we obtain This completes the extraction of latent variables for the entire inference network.

[0074] S2-8. Introducing reparameterization techniques to facilitate network training using gradient-based optimization methods. The reparameterization formula is as follows:

[0075]

[0076] in , To sample a random vector from a standard normal distribution.

[0077] S2-9, Construct the decoder network. First, reconstruct the input data. Obtained through the reparameterization module Input it into the FNN to generate the reconstruction (RS). The program is formalized by the following equation:

[0078]

[0079] in This indicates that the FNN neural network performs the computation. This represents the reconstruction (RS) learning parameters in the decoder network.

[0080] Then predict the future The length of the label value is used to reduce the burden on the input data. The impact of uncertainty on prediction results, utilizing As input for generating predictions, the procedure is formalized by the following equation:

[0081]

[0082] in, , This represents the prediction (PR) learning parameters in the decoder network.

[0083] S3: Determining the final objective function applicable to SVAE can effectively iterate and update the model parameters to a local optimum, thereby achieving online estimation of quality variables and providing a strong foundation for subsequent process monitoring, optimization and control operations.

[0084] S3-1. Based on Bayes' theorem and its properties, the Karouch-Kuhn-Tucker condition, and the Lagrange multiplier method, the training loss function of the SVAE soft measurement model in this example can be expressed as follows:

[0085]

[0086] in Represents the true probability distribution. Used to represent approximate probability distributions. Symbols Indicates to Expectations This represents the KL divergence. It acts as a regularization coefficient, directly affecting the weight of the KL divergence term in the total loss function.

[0087] S3-2. Considering that the LVs at adjacent time steps are independent, the KL divergence can be expanded. The loss function for batch training of the SVAE model is as follows:

[0088]

[0089] Based on basic Monte Carlo inference, the loss function can be transformed into the following expression:

[0090]

[0091] S4: Employ a trained SVAE soft measurement model to perform real-time online prediction of the output target variable using the current input feature variables. Let the query sample be... The inputs to sparse attention encoding module 1 and sparse attention encoding module 3 are obtained through a sliding window. , .in , , The entire online prediction can then be expressed by the following formula:

[0092]

[0093] in For sparse attention coding module 1 network computation, It is its optimal parameters after training. For sparse attention coding module 3 network computation, It is its optimal parameters after training. Represents the calculation of feedforward neural network 1. It is its optimal parameters after training.

Claims

1. A method for predicting hydrogen load in green ammonia production based on sparse attention variational Bayes, characterized in that, In the green hydrogen ammonia synthesis process, a sequence variational autoencoder (SVAE) is used as a soft measurement model to predict the green hydrogen flow rate in the reactor online using the input feature variables in the green hydrogen ammonia synthesis process. The steps of the prediction method include: S1: Collect historical data of the electrolytic hydrogen production industry and divide the historical data into training set, validation set and test set; the historical data includes input feature variables and output target variables; S2: Standardize the training set obtained in S1 to obtain standardized training features and standardized parameters; use the standardized parameters to standardize the validation set and test set; And a soft measurement training model for SVAE is constructed based on sparse attention and variational Bayes; S3: Determine the final objective function applicable to SVAE and effectively iterate and update the model parameters to a local optimum; S4: Employ a trained SVAE soft measurement model to perform real-time online prediction of the output target variable using the current input feature variables; The SVAE is based on the Variational Autoencoder (VAE), and explicitly introduces the positional information of the sequence using a position embedding module in the sparse attention coding module; the network structure of the SVAE includes an encoder module, a reparameterization module, and a decoder module, wherein: The encoder module includes a sparse attention encoding module and two cross-attention modules, which are used to handle the information interaction at different positions in the input sequence; There are three sparse attention coding modules, of which: the first and second sparse attention coding modules are the same, and their structures include the same first and second position embedding modules, as well as the same first and second multi-head sparse self-attention modules; the third sparse attention coding module is the same, and its structure includes the third position embedding module and the third multi-head sparse self-attention module; the first, second and third position embedding modules are the same. There are two identical cross-attention modules. Each cross-attention module includes a multi-head attention computation layer, a convolutional layer, a residual connection layer, and an activation layer. The reparameterization module makes the sampling process differentiable, thus enabling efficient gradient updates through backpropagation; The decoder module includes two feedforward neural networks used to obtain reconstructed samples and predict label values.

2. The prediction method according to claim 1, characterized in that, In S1, the training set samples are obtained by acquiring historical industrial data on hydrogen production via green ammonia electrolysis from sensors. ; Historical data includes input feature variables and output target variables, and the historical data is divided into training sets. Validation set and test set .

3. The prediction method according to claim 2, characterized in that, The data processing in S2 includes: S2-1, Given the training set , eigenvalues , tag value , represents the total number of samples, and i represents the sample number; Indicates the dimension of each data point. and Let x and y represent the dimensions, respectively. Through the sliding window mechanism, dataset O provides information for the input sequence of the sparse attention encoding PSAE module; The first and second sparse attention encoding modules share the same input. The input to the third sparse attention encoding module is ; As the initial input, For predicting input; This indicates the length of the input sequence for the first and second sparse attention coding modules; and These represent the lengths of the label sequence and the prediction sequence of the third sparse attention coding module, respectively. ,in ; S2-2, For the location embedding module The location embedding module first performs a convolutional activation operation on the input data; then, it adds contextual location information using a location information encoder; the location information encoding is defined as: ， in and The corresponding row and column entries of the input matrix are specified respectively. As an index variable, it is used to represent the odd and even columns of the encoding matrix. , It is the embedding dimension, that is, the length of the input feature vector; S2-3, in the first multi-head sparse self-attention module (a) The t-th input matrix after encoding by the first position embedding module is denoted as The input first passes through three linear layers, resulting in three weight matrices. , and , respectively represented as: , and ,in, Represents a linear transformation function. Indicates the relevant learning parameters; Will Divide the input space into h separate headers, each focusing on a unique portion of the input space; then denote... , and They are the first The matrix corresponding to each head is , and ; (ii) Sparse attention computation 1) To identify specific query vectors The significance of the key matrix A strategy of random sampling from row vectors to generate vectors containing A subset of samples This subset is used to calculate the attention probability distribution: ， in for No. A vector formed by rows for The vector formed by the j-th row, for The A vector formed by rows, , , ; 2) Define uniform distribution for calculation: ， 3) Use KL divergence calculation to measure and The similarity is calculated as follows: ， This is the transpose operation for a matrix; 4) Discarding the constant term Approximate assessment ， ， After simplification, the formula for the sparsity criterion can be obtained: ， In the formula, d' represents the dimension assigned to each attention head; 5) Calculate each Find the sparsity of the highest sparsity. indivual Composition matrix for The matrix after sparse transformation A matrix composed of row vectors; The ones found and Calculate the attention score and pass it through a softmax layer to obtain the attention weight matrix; Multiply to get the attention score : ， Use for the remaining subsequent elements The mean was used instead; for The matrix formed after averaging; Will and By splicing them together, we can obtain the first one. Complete attention score matrix under size ; Will The output results of each single head are concatenated to obtain the final multi-head output. , For the first The first sliding window The multi-head output is obtained by concatenating the results of a single head; For the first The first sliding window A single-head attention score matrix; This completes the first... Calculation of multi-head sparse attention scores under a sliding window; (iii) Output after sparse attention computation The output is obtained by passing through a linear layer first. Then, after two more residual joins: first, go through After operation and input The first residual connection output is obtained by performing residual connection on the common input. ;Then, The output is obtained after passing through two more one-dimensional convolutional layers. ; after, and Summation, then through The second residual connection output is obtained by performing residual connection on the layer. ; That is, the input data of the first sparse attention encoding module is at the th Output of a sliding window; The input data for the second sparse attention coding module is obtained using the same method at the [number]th [year]. Output of a sliding window ; S2-4. In the third sparse attention coding module... enter The position information is first encoded by the third position embedding module; then it is processed by the third multi-head sparse self-attention module. The difference between the third multi-head sparse self-attention module and the first multi-head sparse self-attention module is as follows: (1) In step 5) of the sparse attention calculation in step (II), Attention Score The calculation formula is changed to: ， (2) Use cumulative summation to fill subsequent elements; ， in This indicates that the matrix is ​​summed along its column dimensions, and is used for filling. After OK; (3) In step (iii) after calculating the sparse attention, the third sparse attention encoding module uses an additional residual connection to implement the final output. ; S2-5, First, regarding Perform a linear transformation to obtain The corresponding rules are as follows: ,in, , , These are the learning parameters for each linear layer in the cross-attention module; Then, the attention score of the first cross-attention module is calculated using the following formula: ； Attention score After performing a residual connection, the output is obtained through two one-dimensional convolutions; finally, it is obtained through another residual connection. ; right Perform a linear transformation to obtain The corresponding rules are as follows: ,in, , , These are the learning parameters for each linear layer in the cross-attention module; Then, the attention score of the first cross-attention module is calculated using the following formula: ； Attention score After performing a residual connection, the output is obtained through two one-dimensional convolutions; finally, it is obtained through another residual connection. ; This completes the extraction of latent variables; S2-6. In the reparameterization module, the reparameterization formula is as follows: ， in , To sample a random vector from a standard normal distribution; S2-7. In the decoder First, reconstruct the input data. :Will Input to a second feedforward neural network to generate a reconstruction The formula is: ， in This indicates that the feedforward neural network is being computed. Represents the reconstruction learning parameters of the decoder; Then, a first feedforward neural network is used to... As input, predict the future The length of the tag value is calculated using the following formula: ， in, , This represents the prediction learning parameters of the decoder.

4. The prediction method according to claim 2, characterized in that, In S3, the final objective function applicable to SVAE is determined: The overall loss function of SVAE is: ， in, This is the prediction error term; To counteract the loss, Input the refactoring item; The normalization coefficient directly affects the weight of the KL difference term in the overall loss function, thereby encouraging the learning of separation and meaningful representations by imposing a strict penalty on the KL difference. S3-1. Based on Bayes' theorem and its properties, the Karouch-Kuhn-Tucker condition and the Lagrange multiplier method, the training loss function of the soft measurement model is expressed as follows: ， in Represents the true probability distribution. Used to represent an approximate probability distribution, symbol Indicates to Expectations Indicates KL divergence; It acts as a regularization coefficient, directly affecting the weight of the KL divergence term in the total loss function; S3-2, The loss function for batch training of the soft measurement model is as follows: ， Based on basic Monte Carlo inference, the loss function is transformed into the following expression: 。 5. The prediction method according to claim 2, characterized in that, In S4, a trained soft measurement model is used to perform real-time online prediction of the output target variable using the current input feature variables. Let the query sample be The inputs to the first and third sparse attention coding modules are obtained through a sliding window. , ,in , , The entire online prediction is then expressed by the following formula: ,in: For the first sparse attention coding module network computation, These are its optimal parameters after training; For the network computation of the third sparse attention encoding module, These are its optimal parameters after training; Calculated for the first feedforward neural network, It is its optimal parameters after training.

Images