Construction method of complex multivariate system network prediction model based on informer architecture

Abstract

A method for constructing a complex multivariate system network prediction model based on an Informer architecture, obtaining complex industrial process production data; performing ElasticNet feature selection on the collected time series data to remove redundant variables; performing VMD decomposition and noise reduction on the sequence data obtained after feature selection; window setting and sequence division; performing embedding operation on the sequence data, i.e., performing preliminary feature extraction; constructing an encoder to encode S E ; constructing a decoder to decode S D ; using a 1-layer linear layer to map the output sequence F D of the decoder to obtain a prediction result; training the constructed model using training set data; and using a retraining strategy to evaluate the model for the test set, thereby preventing information leakage. This method solves the shortcomings of large prediction error and poor adaptability of traditional networks for systems containing input noise and redundant variables, and can accurately predict key variables of complex multivariate processes.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically a method for constructing a network prediction model for complex multivariable systems based on the Informer architecture. Background Technology

[0002] When modeling and predicting complex industrial processes, intermediate inputs and state signals that have a critical impact on the output response need to be obtained through measurement. Due to the difficulty in performing explicit mechanistic analysis on complex multivariate systems, it is impossible to obtain the specific impact of each measurable variable on the key output variable. Training the network using all available signals would not only increase the training and testing burden due to variable redundancy, but also amplify prediction errors due to irrelevant input variables. Therefore, feature extraction and analysis techniques are needed to select principal input variables that express the key output variables. When selecting principal input variables for training, feature extraction techniques are required to obtain the main feature information of the time series to facilitate network training. When measuring signals using instruments, interference from measurement noise is unavoidable. Using noisy input signals for network training not only makes it difficult to achieve fast and effective convergence, but also results in a large prediction error in practical use. To improve the robustness and prediction accuracy of the network, the input signal needs to be processed appropriately. Simply using filtering and smoothing techniques to process the input signal will distort it, leading to large errors and poor adaptability in the trained network. The key difference between measurement noise and useful system signals lies in the fact that measurement noise often follows a Gaussian distribution and exhibits significant high-frequency characteristics, while the useful signals from intermediate inputs and process states are low-frequency signals with slowly changing characteristics. Therefore, signal decomposition techniques can be used to decompose the acquired input signal into a high-frequency signal containing measurement noise and a low-frequency signal containing only useful information. Since the high-frequency signal obtained from the decomposition may also contain useful process signals, it cannot be simply deleted. First, a lower weight can be assigned to the high-frequency signal obtained from the decomposition to reduce the impact of noise without completely discarding the effective signal. Then, signal recombination techniques can be used to recombine the low-weighted high-frequency signal and the low-frequency useful signal to obtain new input data with a lower noise level. This recombined signal is expected to improve the network's prediction accuracy and environmental adaptability. Therefore, feature extraction and signal analysis processing of complex industrial process time-series data hold promise for establishing intelligent prediction networks with fast training speeds, high accuracy, and strong adaptability.

[0003] In recent years, scientists and researchers have developed many signal reconstruction and variable selection techniques. Time series networks based on regression neural networks (RNNs), long short-term memory networks (LSTMs), and gated recurrent units (GRUs) can only effectively predict key variables in simple industrial processes. They cannot extract complex correlations within time series data or effectively handle redundant variables. Existing Transformer and Informer network architectures have laid a good foundation for the design and development of complex networks; however, intelligent networks designed based on them cannot well adapt to industrial processes with input noise and complex variable relationships, thus limiting the practical application value of these network models in the field of intelligent prediction.

[0004] Therefore, there is an urgent need to provide an intelligent prediction network that can adapt to different working scenarios and data distribution characteristics, in order to solve the problem that existing networks have large prediction errors for key outputs of complex industrial processes containing input noise and redundant variables. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method for constructing a network prediction model for complex multivariable systems based on the Informer architecture. This method can solve the problem of large prediction errors of existing networks for key outputs of complex industrial processes containing input noise and redundant variables.

[0006] To achieve the above objectives, this invention provides a method for constructing a network prediction model for complex multivariate systems based on the Informer architecture, comprising the following steps:

[0007] Step 1: Acquire production data for complex industrial processes;

[0008] Various sensors are used to measure and collect various parameters in complex production processes to obtain production data for complex production processes;

[0009] Step 2: Perform ElasticNet feature selection on the collected time series data to remove redundant variables;

[0010] S21: Construct a linear regression model for time series data using formula (1);

[0011]

[0012] In the formula, X(i) and Let represent the input features and predicted output of the i-th sample; ω is the coefficient vector matrix.

[0013] S22: Define the objective function in formula (2) with L1 regularization and L2 regularization terms, and minimize the objective function to obtain the optimal parameters for fitting the time series;

[0014]

[0015] In the formula, Y(i) is the true output of the i-th sample; α is the regularization strength coefficient; ρ is the L1 regularization strength; α and ρ are used together to control the L1 and L2 regularization strengths;

[0016] Step 3: Perform VMD decomposition and noise reduction on the sequence data obtained after feature selection;

[0017] S31: The sequence data samples after feature selection have N dimensions, where the first m dimensions are the input features, represented as X′={x1,x2,...,x m}, where the Nm dimensions are the system output, represented as Y = {y m+1 ,y m+2 ,...,y N VMD for each dimension of feature x i Mode decomposition is performed, where i = 1, ..., m, and the number of mode components obtained from each eigenvalue decomposition is set to K = 2. The specific process is as follows:

[0018] A1: VMD technology iteratively processes x i The K modal component signals are solved for each mode of (t); before iteration, wavelet transform or Fourier transform is used to initialize the K modal component signals to obtain each modal component signal as v. k (t), where k = 1, ..., K;

[0019] A2: Regarding the modal component v k (t) is analyzed and negative frequencies are eliminated. The specific process is as follows:

[0020] A21: According to formula (3), v k (t) Perform Hilbert transform;

[0021]

[0022] In the formula, * represents the convolution operation; Where sgn(ω) is the sign function,

[0023] A22: Convert formula (3) to the frequency domain to obtain formula (4);

[0024]

[0025] In the formula, F(ω) is vk Fourier transform of (t); for The Fourier transform of h(t); h(ω) is the Fourier transform of h(t);

[0026] A23: Transform formula (4) to obtain formula (5);

[0027]

[0028] In the formula, j satisfies j 2 =-1;

[0029] A24: Define v according to formula (6) k The analytic signal z(t) of (t);

[0030]

[0031] A25: Combining formula (3), and based on the commutative and distributive laws of convolution, formula (6) is transformed to obtain formula (7);

[0032]

[0033] In the formula, δ(t) is the Dirac function;

[0034] A3: Let the modal components v k The center frequency of (t) is ω k According to formula (8), the frequency of formula (7) is tuned to v. k The spectrum of (t) is modulated to the corresponding baseband;

[0035]

[0036] A4: Using Gaussian smoothing, calculate the gradient and corresponding L2 norm of formula (8), and take the square root to obtain the modal component v in formula (9). k Bandwidth estimation of (t);

[0037]

[0038] A5: VMD establishes the constrained variational optimization problem in formula (10) based on the principle of minimizing the sum of bandwidth estimates of each modal component, while ensuring that the sum of each modal component is still the original signal.

[0039]

[0040] In the formula, v k (k = 1, ..., K) represents each modal component; ω k (k = 1, ..., K) represents the center frequency of each modal component;

[0041] A6: Introducing second-order multiplication factors and Lagrange multipliers, the constrained variational problem shown in formula (10) is transformed into an unconstrained variational problem using the augmented Lagrange function in formula (11);

[0042]

[0043] A7: The problem in the time domain shown in formula (11) is transformed into the frequency domain for solution. At the same time, the alternating direction multiplier method is used to address the modal components respectively. Center frequency of modal components and Lagrange multipliers Perform iterative calculations. and The iterative formulas are formula (12), formula (13) and formula (14), respectively;

[0044]

[0045]

[0046] In the formula, τ is the noise tolerance; n is the number of iterations; and These are the original signals x i (t), modal component v k Fourier transform of (t) and Lagrange multiplier λ(t);

[0047] A8: Repeat formulas (12) to (14) for calculation until the modal component v k (t) satisfies the iterative constraint conditions in formula (15);

[0048]

[0049] In the formula, ε is a given error constant;

[0050] S32: Obtain the characteristic signals x after noise reduction according to formula (16). i ′(t), where i = 1,...,m;

[0051] x i ′(t)=ηv i1 (t)+v i2 (t)(16);

[0052] In the formula, v i1 (t) and v i2 (t) represents the original feature signal x i (t) The high-frequency and low-frequency modes obtained after VMD decomposition with K=2;

[0053] S33: The new input features after VMD denoising are represented as X″={x1″,x2″,...,x′ m The output is Y;

[0054] Step 4: Window Setup and Sequence Partitioning; A sliding window approach is used, with single-step prediction, meaning only one data point is predicted at a time based on historical data; the sliding window length is set to L. E That is, when making a prediction, take the length L in front of it. E Historical data, denoted as S. E Simultaneously, the historical data sequence S E L in the middle Token Each data point is merged with one prediction point, with a length of L. D Let the sequence be S. D ;

[0055] Step 5: Perform embedding operations on the sequence data, i.e., perform preliminary feature extraction to capture local and positional features;

[0056] S51: Set up an embedding layer, including one convolutional layer and one positional embedding layer, and arrange them in parallel; the convolutional layer has a kernel size of 3, a stride of 1, and padding of 1 on both sides; the positional embedding layer uses the trigonometric functions in formula (17) and formula (18) to capture the positional features in the sequence data, so as to convert the positional information of the sequence data into a feature matrix; after the sequence is processed by the convolutional layer and the positional embedding layer in the embedding layer, the two are added together;

[0057]

[0058] In the formula, pos is the position of the point in the sequence; i starts from 0; 2i and 2i+1 represent the even-dimensional and odd-dimensional features of the point, respectively; d_model is the hyperparameter of the model;

[0059] S52: Embedding operation on sequence data; for S E First, a single LSTM network is used to capture long-term dependencies, and then the feature is extracted in the embedding layer; for S D It directly enters the embedding layer for feature extraction;

[0060] Step Six: Build the encoder for S E Encode;

[0061] S61: Three coding layers are stacked, and the coding layer structure is as follows:

[0062] B1: Multi-head ProbSparse Attention technology is used to extract correlation features from the sequence;

[0063] B11: The Q-matrix and K-matrix of the attention mechanism are respectively derived from S E The V matrix is ​​obtained by performing causal convolution, which involves first adding two zero-padding operations at the beginning of the sequence, followed by a convolution operation with a kernel size of 3, and setting the number of kernels to d_model, meaning the feature dimension is mapped to d_model. The V matrix is ​​derived from S. E A linear mapping is performed, and the dimension of the mapping is also set to d_model. This leads to a result where all dimensions are L. E The Q, K, and V matrices of ×d_model;

[0064] B12: Set the number of heads for the attention mechanism to n_head, that is, divide the feature dimensions of Q, K and V into n_head parts respectively, and the feature dimension of each head is d = d_model / n_head, with a length of l = L. E ;

[0065] B13: For the i-th data point in a feature sequence of length l, let its q, k, and v values ​​be denoted as qi, ki, and vi, respectively. i k i and v i For q in the time series i With k j The similarity is calculated according to formula (19);

[0066]

[0067] B14: For k j Calculate using formula (19) to obtain q. i The similarity matrix Σ i , Σ i =[S1,S2,...S l ], where j = 1, 2, ..., l, and then the Softmax function is used to apply the Σ i Normalization is performed to convert it into a probability distribution, resulting in q. i The probability distribution p(k) j |q i As shown in formula (20);

[0068]

[0069] B15: Note Calculate the distribution p(k) j |q i ) and uniform distribution q(k j |q i) = 1 / L K The Kullback-Leibler divergence, used to represent the difference from a uniform distribution, is a measure of q. i The volatility of L, where L K Let l be the number of k values; calculate p(k) according to formula (21). j |q i ) and q(k j |q i The Kullback-Leibler divergence of ).

[0070]

[0071] B16: Discard the constant term lnL K According to formula (22), q is defined i sparsity M(q) i ,K);

[0072]

[0073] B17: Utilization replace Rewrite q using formula (23) i sparsity;

[0074]

[0075] B18: Calculate q for each term i corresponding Scores, where i = 1, 2, ..., l, retain the 30 highest-scoring q values ​​U = 30. i The remaining positions are replaced by the mean, resulting in a new Q matrix, denoted as Q′;

[0076] B19: For the i-th term in the sequence, calculate q according to formula (24). i ′ and k j Similarity S j ′;

[0077]

[0078] B110: For k j The similarity between the i-th item in the sequence and each item is calculated sequentially using formula (24), where j = 1, 2, ..., l, and denoted as the similarity vector Σ. i = [S1′,S2′,...S l Then, the attention weight matrix of the i-th term is calculated using the Softmax function, as shown in formula (25);

[0079]

[0080] B111: α i Multiplying by V yields the attention score A for the i-th term. i As shown in formula (26);

[0081] A i =α i V(26);

[0082] B112: For each item in the sequence, repeat steps B19 to B111 to obtain the attention score for each item, thus obtaining the attention score matrix Λ = (A1, A2, ... A l );

[0083] B2: Activate the result of the attention mechanism using the Morlet mother wavelet basis function in formula (27) to obtain its first derivative. As shown in formula (28);

[0084]

[0085] In the formula, t is time; C and m are constants;

[0086] B3: Decompose the feature sequence, i.e., F = [x1, x2, ..., x L The sequence is divided into three parts, and each part is extracted using a single-layer LSTM network to obtain F1′, F2′ and F3′. The three feature sequences are then merged and denoted as F′.

[0087] B4: Use a convolution kernel of size 3 with a stride of 1, and perform zero padding at the beginning and end to aggregate information from the feature matrix F′; then perform max pooling with a kernel of size 3, a stride of 2, and zero padding of 1, reducing the length of the feature vector to half its original length to improve computational efficiency, resulting in the feature matrix F′. Conv ;

[0088] S62: Same as B1, for F Conv Multi-head ProbSparse Attention is performed to obtain contextual information in the feature sequence, and then the wavelet function shown in formula (27) is used to activate it;

[0089] S63: Obtain the encoder's output feature sequence F E ;

[0090] Step 7: Construct the decoder for S D Decode;

[0091] S71: Use Multi-head ProbSparse Attention with a mask on the feature sequence S D The correlation model is performed to learn the correlation features in the sequence, and then the wavelet function shown in formula (27) is used to activate it to obtain the feature sequence F. Att ;

[0092] S72: Feature extraction is performed using Multi-head Attention, combined with F Att With the encoder output F E The inputs are fed into the Multi-head Attention layer, and a new feature sequence F is obtained according to formulas (24) to (26). Comb ;

[0093] S73: Uses a single convolutional layer to aggregate features from the attention mechanism output;

[0094] S74: Obtain the output feature sequence F of the decoder. D ;

[0095] Step 8: Using a single linear layer, convert the decoder's output sequence F... D Mapping yields the prediction results;

[0096] Step 9: Train the constructed model using the training set data. For the test set, evaluate the model using a retraining strategy to prevent information leakage.

[0097] S91: Divide the time series data after feature selection into training and test sets in a 5:3 ratio;

[0098] S92: Perform VMD denoising on the training set and input it into the model to train the model parameters;

[0099] S93: Set model parameters, set sliding window size L E This also sets the input sequence length L of the decoder, which is the length of the encoder's input sequence. D Set the length L of the overlapping sequence. Token Set the model hyperparameter d_model;

[0100] S94: Employ a small-batch strategy for batch training; set the batch size (batch_size) to divide the training set data into multiple batches for model training;

[0101] S95: Construct the loss function using formula (29);

[0102]

[0103] In the formula, n is the number of samples in the batch; y i Output the true value for the sample; These are the model's predicted values.

[0104] S96: The loss function is optimized using the Adam optimization algorithm, and a decaying learning rate strategy is adopted;

[0105] S97: Save the network weight parameters to obtain a prediction model for complex multivariate systems;

[0106] S98: Test the test set using a retraining strategy, as follows:

[0107] C1: Set the retraining interval to L. retrain .

[0108] C2: For the test set after feature selection, segment the first sample sequence. The VMD technique is used to decompose the input features of S1 in each dimension and reduce high-frequency noise, and then recombined to obtain the denoised samples.

[0109] C3: Load the model and make predictions for S1′.

[0110] C4: Add S1′ to the training set and re-perform VMD denoising on the new training set.

[0111] C5: For the new training set after denoising, retrain the model.

[0112] C6: For each sample in the test set Repeat the process from C2 to C5.

[0113] As a preferred option, in step nine, S91, the ratio is set to 5:3.

[0114] As a preferred embodiment, in step nine, S93, the sliding window size L is set. E The length L of the decoder's input sequence is 256. D The length of the overlapping sequence is 33, and the length of the overlapping sequence is L. Token Set the model hyperparameter d_model to 64, with a value of 32.

[0115] Furthermore, in order to effectively improve the training effect, during the model training process in step S94 of step nine, the batch size is set to 128 and the number of rounds is set to 50.

[0116] Furthermore, in order to effectively ensure the optimization effect, in step nine, S96, when the Adam optimization algorithm is used to optimize the loss function, the initial learning rate is set to 0.002.

[0117] Furthermore, to facilitate subsequent calculations, in step nine, S96, when employing the decay learning rate strategy, the decay step size is set to 15 and the decay factor is set to 0.1.

[0118] This invention provides a network prediction model for complex multivariable systems based on the Informer architecture and integrating VMD denoising and ElasticNet variable selection. For complex input-output nonlinear industrial processes, the collected production data often contains random noise. Not only is feature selection of the input signal necessary, but directly using noisy data for modeling can lead to significant prediction errors. Variational Mode Decomposition (VMD) is used to decompose the input features of the time series, dividing the input signal features into noisy high-frequency modes and low-frequency modes containing only useful signals. Under the Informer network architecture, for complex multivariable systems, this invention provides a robust prediction network that integrates ElasticNet variable selection and VMD denoising. Since the high-frequency modes obtained from VMD decomposition contain a large amount of noise and little effective information, this invention assigns certain low weights to the high-frequency modes and recombines them with the low-frequency noise-free modes according to system characteristics and signal distribution features, thereby reducing the noise of the original input features. Specifically, the original signal v is first... k (t) Performing the Hilbert transform shifts the phase of the positive and negative frequencies of the original signal by -π / 2 and +π / 2 respectively, and then modifies the imaginary units j and v. k The result of the (t) transformation Multiplying them shifts the phase of both the positive and negative frequencies of the original signal by +π / 2, and then multiplying the product... With the original signal v k Adding (t) together can eliminate the original signal v. k By finding the negative frequency of (t), its single-sided spectrum is obtained for analytical processing. This reduces the occupied bandwidth of the signal without losing useful information, greatly facilitating subsequent calculations. Since VMD applies this to each dimension of feature x... iIn modal decomposition, each feature is decomposed into K = 2 modal components. Therefore, each feature is decomposed into high-frequency and low-frequency modes. The high-frequency modes contain a large amount of high-frequency noise, but also contain some effective high-frequency signals. Therefore, instead of directly removing the high-frequency modes, the high-frequency modes are added with a certain weight η∈[0,1) to the low-frequency modes containing only effective signals. This allows for the use of signal decomposition and reconstruction techniques to assign small weights to the noise signal components, achieving a reduction in high-frequency noise with minimal loss of effective high-frequency signals. Since time-series data from complex systems contain many input features, using ElasticNet technology to select principal components in the new time-series data after VMD denoising can effectively eliminate redundant variables, ensuring that the model provided in this invention has a low computational burden and high modeling efficiency during network training and prediction, thus improving network efficiency. Specifically, ElasticNet obtains the optimal coefficient vector ω corresponding to each sample by minimizing the objective function. iThe system aims to fit time series data. L1 regularization favors the sparsity of feature coefficients, tending to reduce or set the coefficients of each feature to 0. L2 regularization, on the other hand, tends to retain multiple features. Therefore, during optimization, the coefficients of the optimal features can be obtained, enabling feature selection for time series data. The collected dataset is divided into training and testing sets. The training data is then divided into multiple batches for model training. A sliding window approach and single-step prediction are used in each batch to effectively ensure good prediction accuracy. Based on the Informer architecture, an embedding layer, an encoder layer, a decoder layer, and a fully connected mapping layer are constructed. The encoder layer consists of stacked multi-layer causal convolutions, a multi-head probabilistic sparse attention mechanism, wavelet activation functions, sequence decomposition operations, and one-dimensional convolutional pooling operations. The decoder layer consists of a masked multi-head probabilistic sparse attention mechanism, wavelet activation functions, a multi-head attention mechanism, and one-dimensional convolutional operations. The training data is divided into two parts according to a certain ratio, and features are extracted through the Encoder and Decoder layers respectively. For the input sequence of the Encoder, a single LSTM network is used for preliminary feature extraction, which can extract the correlations and long-term dependencies in the time series. The features extracted by the Encoder and Decoder layers are then further processed through an Embedding layer to extract local and positional features. For the multi-head attention mechanism in the Decoder, the input is the result of mapping the Decoder's intermediate output and the Encoder's output through two layers of FNN (Feedforward Neural Network). This operation achieves "distillation" of the feature sequence, that is, the attention mechanism can comprehensively extract the contextual information in the sequence. Simultaneously, the sequence length is reduced through convolutional pooling operations, resulting in a gradual decrease in computational burden. The Decoder's output is mapped through a fully connected layer to obtain the model's prediction result. The performance of the invented network model is tested and evaluated using actual chemical process data. Test results show that the model provided by this invention has high prediction accuracy, robustness, and nonlinear fitting ability, and it exhibits high prediction accuracy and adaptability for complex multivariate systems.

[0119] This invention effectively solves the shortcomings of traditional networks in predicting large errors and poor adaptability for systems with input noise and redundant variables. It can accurately predict key variables in complex multivariate processes, providing an accurate model foundation for subsequent model-based production process controller design and operation monitoring. Attached Figure Description

[0120] Figure 1 This is a flowchart of the present invention;

[0121] Figure 2 This is a bar chart of the determination coefficients of the fitting results of the model proposed in this invention under different weights and high-frequency noise.

[0122] Figure 3 This is a training loss curve for each retraining of the test set in this invention;

[0123] Figure 4 This is a graph showing the relative error of the test set fitting results for different models in this invention;

[0124] Figure 5 This is a graph showing the test set fitting results of different models in this invention. Detailed Implementation

[0125] The invention will now be further described with reference to the accompanying drawings.

[0126] like Figure 1 As shown, this invention provides a method for constructing a network prediction model for complex multivariable systems based on the Informer architecture, comprising the following steps:

[0127] Step 1: Acquire production data for complex industrial processes;

[0128] Various sensors are used to measure and collect various parameters in complex production processes to obtain production data for complex production processes;

[0129] Step 2: Perform ElasticNet feature selection on the collected time series data to remove redundant variables;

[0130] S21: Construct a linear regression model for time series data using formula (1);

[0131]

[0132] In the formula, X(i) and Let represent the input features and predicted output of the i-th sample; ω is the coefficient vector matrix.

[0133] S22: Define the objective function in formula (2) with L1 regularization and L2 regularization terms, and minimize the objective function to obtain the optimal parameters for fitting the time series;

[0134]

[0135] In the formula, Y(i) is the true output of the i-th sample; α is the regularization strength coefficient; ρ is the L1 regularization strength; α and ρ are used together to control the L1 and L2 regularization strengths;

[0136] Step 3: Perform VMD decomposition and noise reduction on the sequence data obtained after feature selection;

[0137] S31: The sequence data samples after feature selection have N dimensions, where the first m dimensions are the input features, represented as X′={x1,x2,...,x m}, where the Nm dimensions are the system output, represented as Y = {y m+1 ,y m+2 ,...,y N VMD for each dimension of feature x i Mode decomposition is performed, where i = 1, ..., m, and the number of mode components obtained from each eigenvalue decomposition is set to K = 2. The specific process is as follows:

[0138] A1: VMD technology iteratively processes x i The K modal component signals are solved for each mode of (t); before iteration, wavelet transform or Fourier transform is used to initialize the K modal component signals to obtain each modal component signal as v. k (t), where k = 1, ..., K;

[0139] A2: Regarding the modal component v k (t) is analyzed and negative frequencies are eliminated. The specific process is as follows:

[0140] A21: According to formula (3), v k (t) Perform Hilbert transform;

[0141]

[0142] In the formula, * represents the convolution operation; Where sgn(ω) is the sign function,

[0143] A22: Convert formula (3) to the frequency domain to obtain formula (4);

[0144]

[0145] In the formula, F(ω) is v k Fourier transform of (t); for The Fourier transform of h(t); h(ω) is the Fourier transform of h(t);

[0146] A23: Transform formula (4) to obtain formula (5);

[0147]

[0148] In the formula, j satisfies j 2 =-1;

[0149] A24: Define v according to formula (6) kThe analytic signal z(t) of (t);

[0150]

[0151] A25: Combining formula (3), and based on the commutative and distributive laws of convolution, formula (6) is transformed to obtain formula (7);

[0152]

[0153] In the formula, δ(t) is the Dirac function;

[0154] A3: Let the modal components v k The center frequency of (t) is ω k According to formula (8), the frequency of formula (7) is tuned to v. k The spectrum of (t) is modulated to the corresponding baseband;

[0155]

[0156] A4: Using Gaussian smoothing, calculate the gradient and corresponding L2 norm of formula (8), and take the square root to obtain the modal component v in formula (9). k Bandwidth estimation of (t);

[0157]

[0158] A5: VMD establishes the constrained variational optimization problem in formula (10) based on the principle of minimizing the sum of bandwidth estimates of each modal component, while ensuring that the sum of each modal component is still the original signal.

[0159]

[0160] In the formula, v k (k = 1, ..., K) represents each modal component; ω k (k = 1, ..., K) represents the center frequency of each modal component;

[0161] A6: Introducing second-order multiplication factors and Lagrange multipliers, the constrained variational problem shown in formula (10) is transformed into an unconstrained variational problem using the augmented Lagrange function in formula (11);

[0162]

[0163] A7: The problem in the time domain shown in formula (11) is transformed into the frequency domain for solution. At the same time, the alternating direction multiplier method is used to address the modal components respectively. Center frequency of modal components and Lagrange multipliers Perform iterative calculations. and The iterative formulas are formula (12), formula (13) and formula (14), respectively;

[0164]

[0165] In the formula, τ is the noise tolerance; n is the number of iterations; and These are the original signals x i (t), modal component v k Fourier transform of (t) and Lagrange multiplier λ(t);

[0166] A8: Repeat formulas (12) to (14) for calculation until the modal component v k (t) satisfies the iterative constraint conditions in formula (15);

[0167]

[0168] In the formula, ε is a given error constant;

[0169] S32: Obtain the characteristic signals x after noise reduction according to formula (16). i ′(t), where i = 1, ..., m;

[0170] x i ′(t)=ηv i1 (t)+v i2 (t)(16);

[0171] In the formula, v i1 (t) and v i2 (t) represents the original feature signal x i (t) The high-frequency and low-frequency modes obtained after VMD decomposition with K=2;

[0172] S33: The new input features after VMD denoising are represented as X″={x1″,x2″,...,x′ m The output is Y;

[0173] Step 4: Window Setup and Sequence Partitioning; A sliding window approach is used, with single-step prediction, meaning only one data point is predicted at a time based on historical data; the sliding window length is set to L. E That is, when making a prediction, take the length L in front of it. E Historical data, denoted as S. E Simultaneously, the historical data sequence S E L in the middle Token Each data point is merged with one prediction point, with a length of L. D Let the sequence be S.D ;

[0174] Step 5: Perform embedding operations on the sequence data, i.e., perform preliminary feature extraction to capture local and positional features;

[0175] S51: Set up an embedding layer, including one convolutional layer and one positional embedding layer, and arrange them in parallel; the convolutional layer has a kernel size of 3, a stride of 1, and padding of 1 on both sides; the positional embedding layer uses the trigonometric functions in formula (17) and formula (18) to capture the positional features in the sequence data, so as to convert the positional information of the sequence data into a feature matrix; after the sequence is processed by the convolutional layer and the positional embedding layer in the embedding layer, the two are added together;

[0176]

[0177] In the formula, pos is the position of the point in the sequence; i starts from 0; 2i and 2i+1 represent the even-dimensional and odd-dimensional features of the point, respectively; d_model is the hyperparameter of the model;

[0178] S52: Embedding operation on sequence data; for S E First, a single LSTM network is used to capture long-term dependencies, and then the feature is extracted in the embedding layer; for S D It directly enters the embedding layer for feature extraction;

[0179] Step Six: Build the encoder for S E Encode;

[0180] S61: Three coding layers are stacked, and the coding layer structure is as follows:

[0181] B1: Multi-head ProbSparse Attention technology is used to extract correlation features from the sequence;

[0182] B11: The Q-matrix and K-matrix of the attention mechanism are respectively derived from S E The V matrix is ​​obtained by performing causal convolution, which involves first adding two zero-padding operations at the beginning of the sequence, followed by a convolution operation with a kernel size of 3, and setting the number of kernels to d_model, meaning the feature dimension is mapped to d_model. The V matrix is ​​derived from S. E A linear mapping is performed, and the dimension of the mapping is also set to d_model. This leads to a result where all dimensions are L. E The Q, K, and V matrices of ×d_model;

[0183] B12: Set the number of heads for the attention mechanism to n_head, that is, divide the feature dimensions of Q, K and V into n_head parts respectively, and the feature dimension of each head is d = d_model / n_head, with a length of l = L. E ;

[0184] B13: For the i-th data point in a feature sequence of length l, let its q, k, and v values ​​be denoted as qi, ki, and vi, respectively. i k i and v i For q in the time series i With k j The similarity is calculated according to formula (19);

[0185]

[0186] B14: For k j Calculate using formula (19) to obtain q. i The similarity matrix Σ i , Σ i =[S1,S2,...S l ], where j = 1, 2, ..., l, and then the Softmax function is used to apply the Σ i Normalization is performed to convert it into a probability distribution, resulting in the probability distribution p(k) of qi. j |q i As shown in formula (20);

[0187]

[0188] B15: Note Calculate the distribution p(k) j |q i ) and uniform distribution q(k j |q i ) = 1 / L K The Kullback-Leibler divergence, used to represent the difference from a uniform distribution, is a measure of q. i The volatility of L, where L K Let l be the number of k values; calculate p(k) according to formula (21). j |q i ) and q(k j |q i The Kullback-Leibler divergence of ).

[0189]

[0190] B16: Discard the constant term lnL K According to formula (22), q is defined isparsity M(q) i ,K);

[0191]

[0192] B17: Utilization replace Rewrite q using formula (23) i sparsity;

[0193]

[0194] B18: Calculate q for each term i corresponding Scores, where i = 1, 2, ..., l, retain the 30 highest-scoring q values ​​U = 30. i The remaining positions are replaced by the mean, resulting in a new Q matrix, denoted as Q′;

[0195] B19: For the i-th term in the sequence, calculate q according to formula (24). i ′ and k j Similarity S j ′;

[0196]

[0197] B110: For k j The similarity between the i-th item in the sequence and each item is calculated sequentially using formula (24), where j = 1, 2, ..., l, and denoted as the similarity vector Σ. i = [S1′,S2′,...S l Then, the attention weight matrix of the i-th term is calculated using the Softmax function, as shown in formula (25);

[0198]

[0199] B111: α i Multiplying by V yields the attention score A for the i-th term. i As shown in formula (26);

[0200] A i =α i V(26);

[0201] B112: For each item in the sequence, repeat steps B19 to B111 to obtain the attention score for each item, thus obtaining the attention score matrix Λ = (A1, A2, ... A l );

[0202] B2: Activate the result of the attention mechanism using the Morlet mother wavelet basis function in formula (27) to obtain its first derivative. As shown in formula (28);

[0203]

[0204] In the formula, t is time; C and m are constants;

[0205] B3: Decompose the feature sequence, that is, F = [x1, x2, ..., x L The sequence is divided into three parts, and each part is extracted using a single-layer LSTM network to obtain F1′, F2′ and F3′. The three feature sequences are then merged and denoted as F′.

[0206] B4: Use a convolution kernel of size 3 with a stride of 1, and perform zero padding at the beginning and end to aggregate information from the feature matrix F′; then perform max pooling with a kernel of size 3, a stride of 2, and zero padding of 1, reducing the length of the feature vector to half its original length to improve computational efficiency, resulting in the feature matrix F′. Conv ;

[0207] S62: Same as B1, for F Conv Multi-head ProbSparse Attention is performed to obtain contextual information in the feature sequence, and then the wavelet function shown in formula (27) is used to activate it;

[0208] S63: Obtain the encoder's output feature sequence F E ;

[0209] Step 7: Construct the decoder for S D Decode;

[0210] S71: Use Multi-head ProbSparse Attention with a mask on the feature sequence S D The correlation model is performed to learn the correlation features in the sequence, and then the wavelet function shown in formula (27) is used to activate it to obtain the feature sequence F. Att ;

[0211] S72: Feature extraction is performed using Multi-head Attention, combined with F Att With the encoder output F E The inputs are fed into the Multi-head Attention layer, and a new feature sequence F is obtained according to formulas (24) to (26). Comb ;

[0212] S73: Uses a single convolutional layer to aggregate features from the attention mechanism output;

[0213] S74: Obtain the output feature sequence F of the decoder. D ;

[0214] Step 8: Using a single linear layer, convert the decoder's output sequence F... D Mapping yields the prediction results;

[0215] Step 9: Train the constructed model using the training set data. For the test set, evaluate the model using a retraining strategy to prevent information leakage.

[0216] S91: Divide the time series data after feature selection into training and test sets in a 5:3 ratio;

[0217] S92: Perform VMD denoising on the training set and input it into the model to train the model parameters;

[0218] S93: Set model parameters, set sliding window size L E This also sets the input sequence length L of the decoder, which is the length of the encoder's input sequence. D Set the length L of the overlapping sequence. Token Set the model hyperparameter d_model;

[0219] S94: Employ a small-batch strategy for batch training; set the batch size (batch_size) to divide the training set data into multiple batches for model training;

[0220] S95: Construct the loss function using formula (29);

[0221]

[0222] In the formula, n is the number of samples in the batch; y i Output the true value for the sample; These are the model's predicted values.

[0223] S96: The loss function is optimized using the Adam optimization algorithm, and a decaying learning rate strategy is adopted;

[0224] S97: Save the network weight parameters to obtain a prediction model for complex multivariate systems;

[0225] S98: Test the test set using a retraining strategy, as follows:

[0226] C1: Set the retraining interval to L. retrain .

[0227] C2: For the test set after feature selection, segment the first sample sequence. The VMD technique is used to decompose the input features of S1 in each dimension and reduce high-frequency noise, and then recombined to obtain the denoised samples.

[0228] C3: Load the model and make predictions for S1′.

[0229] C4: Add S1′ to the training set and re-perform VMD denoising on the new training set.

[0230] C5: For the new training set after denoising, retrain the model.

[0231] C6: For each sample in the test set Repeat the process from C2 to C5.

[0232] As a preferred option, in step nine, S91, the ratio is set to 5:3.

[0233] As a preferred embodiment, in step nine, S93, the sliding window size L is set. E The length L of the decoder's input sequence is 256. D The length of the overlapping sequence is 33, and the length of the overlapping part is L. Token Set the model hyperparameter d_model to 64, with a value of 32.

[0234] Furthermore, in order to effectively improve the training effect, during the model training process in step S94 of step nine, the batch size is set to 128 and the number of rounds is set to 50.

[0235] Furthermore, in order to effectively ensure the optimization effect, in step nine, S96, when the Adam optimization algorithm is used to optimize the loss function, the initial learning rate is set to 0.002.

[0236] Furthermore, to facilitate subsequent calculations, in step nine, S96, when employing the decay learning rate strategy, the decay step size is set to 15 and the decay factor is set to 0.1.

[0237] Validation Analysis:

[0238] This invention uses the high- and low-temperature conversion furnace process of industrial ammonia synthesis to verify the effectiveness and applicability of the network. The ammonia synthesis process involves 26 input variables and one output variable regarding the amount of carbon monoxide synthesized. The invented network was constructed using the PyCharm software platform. For 4000 sets of time series data collected, the first 2500 sets were selected as the training set, and the remaining 1500 sets were selected as the test set. The batch size for training and testing was set to 128. The number of iteration rounds was set to 50 rounds. The Adam network optimizer was selected. The initial learning rate was set to 0.002, and a learning rate decay strategy was adopted. After parameter adjustment, the update step size of the learning rate was selected to be 20; the decay factor was set to 0.1. For the weights of high-frequency signals when noise attenuation after VMD decomposition, a comparative experiment was designed, and the fitting effect was compared by setting the weights to 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, and 0.9 respectively. At the same time, Informer and LSTM networks were used to predict the experimental data. To quantitatively evaluate the predictive performance of different models, the mean square error (MSE) and the coefficient of determination R are defined according to formulas (30) and (31), respectively. 2 ;

[0239]

[0240] In the formula, n is the number of samples; For the network's predicted output; y i This is the actual sampled output; This is the average value of the actual sampled output.

[0241] For the test set, the MSE and R of different models 2 The values ​​are shown in Table 1, indicating that the proposed model has the highest prediction accuracy. For noise reduction using VMD, the coefficients of determination of the model fitting results under different weights of the high-frequency modes are as follows: Figure 2 The bar chart is shown. It can be seen that the model achieves the best prediction performance when the high-frequency modes are weakened by a weight of 0.1. For the test data, the loss function value of the proposed network changes with the number of iterations under each training round as shown below. Figure 3 As shown, the proposed model exhibits fast convergence speed in each retraining round. For the test set, the relative error curves of the fitting results for different models are shown below. Figure 4 As shown. Fitting curves for different models, as follows. Figure 5 As shown, the proposed model has the most accurate prediction performance for the system output. VMD denoising and ElasticNet key variable selection also provide better data for the model of this invention, making the model more robust.

[0242] In summary, this invention proposes a method for constructing a network prediction model for complex multivariate systems based on the Informer architecture and integrating VMD denoising and ElasticNet variable selection. VMD technology is used to separate high-frequency and low-frequency signals in the time series, and the high-frequency signals are weighted and recombine with the low-frequency signals to obtain a noise-reduced signal. ElasticNet technology is used to remove redundant input variables. Multi-head ProbSparseAttention and wavelet activation functions are used to encode the time series data. In the decoder, the data is processed through a masked Multi-head ProbSparseAttention to capture the correlations. The decoder results are mapped to the prediction results through a fully connected layer. Experimental comparisons verify that the proposed model has higher prediction accuracy. This invention effectively solves the shortcomings of traditional networks in predicting large errors and poor adaptability for systems with input noise and redundant variables, providing a precise model foundation for accurate prediction of key variables in complex multivariate processes, and enabling subsequent model-based production process controller design and operation monitoring.

[0243] Table 1. Calculation results of prediction error evaluation indicators for each model.

[0244]

Claims

1. A method for constructing a network prediction model for complex multivariable systems based on the Informer architecture, characterized in that, Includes the following steps: Step 1: Acquire production data for complex industrial processes; Various sensors are used to measure and collect various parameters in complex production processes to obtain production data for complex production processes; Step 2: Perform ElasticNet feature selection on the collected time series data to remove redundant variables; S21: Construct a linear regression model for time series data using formula (1); In the formula, X(i) and Let i be the input features and the predicted output for the i-th sample; ω is the coefficient vector matrix; S22: Define the objective function in formula (2) with L1 regularization and L2 regularization terms, and minimize the objective function to obtain the optimal parameters for fitting the time series; In the formula, Y(i) is the true output of the i-th sample; α is the regularization strength coefficient; ρ is the L1 regularization strength; α and ρ are used together to control the L1 and L2 regularization strengths; Step 3: Perform VMD decomposition and noise reduction on the sequence data obtained after feature selection; S31: The sequence data samples after feature selection have N dimensions, where the first m dimensions are the input features, represented as X′={x1,x2,...,x m }, where the Nm dimensions are the system output, represented as Y = {y m+1 ,y m+2 ,...,y N VMD for each dimension of feature x i Mode decomposition is performed, where i = 1, ..., m, and the number of mode components obtained from each eigenvalue decomposition is set to K = 2. The specific process is as follows: A1: VMD technology iteratively processes x i The K modal component signals are solved for each mode of (t); before iteration, wavelet transform or Fourier transform is used to initialize the K modal component signals to obtain each modal component signal as v. k (t), where k = 1, ..., K; A2: Regarding the modal component v k (t) is analyzed and negative frequencies are eliminated. The specific process is as follows: A21: According to formula (3), v k (t) Perform Hilbert transform; In the formula, * represents the convolution operation; Where sgn(ω) is the sign function, A22: Convert formula (3) to the frequency domain to obtain formula (4); In the formula, F(ω) is v k Fourier transform of (t); for The Fourier transform of h(t); h(ω) is the Fourier transform of h(t); A23: Transform formula (4) to obtain formula (5); In the formula, j satisfies j 2 =-1; A24: Define v according to formula (6) k The analytic signal z(t) of (t); A25: Combining formula (3), and based on the commutative and distributive laws of convolution, formula (6) is transformed to obtain formula (7); In the formula, δ(t) is the Dirac function; A3: Let the modal components v k The center frequency of (t) is ω k According to formula (8), the frequency of formula (7) is tuned to v. k The spectrum of (t) is modulated to the corresponding baseband; A4: Using Gaussian smoothing, calculate the gradient and corresponding L2 norm of formula (8), and take the square root to obtain the modal component v in formula (9). k Bandwidth estimation of (t); A5: VMD establishes the constrained variational optimization problem in formula (10) based on the principle of minimizing the sum of bandwidth estimates of each modal component, while ensuring that the sum of each modal component is still the original signal. In the formula, v k (k = 1, ..., K) represents each modal component; ω k (k = 1, ..., K) represents the center frequency of each modal component; A6: Introducing second-order multiplication factors and Lagrange multipliers, the constrained variational problem shown in formula (10) is transformed into an unconstrained variational problem using the augmented Lagrange function in formula (11); A7: The problem in the time domain shown in formula (11) is transformed into the frequency domain for solution. At the same time, the alternating direction multiplier method is used to address the modal components respectively. Center frequency of modal components and Lagrange multipliers Perform iterative calculations. and The iterative formulas are formula (12), formula (13) and formula (14), respectively; In the formula, τ is the noise tolerance; n is the number of iterations; and The original signal x i (t), modal component v k Fourier transform of (t) and Lagrange multiplier λ(t); A8: Repeat formulas (12) to (14) for calculation until the modal component v k (t) satisfies the iterative constraint conditions in formula (15); In the formula, ε is a given error constant; S32: Obtain the characteristic signals x after noise reduction according to formula (16). i ′(t), where i = 1, ..., m; x i ′(t)=ηv i1 (t)+v i2 (t)(16); In the formula, v i1 (t) and v i2 (t) represents the original feature signal x i (t) The high-frequency and low-frequency modes obtained after VMD decomposition with K=2; S33: The new input features after VMD denoising are represented as X″={x″1,x″2,...,x″} m The output is Y; Step 4: Window Setup and Sequence Partitioning; A sliding window approach is used, with single-step prediction, meaning only one data point is predicted at a time based on historical data; the sliding window length is set to L. E That is, when making a prediction, take the length L in front of it. E Historical data, denoted as S. E Simultaneously, the historical data sequence S E L in the middle Token Each data point is merged with one prediction point, with a length of L. D Let the sequence be S. D ; Step 5: Perform embedding operations on the sequence data, i.e., perform preliminary feature extraction to capture local and positional features; S51: Set up an embedding layer, including one convolutional layer and one positional embedding layer, and arrange them in parallel; the convolutional layer has a kernel size of 3, a stride of 1, and padding of 1 on both sides; the positional embedding layer uses the trigonometric functions in formula (17) and formula (18) to capture the positional features in the sequence data, so as to convert the positional information of the sequence data into a feature matrix; after the sequence is processed by the convolutional layer and the positional embedding layer in the embedding layer, the two are added together; In the formula, pos is the position of the point in the sequence; i starts from 0; 2i and 2i+1 represent the even-dimensional and odd-dimensional features of the point, respectively; d_model is the hyperparameter of the model; S52: Embedding operation on sequence data; for S E First, a single LSTM network is used to capture long-term dependencies, and then the feature is extracted in the embedding layer; for S D It directly enters the embedding layer for feature extraction; Step Six: Build the encoder for S E Encode; S61: Three coding layers are stacked, and the coding layer structure is as follows: B1: Multi-head ProbSparse Attention technology is used to extract correlation features from the sequence; B11: The Q-matrix and K-matrix of the attention mechanism are respectively derived from S E The V matrix is ​​obtained by performing causal convolution, which involves first adding two zero-padding operations at the beginning of the sequence, followed by a convolution operation with a kernel size of 3, and setting the number of kernels to d_model, meaning the feature dimension is mapped to d_model; the V matrix is ​​obtained by performing a convolution operation on S. E A linear mapping is performed, and the dimension of the mapping is also set to d_model; thus, a model with dimensions of L can be obtained. E The Q, K, and V matrices of ×d_model; B12: Set the number of heads for the attention mechanism to n_head, that is, divide the feature dimensions of Q, K and V into n_head parts respectively, and the feature dimension of each head is d = d_model / n_head, with a length of l = L. E ; B13: For the i-th data point in a feature sequence of length l, let its q, k, and v values ​​be denoted as qi, ki, and vi, respectively. i k i and v i For q in the time series i With k j The similarity is calculated according to formula (19); B14: For k j Calculate using formula (19) to obtain q. i The similarity matrix Σ i , Σ i =[S1,S2,...S l ], where j = 1, 2, ..., l, and then the Softmax function is used to apply the Σ i Normalization is performed to convert it into a probability distribution, resulting in the probability distribution p(k) of qi. j |q i As shown in formula (20); B15: Note Calculate the distribution p(k) j |q i ) and uniform distribution q(k j |q i ) = 1 / L K The Kullback-Leibler divergence, used to represent the difference from a uniform distribution, is a measure of q. i The volatility of L, where L K Let l be the number of k values; calculate p(k) according to formula (21). j |q i ) and q(k j |q i The Kullback-Leibler divergence of ). B16: Discard the constant term lnL K According to formula (22), q is defined i sparsity M(q) i ,K); B17: Utilization replace Rewrite q using formula (23) i sparsity; B18: Calculate q for each term i corresponding Scores, where i = 1, 2, ..., l, retain the 30 highest-scoring q values ​​U = 30. i The remaining positions are replaced by the mean, resulting in a new Q matrix, denoted as Q′; B19: For the i-th term in the sequence, calculate q according to formula (24). i ′ and k j Similarity S j ′; B110: For k j The similarity between the i-th item in the sequence and each item is calculated sequentially using formula (24), where j = 1, 2, ..., l, and denoted as the similarity vector Σ. i = [S1′,S2′,...S l Then, the attention weight matrix of the i-th term is calculated using the Softmax function, as shown in formula (25); B111: α i Multiplying by V yields the attention score A for the i-th term. i As shown in formula (26); A i =a i V(26); B112: For each item in the sequence, repeat steps B19 to B111 to obtain the attention score for each item, thus obtaining the attention score matrix Λ = (A1, A2, ... A l ); B2: Activate the result of the attention mechanism using the Morlet mother wavelet basis function in formula (27) to obtain its first derivative. As shown in formula (28); In the formula, t is time; C and m are constants; B3: Decompose the feature sequence, i.e., F = [x1, x2, ..., x L The sequence is divided into three parts, and each part is extracted using a single-layer LSTM network to obtain F1′, F2′ and F3′. The three feature sequences are then merged and denoted as F′. B4: Use a convolution kernel of size 3 with a stride of 1, and perform zero padding at the beginning and end to aggregate information from the feature matrix F′; then perform max pooling with a kernel of size 3, a stride of 2, and zero padding of 1, reducing the length of the feature vector to half its original length to improve computational efficiency, resulting in the feature matrix F′. Conv ; S62: Same as B1, for F Conv Multi-head ProbSparse Attention is performed to obtain contextual information in the feature sequence, and then the wavelet function shown in formula (27) is used to activate it; S63: Obtain the encoder's output feature sequence F E ; Step 7: Construct the decoder for S D Decode; S71: Use Multi-head ProbSparse Attention with a mask on the feature sequence S D The correlation model is performed to learn the correlation features in the sequence, and then the wavelet function shown in formula (27) is used to activate it to obtain the feature sequence F. Att ; S72: Feature extraction is performed using Multi-head Attention, combined with F Att With the encoder output F E The inputs are fed into the Multi-head Attention layer, and a new feature sequence F is obtained according to formulas (24) to (26). Comb ; S73: Uses a single convolutional layer to aggregate features from the attention mechanism output; S74: Obtain the output feature sequence F of the decoder. D ; Step 8: Using a single linear layer, convert the decoder's output sequence F... D Mapping yields the prediction results; Step 9: Train the constructed model using the training set data; evaluate the model using a retraining strategy on the test set to prevent information leakage. S91: Divide the time series data after feature selection into training and test sets in a 5:3 ratio; S92: Perform VMD denoising on the training set and input it into the model to train the model parameters; S93: Set model parameters, set sliding window size L E This also sets the input sequence length L of the decoder, which is the length of the encoder's input sequence. D Set the length L of the overlapping sequence. Token Set the model hyperparameter d_model; S94: Employ a small-batch strategy for batch training; set the batch size (batch_size) to divide the training set data into multiple batches for model training; S95: Construct the loss function using formula (29); In the formula, n is the number of samples in the batch; y i Output the true value for the sample; These are the model's predicted values; S96: The loss function is optimized using the Adam optimization algorithm, and a decaying learning rate strategy is adopted; S97: Save the network weight parameters to obtain a prediction model for complex multivariate systems; S98: Test the test set using a retraining strategy, as follows: C1: Set the retraining interval to L. retrain ; C2: For the test set after feature selection, segment the first sample sequence S1={(X1′,Y1),...,(X L ′ retrain ,Y Lretrain The VMD technique is used to perform mode decomposition of each dimension of the input features of S1 and to reduce high-frequency noise. The samples are then recombined to obtain the denoised samples S1′={(X1″,Y1),...,(X L " retrain ,Y Lretrain )}; C3: Load the model and predict S1′; C4: Add S1′ to the training set and re-perform VMD denoising on the new training set; C5: For the new training set after denoising, retrain the model; C6: For each sample in the test set Repeat the process from C2 to C5.

2. The method for constructing a network prediction model for complex multivariable systems based on the Informer architecture according to claim 1, characterized in that, In step nine, S91, the ratio is set to 5:

3.

3. The method for constructing a network prediction model for complex multivariable systems based on the Informer architecture according to claim 2, characterized in that, In step nine, S93, the sliding window size L is set. E The length L of the decoder's input sequence is 256. D The length of the overlapping sequence is 33, and the length of the overlapping sequence is L. Token Set the model hyperparameter d_model to 64, with a value of 32.

4. The method for constructing a network prediction model for complex multivariable systems based on the Informer architecture according to claim 3, characterized in that, In step nine, S94, during model training, the batch size is set to 128 and the number of rounds is set to 50.

5. The method for constructing a network prediction model for complex multivariable systems based on the Informer architecture according to claim 4, characterized in that, In step nine, S96, during the optimization of the loss function using the Adam optimization algorithm, the initial learning rate is set to 0.

002.

6. The method for constructing a network prediction model for complex multivariable systems based on the Informer architecture according to claim 5, characterized in that, In step nine, S96, during the process of adopting the decay learning rate strategy, the decay step size is set to 15 and the decay factor is set to 0.1.

Images