Cascade system online monitoring and prediction method based on sparse self-attention mechanism

Abstract

The invention discloses a cascade system online monitoring and prediction method based on a sparse self-attention mechanism, and the method comprises the steps: S1, obtaining and preprocessing multi-working-condition data of a cascade system, and constructing a time series data set; s2, performing embedded conversion and position coding on the data, and capturing sequence position information; s3, pre-fusion of adjacent time step information is realized through one-dimensional convolution; s4, a multi-head sparse attention mechanism is introduced, and a ReLU2 activation function is adopted to replace softmax so as to reduce calculation overhead; s5, completing information fusion through one-dimensional convolution, ELU activation and maximum pooling; s6, constructing an encoder containing a multi-head sparse attention mechanism; s7, designing an autoregressive decoder, and combining self-attention with cross attention; s8, adopting a HuberLoss loss function to train the model; and S9, carrying out reverse normalization on the model output to obtain a final prediction value. According to the invention, by optimizing the Transform architecture, 60 s effective prediction of the key parameters of the cascade system is realized, the prediction error is significantly reduced, and the intelligent early warning capability and the operation stability of the system are improved.

Description

Technical Field

[0001] This invention relates to the field of uranium enrichment engineering control application technology, and in particular to an online monitoring and prediction method for cascaded systems based on a sparse self-attention mechanism. Background Technology

[0002] Centrifugal cascade systems are core devices composed of multiple separation units connected in series and parallel. The fluid state directly affects separation performance and operational safety. Factors such as changes in external temperature, electrical frequency fluctuations, valve status changes, and machine malfunctions can all cause changes in the fluid state, leading to pressure disturbances that propagate within the system. Traditional monitoring methods rely on threshold alarms and manual experience, which have significant limitations: they cannot promptly capture subtle pressure changes in the early stages of disturbances, and cascade systems have nonlinear and multivariate coupling characteristics with complex parameter correlations, making it difficult for operators to predict the propagation trend and impact range of disturbances, easily leading to decreased production efficiency and safety hazards.

[0003] Time series forecasting techniques mainly include traditional statistical methods, machine learning methods, and deep learning methods. Traditional statistical methods (such as ARIMA and GARCH) are suitable for simple linear stationary data but struggle to handle complex nonlinear data. Machine learning methods (such as SVM and Random Forest) require extensive feature engineering, have poor interpretability, and are prone to overfitting. Deep learning methods (such as RNN, LSTM, and Transformer) are well-suited for complex data but suffer from high computational resource requirements and poor interpretability.

[0004] The Transformer model possesses the ability to uncover inherent patterns and correlations in data, but it faces key challenges in predicting the pressure of cascaded systems: high computational complexity when processing long sequence data leads to large memory consumption and low efficiency; and the lack of strict autoregressive dependencies in sequence generation tasks easily results in inconsistent prediction outcomes. Therefore, there is an urgent need for an online monitoring and prediction method for cascaded systems based on a sparse self-attention mechanism that balances computational efficiency and prediction accuracy. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing an online monitoring and prediction method for cascaded systems based on a sparse self-attention mechanism. By optimizing the Transformer architecture, an efficient and accurate time series prediction model is constructed to achieve real-time monitoring and trend prediction of key parameters of the cascaded system, quickly identify operational anomalies, assist operators in predicting the propagation trend and impact range of disturbances, and improve the system's intelligent early warning capability and operational stability.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A method for online monitoring and prediction of parameters of a cascaded system based on a sparse self-attention mechanism autoregressive model includes the following steps: S1. Data Processing: Obtain multivariate time series data of the cascaded system under stable operating conditions and different disturbance conditions, perform preprocessing operations, and construct a time series dataset. ; S2, Position Encoding: (This will...) Convert to embedding vector For each Add position encoding ,get ; S3. Information Pre-fusion: One-dimensional convolution is used to fuse information from adjacent time steps to obtain the output. ; S4. Introduce a multi-head sparse attention mechanism: Sparsify the query vector Q using a sparsity evaluation formula to obtain a probability sparse matrix. ;use Activation function, constructing the self-attention computation formula: ,in The attention score is represented by SA, which stands for self-attention mechanism. , That is, through Then perform the squaring operation; S5. Post-information fusion: After stacked attention layers, features are integrated sequentially through a one-dimensional convolutional layer, an ELU activation function, and a max-pooling layer to obtain the feature matrix. ; S6. Encoder Setup: Composed of multiple sub-layers, each sub-layer includes a multi-head sparse self-attention mechanism, a feedforward neural network, layer normalization, and residual connections. The encoder layer calculation formula is as follows:

[0007]

[0008] ; Among them, FFN is the feedforward neural network layer, LN is the layer normalization, and MHSA is the multi-head self-attention mechanism; S7. Decoder Setup: A self-attention plus cross-attention structure is adopted, both using sparse attention mechanisms. The original input sequence X is used as the decoder input prefix, and autoregressive time-step decoding is performed. The decoder is formally represented as follows: ; S8. Training the model: Set the model parameters, use HuberLoss as the loss function, and train the model using the Adam optimizer. S9. Parameter Prediction: Perform inverse normalization on the decoder output to obtain the final predicted values ​​of key parameters of the cascaded system.

[0009] Furthermore, the preprocessing operation of S1 includes normalization of the maximum and minimum values, and the normalization formula is:

[0010] in For a certain measuring point, These are the maximum and minimum values ​​of the measurement point, respectively, with a normalization range of [0,1]. The preprocessed data is divided into training and test sets in an 8:2 ratio.

[0011] Furthermore, S2 uses the embedding function Embedding to... Convert to embedding vector Specifically: For input Each element It is converted into an embedding vector using an embedding function called Embedding. ,Right now

[0012] in It is a two-dimensional convolutional layer that processes the discrete elements of the input. Mapped to a continuous vector space of fixed dimension:

[0013] To capture positional information in the sequence, for each embedding vector Add position encoding ,Right now ′= ; in Using trigonometric function absolute position encoding:

[0014]

[0015] in It is a frequency parameter used to control the periodicity of position coding. It is the position index in the sequence. It is the dimension index in the embedded vector. is the dimension of the embedding vector.

[0016] Furthermore, the multi-head sparse attention mechanism of S4 achieves sparsity in two ways: First, in a time series of length 60, each time step only considers the attention weights of the 15 time steps before and after it, with 15 determined through hyperparameter tuning; Second, in the cascaded system, the Nth level has the most direct impact on the Nth level, with the influence decreasing sequentially outwards; the calculation for each level only considers the attention weights of the 5 levels before and after it, and the sparsity processing employs... The values ​​are sorted by metric, and then the Top-u values ​​are used for calculation.

[0017] Furthermore, the attention score of S4 The calculation formula is W T W Q W K Let b be the learnable weight matrix, and b be the bias term.

[0018] Furthermore, the feedforward neural network of the encoder in S6 consists of two linear layers stacked together, with ReLU as the activation function, calculated as follows: Where W1 and W2 are weight matrices, and b1 and b2 are bias terms; the layer normalization calculation formula is: ,in, These are the mean and standard deviation of the input, respectively. These are learnable parameters.

[0019] Furthermore, the decoder of S7 includes two multi-head attention layers: the first is a self-attention layer, with the input being... After embedding transformation, the input is given to a multi-head sparse attention layer, and the output is... The second is the cross-attention layer, through... The calculation is performed, where CA stands for cross-attention. Query as an attention mechanism, and The key and value of the attention mechanism are used; the final output of the decoder is obtained through a linear transformation. ,in The output weight matrix is ​​denoted by b, which is the bias term.

[0020] Furthermore, the HuberLoss loss function of S8 is: ,in These are parameters that control the behavior of the loss function and are called "smoothing parameters"; The actual label for the corresponding sample. The predicted value of the model; the first-order momentum decay rate of the Adam optimizer. Second-order momentum decay rate The learning rate is 0.001. The learning rate is adjusted using an early stop + linear descent method, with a patience step count of 5.

[0021] Furthermore, the model's encoder contains 2 layers, the decoder contains 1 layer, and each hidden layer has a dimension of 32; the multi-head attention mechanism has 8 heads, an embedding dimension of 512, an output layer dimension of 32, and the Dropout parameter is set to 0.05; the model training iterations are 100, and the batch size is 200.

[0022] Furthermore, it also includes an online anomaly monitoring step: setting a 5-second monitoring window and calculating the first-order difference of the data at each measuring point within the window. ,when When the threshold is exceeded, the parameter prediction process in step S9 is triggered.

[0023] Beneficial Effects: This invention addresses the challenge of parameter prediction caused by the complex temporal dynamic relationships between data in cascaded systems. It proposes a monitoring and prediction method based on a Transformer model. By constructing an autoregressive time series prediction model with a sparse self-attention mechanism, real-time monitoring and trend prediction of key parameters in cascaded systems are achieved. Compared with the traditional Transformer model, this method achieves a mean absolute error, mean absolute error percentage, and root mean square error of 0.42%, 1.5%, and 0.48 for pressure prediction, respectively. These represent reductions of 59.6%, 57.1%, and 61.9% compared to the traditional Transformer model. It also achieves effective prediction of key parameters within 60 seconds, effectively improving the system's safety and stability. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is the main flowchart of the online monitoring and prediction method for cascade system parameters based on a sparse self-attention mechanism autoregressive model as described in the embodiments of the present invention; Figure 2 This is a schematic diagram of the predictive model in the online monitoring and prediction method for cascade system parameters based on a sparse self-attention mechanism autoregressive model as described in this embodiment of the invention. Figure 3 This is a schematic diagram of the multi-head sparse self-attention layer in the online monitoring and prediction method for cascade system parameters based on an autoregressive model using a sparse self-attention mechanism, as described in an embodiment of the present invention. Figure 4 This is the pressure prediction curve 1 in the online monitoring and prediction method for cascade system parameters based on a sparse self-attention mechanism autoregressive model described in this embodiment of the invention. Figure 5This is the second pressure prediction curve in the online monitoring and prediction method for cascade system parameters based on a sparse self-attention mechanism autoregressive model described in this embodiment of the invention. Figure 6 The loss curve during the training process in the online monitoring and prediction method for cascade system parameters based on a sparse self-attention mechanism autoregressive model as described in this embodiment of the invention; Figure 7 This is a verification loss curve used in the online monitoring and prediction method for cascade system parameters based on a sparse self-attention mechanism autoregressive model, as described in this embodiment of the invention. Figure 8 This is the abnormal signal curve of a certain measuring point simulated by the simulator in the online monitoring and prediction method of cascade system parameters based on the autoregressive model of sparse self-attention mechanism described in the embodiments of the present invention. Figure 9 This is the first-order difference curve of the abnormal signal in the online monitoring and prediction method for cascade system parameters based on the autoregressive model of sparse self-attention mechanism described in the embodiments of the present invention. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] This embodiment is implemented on the Ubuntu 22.04 operating system, based on the PyTorch 2.4.0 deep learning framework, and using the Python 3.10 programming language.

[0028] Example 1 See Figure 1-9 A method for online monitoring and prediction of parameters of a cascaded system based on a sparse self-attention mechanism autoregressive model, comprising the following steps: S1. Data Processing: Obtain multivariate time series data of the cascaded system under stable operating conditions and different disturbance conditions, perform preprocessing operations, and construct a time series dataset. ; S2, Position Encoding: (This will...) Convert to embedding vector For each Add position encoding ,get ; S3. Information Pre-fusion: One-dimensional convolution is used to fuse information from adjacent time steps to obtain the output. ; The final time series embedding in this embodiment Through one-dimensional convolution operation Perform local feature extraction to obtain the output. In this embodiment, the kernel size is set to 7 and the stride is set to 1. This operation captures local patterns in the sequence and compresses the dimensionality of features or reorganizes the feature distribution.

[0029] S4. Introduce a multi-head sparse attention mechanism: Sparsify the query vector Q using a sparsity evaluation formula to obtain a probability sparse matrix. ;use Activation function, constructing the self-attention computation formula: ,in The attention score is represented by SA, which stands for self-attention mechanism. , That is, through Then perform the squaring operation; S5. Post-information fusion: After stacked attention layers, features are integrated sequentially through a one-dimensional convolutional layer, an ELU activation function, and a max-pooling layer to obtain the feature matrix. ; S6. Encoder Setup: Composed of multiple sub-layers, each sub-layer includes a multi-head sparse self-attention mechanism, a feedforward neural network, layer normalization, and residual connections. The encoder layer calculation formula is as follows:

[0030]

[0031] ; Among them, FFN is the feedforward neural network layer, LN is the layer normalization, and MHSA is the multi-head self-attention mechanism; S7. Decoder Setup: A self-attention plus cross-attention structure is adopted, both using sparse attention mechanisms. The original input sequence X is used as the decoder input prefix, and autoregressive time-step decoding is performed. The decoder is formally represented as follows: ; S8. Training the model: Set the model parameters, use HuberLoss as the loss function, and train the model using the Adam optimizer. S9. Parameter Prediction: Perform inverse normalization on the decoder output to obtain the final predicted values ​​of key parameters of the cascaded system.

[0032] This invention employs a sparse self-attention mechanism to replace the traditional full attention mechanism of Transformer, combined with ReLU. 2 The activation function and attention computation complexity are significantly reduced, significantly reducing computational resource consumption, improving model running efficiency, and adapting to the real-time requirements of online monitoring in cascaded systems.

[0033] In a specific example, the preprocessing operation of S1 includes normalization of the maximum and minimum values, and the normalization formula is:

[0034] in For a certain measuring point, These are the maximum and minimum values ​​of the measurement point, respectively, with a normalization range of [0,1]. The preprocessed data is divided into training and test sets in an 8:2 ratio.

[0035] In the specific implementation, based on the operating data of a cascaded system simulator under different working conditions, data from stable and adjustment conditions were collected at 1-second intervals through an interface program. Data from 32 key pressure measurement points on the feed main and lean feed main were obtained, and a total of 260 sets of valid data were extracted. Subsequently, the raw data was preprocessed, and the processed dataset was converted into NumPy array format.

[0036] It should be noted that, given the significant mutual influence between different measuring points, data from all 32 measuring points are simultaneously input into the model to predict the values ​​for the next 60 seconds at all measuring points. Selecting the 60-second pressure data from all 32 measuring points as the feature matrix of a single sample input into the model yields the preprocessed time series described above.

[0037] The time series for each measuring point is as follows:

[0038] Its corresponding tag is , . The dataset and label set are split. Following an 8:2 ratio, 80% of the samples are divided into the training dataset train_X and 20% into the test dataset test_X; similarly, 80% of the samples are divided into the training label set train_y and 20% into the test label set test_y.

[0039] In a specific example, S2 uses the embedding function Embedding to... Convert to embedding vector Specifically: For input Each element It is converted into an embedding vector using an embedding function called Embedding. ,Right now

[0040] in It is a two-dimensional convolutional layer that processes the discrete elements of the input. Mapped to a continuous vector space of fixed dimension:

[0041] To capture positional information in the sequence, for each embedding vector Add position encoding ,Right now ′= ; in Using trigonometric function absolute position encoding:

[0042]

[0043] in It is a frequency parameter used to control the periodicity of position coding. It is the position index in the sequence. It is the dimension index in the embedded vector. is the dimension of the embedding vector.

[0044] In a specific example, the multi-head sparse attention mechanism of S4 achieves sparsity in two ways: First, in a time series of length 60, each time step only considers the attention weights of the 15 time steps before and after it, where 15 is determined through hyperparameter tuning; Second, in a cascaded system, the Nth level has the most direct impact on the Nth level, with the influence decreasing sequentially outwards; the calculation for each level only considers the attention weights of the 5 levels before and after it, and the sparsity processing employs... The values ​​are sorted by metric, and then the Top-u values ​​are used for calculation.

[0045] In the specific implementation, the following was adopted: It replaces the traditional softmax function, greatly reducing computational overhead.

[0046]

[0047] MHSA stands for Multi-Head Self-Attention Mechanism, and Concat is the join function. The self-attention mechanism is as follows:

[0048] SA stands for Self-Attention Mechanism , That is, through Then perform the squaring operation.

[0049] Here, the activation function is improved from the traditional SoftMax to... Soon

[0050] Improved to:

[0051] And for each element :

[0052]

[0053] in The weight matrix is ​​a learnable matrix. The activation function introduces non-linearity into the attention score; During the sparsification process It is a sparse matrix of the same size as q, containing the top-u values ​​obtained according to the ranking score, measured by... ,Right now

[0054] By order This can reduce the computational complexity of attention to a minimum. ,in The length of the sequence.

[0055] Feedforward neural networks perform nonlinear transformations on input sequences. They consist of two linear layers stacked together and employ... As an activation function, the input sequence is first transformed through a linear layer, then nonlinearity is introduced by applying the ReLU function, and finally a second transformation is performed through another linear layer.

[0056] Layer normalization:

[0057] in, These are the mean and standard deviation of the input, respectively. These are learnable parameters.

[0058] Post-fusion, following multiple stacked attention layers, performs feature fusion to integrate deep information from different feature layers, thereby enhancing the expressive power of the final features.

[0059] This method employs a one-dimensional convolutional layer to capture local patterns in the sequence and uses the ELU activation function. Finally, a max-pooling layer is used to downsample the activated features.

[0060] In a specific instance, the attention score of S4 The calculation formula is W T W Q W K Let b be the learnable weight matrix, and b be the bias term.

[0061] In a specific example, the feedforward neural network of the encoder in S6 consists of two linear layers stacked together, with ReLU as the activation function, calculated as follows: Where W1 and W2 are weight matrices, and b1 and b2 are bias terms; the layer normalization calculation formula is: ,in, These are the mean and standard deviation of the input, respectively. These are learnable parameters.

[0062] In its implementation, the encoder layer consists of two sub-layers, each containing a multi-head sparse self-attention mechanism, a feedforward neural network, layer normalization, and residual connections. The encoder layer is designed to progressively extract high-order features from the time series and reduce computational complexity through the sparse self-attention mechanism.

[0063]

[0064]

[0065]

[0066] Here, FFN represents the feedforward neural network layer, and LN represents the layer normalization.

[0067] Multi-head sparse attention mechanism, such as Figure 3 As shown, the dimensions of the input sequence are (batch_size, seq_len, input_dim), set to (200, 60, 32) respectively. The input sequence is mapped to query, key, and value vectors through a linear transformation. These vectors are then split into multiple heads, each with dimensions (batch_size, num_heads, seq_len, head_dim). For this embodiment, the number of heads (num_heads) is determined using hyperparameter tuning since there is no prior experience in selection. The number of heads is used as a hyperparameter and tuned through cross-validation and grid search methods. Different numbers of heads are tried, and the model's performance on the validation set is compared. Finally, a number of heads of 8 is determined.

[0068] In a specific example, the decoder of S7 includes two multi-head attention layers: the first is a self-attention layer, with the input being... After embedding transformation, the input is given to a multi-head sparse attention layer, and the output is... The second is the cross-attention layer, through... The calculation is performed, where CA stands for cross-attention. Query as an attention mechanism, and The key and value of the attention mechanism are used; the final output of the decoder is obtained through a linear transformation. ,in The output weight matrix is ​​denoted by b, which is the bias term.

[0069] It should be noted that the decoder is designed with a self-attention plus cross-attention structure, both of which employ sparse attention mechanisms. To preserve the original input information, the original input sequence is used as an input prefix for the decoder, ensuring that the model can refer to the contextual information of the input data when generating predictions, followed by autoregressive time-step decoding.

[0070]

[0071] Specifically, the decoder contains two multi-head attention layers, the first of which is a self-attention layer whose input is... After transformation, the embedding vector is obtained. Input a multi-head sparse attention layer.

[0072]

[0073] Furthermore, the encoder output and Perform cross-attention calculation

[0074] Here, CA stands for Cross Attention. Query as an attention mechanism, and As Key and Value of the attention mechanism.

[0075] After processing through multiple decoder layers, the final output is After a linear transformation, the final predicted output is generated.

[0076]

[0077] For each sample , The corresponding label was predicted as , .

[0078] In a specific example, the HuberLoss loss function of S8 is: ,in These are parameters that control the behavior of the loss function and are called "smoothing parameters"; The actual label for the corresponding sample. The predicted value of the model; the first-order momentum decay rate of the Adam optimizer. Second-order momentum decay rate The learning rate is 0.001. The learning rate is adjusted using an early stop + linear descent method, with a patience step count of 5.

[0079] It should be noted that during model training, the loss function between the true value and the model prediction value of each sample is calculated, and the Adam optimizer is used to continuously update the model weights during backpropagation to minimize the loss.

[0080] The principle behind using the Adam optimizer to update the model's weights is as follows:

[0081]

[0082]

[0083]

[0084]

[0085] Where m is the first-order moment estimator and v is the second-order moment estimator. It is the decay rate estimated by the first moment. It is the attenuation rate estimated by the second moment. It is the gradient of the loss function with respect to the model parameters, and t is the current iteration number. It's the learning rate. It is a small constant to prevent the denominator from being zero.

[0086] In a specific example, the model's encoder has 2 layers, the decoder has 1 layer, and each hidden layer has a dimension of 32; the multi-head attention mechanism has 8 heads, an embedding dimension of 512, an output layer dimension of 32, and the Dropout parameter is set to 0.05; the model is trained in 100 iterations, and the batch size is 200.

[0087] In the implementation, the encoder consists of two layers, and the decoder consists of one layer. Each layer has a hidden layer dimension of 32. The model employs a multi-head attention mechanism with 8 heads, an embedding dimension of 512, and an output layer dimension of 32. The activation function is ReLU2, and Dropout is set to 0.05. During training, cross-entropy loss is used as the supervisory signal generator, and a margin parameter δ=1 is introduced to construct a robust error surface. The optimizer is the Adam algorithm with a correction term, where the first-order momentum decay rate β1 and the second-order momentum decay rate β2 are configured to 0.9 and 0.999, respectively. The learning rate descent strategy uses early stopping combined with linear descent, with a patience step count of 5 and a learning rate of 0.001. The number of iterations (epochs) is 100, the batch size is 200, and parallel computation is performed using 8 GPUs.

[0088] Given the obtained training dataset train_X and training label set train_y, input them into the model for training. The convergence process of model training is described in [link to documentation]. Figure 6 The model was also validated, and the validation loss is shown in [link to validation loss diagram]. Figure 7 .

[0089] Model testing: Test the model on the test set and display the predicted / true values. Input the test dataset `test_X` and the test label set `test_y` into the model for testing. The output needs to be inversely normalized using the maximum and minimum values ​​to obtain the final predicted value.

[0090] After predicting each sample in the test set, the mean absolute error, mean absolute error percentage, and root mean square error of the 32 channel predictions were 0.42, 1.5%, and 0.48, respectively.

[0091] The neural network involved in this embodiment of the invention can predict data for the next 60 seconds, such as... Figure 4-5 As shown, the comparison between the predicted signal data and the actual data demonstrates a good prediction effect, and the predicted values ​​meet the pressure requirements of the centrifugal cascade system.

[0092] In a specific example, the online anomaly monitoring step is also included: setting a 5-second monitoring window and calculating the first-order difference of the data at each measuring point within the window. ,when When the threshold is exceeded, the parameter prediction process in step S9 is triggered.

[0093] In this embodiment, during the prediction phase, the system implements an online monitoring strategy, namely the maximum first-order difference algorithm. Specifically, the system continuously monitors the status of each measurement point, and triggers the parameter prediction process once an anomaly is detected. For example... Figure 8 , Figure 9 As shown, this algorithm focuses on the changes of the measurement points in a short period of time.

[0094] It should be noted that when the first-order difference of any channel is detected... If the threshold is exceeded, the system considers the measurement point to have an anomaly. At this point, a pre-trained parameter prediction model will be activated to predict the abnormal parameters, so that timely measures can be taken to prevent potential system failures or performance degradation.

[0095] This embodiment introduces a first-order differential anomaly monitoring strategy, which can quickly identify parameter mutations, trigger the prediction process in a timely manner, assist operators in predicting the trend and scope of disturbance propagation, and significantly improve the system's intelligent early warning capability and operational safety and stability.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for online monitoring and prediction of parameters of a cascaded system based on a sparse self-attention mechanism autoregressive model, characterized in that, Includes the following steps: S1. Data Processing: Obtain multivariate time series data of the cascaded system under stable operating conditions and different disturbance conditions, perform preprocessing operations, and construct a time series dataset. ; S2, Position Encoding: (This will...) Convert to embedding vector For each Add position encoding ,get ; S3. Information Pre-fusion: One-dimensional convolution is used to fuse information from adjacent time steps to obtain the output. ; S4. Introduce a multi-head sparse attention mechanism: Sparsify the query vector Q using a sparsity evaluation formula to obtain a probability sparse matrix. ;use Activation function, constructing the self-attention computation formula: ,in The attention score is represented by SA, which stands for self-attention mechanism. , That is, through Then perform the squaring operation; S5. Post-information fusion: After stacked attention layers, features are integrated sequentially through a one-dimensional convolutional layer, an ELU activation function, and a max-pooling layer to obtain the feature matrix. ; S6. Encoder Setup: Composed of multiple sub-layers, each sub-layer includes a multi-head sparse self-attention mechanism, a feedforward neural network, layer normalization, and residual connections. The encoder layer calculation formula is as follows: ； Among them, FFN is the feedforward neural network layer, LN is the layer normalization, and MHSA is the multi-head self-attention mechanism; S7. Decoder Setup: A self-attention plus cross-attention structure is adopted, both using the aforementioned sparse attention mechanism. The original input sequence X is used as the decoder input prefix, and autoregressive time-step decoding is performed. The decoder is formally represented as follows: ; S8. Training the model: Set the model parameters, use HuberLoss as the loss function, and train the model using the Adam optimizer. S9. Parameter Prediction: Perform inverse normalization on the decoder output to obtain the final predicted values ​​of key parameters of the cascaded system.

2. The method according to claim 1, characterized in that, The preprocessing operation of S1 includes normalization of the maximum and minimum values, and the normalization formula is as follows: in For a certain measuring point, These are the maximum and minimum values ​​of the measuring point, respectively, with a normalized range of [0,1]. The preprocessed data was divided into training and test sets in an 8:2 ratio.

3. The method according to claim 1, characterized in that, S2 uses the embedding function Embedding to... Convert to embedding vector Specifically: For input Each element It is converted into an embedding vector using an embedding function called Embedding. ,Right now in It is a two-dimensional convolutional layer that processes the discrete elements of the input. Mapped to a continuous vector space of fixed dimension: To capture positional information in the sequence, for each embedding vector Add position encoding ,Right now ′= ; in Using trigonometric function absolute position encoding: in It is a frequency parameter used to control the periodicity of position coding. It is the position index in the sequence. It is the dimension index in the embedded vector. is the dimension of the embedding vector.

4. The method according to claim 1, characterized in that, The multi-head sparse attention mechanism of S4 achieves sparsity in two ways: First, in a time series of length 60, each time step only considers the attention weights of the 15 time steps before and after it, with 15 determined through hyperparameter tuning; Second, in a cascaded system, the Nth level has the most direct impact on the Nth level, with the influence decreasing sequentially outwards; the calculation for each level only considers the attention weights of the 5 levels before and after it, and the sparsity processing employs... The values ​​are sorted by metric, and then the Top-u values ​​are used for calculation.

5. The method according to claim 1, characterized in that, The attention score of S4 The calculation formula is W T W Q W K Let b be the learnable weight matrix, and b be the bias term.

6. The method according to claim 1, characterized in that, The feedforward neural network of the encoder in S6 consists of two linear layers stacked together, with ReLU as the activation function, and the calculation formula is: Where W1 and W2 are weight matrices, and b1 and b2 are bias terms; the layer normalization calculation formula is: ,in, These are the mean and standard deviation of the input, respectively. These are learnable parameters.

7. The method according to claim 1, characterized in that, The decoder of S7 contains two multi-head attention layers: the first is a self-attention layer, with the input being... After embedding transformation, the input is given to a multi-head sparse attention layer, and the output is... The second is the cross-attention layer, through... The calculation is performed, where CA stands for cross-attention. Query as an attention mechanism, and The key and value of the attention mechanism are used; the final output of the decoder is obtained through a linear transformation. ,in The output weight matrix is ​​denoted by b, which is the bias term.

8. The method according to claim 1, characterized in that, The HuberLoss loss function of S8 is: ,in These are parameters that control the behavior of the loss function and are called "smoothing parameters"; The actual label for the corresponding sample. The predicted value of the model; the first-order momentum decay rate of the Adam optimizer. Second-order momentum decay rate The learning rate is 0.

001. The learning rate is adjusted using an early stop + linear descent method, with a patience step count of 5.

9. The method according to claim 1, characterized in that, The model's encoder has 2 layers, the decoder has 1 layer, and each hidden layer has a dimension of 32; the multi-head attention mechanism has 8 heads, an embedding dimension of 512, an output layer dimension of 32, and the Dropout parameter is set to 0.05; the model is trained in 100 iterations and the batch size is 200.

10. The method according to claim 1, characterized in that, It also includes an online anomaly monitoring step: setting a 5-second monitoring window and calculating the first-order difference of the data at each measuring point within the window. ,when When the threshold is exceeded, the parameter prediction process in step S9 is triggered.

Images