A method for multivariate time series prediction of thermal power generating units based on fine-tuning large models

Abstract

The application discloses a kind of based on fine-tuning large model thermal power unit multivariate time series prediction method and system, belong to industrial time series prediction technical field.The method is first to thermal power unit multivariate time series data denoising and formalization modeling, by Patching technology completes time series tokenization;Based on pure decoder architecture constructs multivariate NTP paradigm, utilizes Kronecker product to realize the time-space attention decoupling of Timer-XL model, in combination with double position encoding accurate capture space-time dependence;Adapt industrial data distribution using end-to-end fine-tuning, finally in autoregressive mode completes prediction, and realizes model explainable analysis by integral gradient.The application greatly reduces the computational complexity, improves the prediction accuracy, adapts to the complex working conditions of thermal power unit, solves the problems of large calculation overhead, space-time modeling inaccuracy and poor explainability of traditional models, and is suitable for thermal power unit operation and maintenance, energy scheduling and other industrial scenarios.

Description

Technical Field

[0001] This invention relates to the fields of industrial time series data prediction, large model fine-tuning, and intelligent operation and maintenance of thermal power units, specifically to a high-precision multivariate time series prediction method for thermal power units based on a fine-tuning Timer-XL large model. Background Technology

[0002] Multivariate time series forecasting is a core technology for the operation and maintenance of thermal power units, energy dispatch, and industrial process control. The operating status of thermal power units is determined by high-dimensional coupled time-series data such as temperature, pressure, and flow rate, simultaneously collected by multiple sensors distributed across different physical nodes. This requires modeling both the non-stationary dynamic laws of the time dimension and the complex spatial interactions of the variables, such as… Figure 1 The image shows a comparison of different time series forecasting architectures.

[0003] The existing technology has the following drawbacks: 1. The standard Transformer flattens multivariate data into a one-dimensional sequence. The expansion of the sequence length causes the computational complexity of self-attention to increase quadratically, which is unacceptable in long context scenarios. 2. The global attention mechanism confuses temporal causal dependencies with variable symmetric dependencies, making it difficult to extract effective spatiotemporal correlations and limiting prediction accuracy; 3. When traditional generative models are directly adapted to multivariate industrial scenarios, they suffer from modal mismatch and feature collapse, and cannot accurately capture the physical characteristics of thermal power units such as thermal inertia and variable coupling. 4. Existing time series large models lack attention decoupling mechanisms for multivariate industrial scenarios, and the model decision-making process is a black box with poor interpretability, making it difficult to meet the safety operation and maintenance needs of industrial sites. Summary of the Invention

[0004] This invention provides a multivariate time series prediction method for thermal power units based on a fine-tuned large model, in order to solve the technical problems existing in the prior art.

[0005] To achieve the above objectives, this invention provides a method for multivariate time series prediction of thermal power units based on a fine-tuned large model, which includes the following steps: (1) Adaptive noise reduction is performed on the multivariate original time series data of thermal power units, and the historical observation matrix and prediction target sequence are formally defined to transform the prediction task into sequence-to-sequence regression modeling. (2) The timing signal is processed by patching block technology to convert the continuous numerical signal into a discrete timing lexical sequence, thereby reducing the sequence length; (3) Based on the pure decoder architecture, a multivariate sub-term prediction (NTP) paradigm is constructed, which flattens the two-dimensional spatiotemporal data with a time-first strategy and ensures the unidirectional flow of information in the time dimension through causal masking; (4) Build the Timer-XL model, decouple global attention into temporal causal attention and variable fully connected attention through Kronecker product, and complete spatiotemporal dependency modeling by combining dual positional encoding; (5) An end-to-end full-sample fine-tuning strategy is adopted to adapt to the data distribution of thermal power units and optimize model parameters; (6) The prediction reasoning is completed in an autoregressive manner, and the interpretability analysis of the model decision is achieved through the integral gradient method.

[0006] In one embodiment of the present invention, optionally, in step (1), the original time series data includes temperature, pressure and flow rate.

[0007] In one embodiment of the present invention, optionally, in step (1), VMD-PE-wavelet is used for adaptive noise reduction.

[0008] In one embodiment of the present invention, optionally, in step (2), the lexicalization adopts non-overlapping segmentation, the patch length P=96, and the patch is mapped to d_model dimensional lexical through linear projection, and the sequence length is compressed from L to L / P.

[0009] In one embodiment of the present invention, optionally, in step (4), the joint mask matrix is ​​calculated by the Kronecker product of the variable dependency matrix and the temporal causal mask matrix, the time dimension adopts rotational position encoding (RoPE), and the variable dimension introduces learnable scalar parameters u and v.

[0010] In one embodiment of the present invention, optionally, in step (5), the fine-tuning adopts the AdamW optimizer, the mean squared error (MSE) loss function, and combines cosine annealing learning rate scheduling, mixed precision training and gradient clipping.

[0011] In one embodiment of the present invention, optionally, in step (6), the integral gradient is based on the all-zero matrix to calculate the attribution matrix of the input features to the prediction result, and to visualize the contribution of time and variable dimensions.

[0012] This invention also provides a multivariate time series prediction system for thermal power units based on a fine-tuned large model, which includes: The data preprocessing module is used to perform noise reduction and formal definition of time-series data of thermal power units; The lexicalization module is used to implement patching and linear projection of time-series signals; The model building module is used to build the Timer-XL model, achieving attention decoupling and dual positional encoding; The fine-tuning module is used to perform end-to-end full-sample fine-tuning; The reasoning and interpretation module is used for autoregressive prediction and integral gradient interpretability analysis.

[0013] In one embodiment of the present invention, the Timer-XL model is optionally an 8-layer pure decoder architecture.

[0014] In one embodiment of the present invention, optionally, the hidden layer dimension of the Timer-XL model is 1024 and the feedforward network dimension is 2048.

[0015] The multivariate time series prediction method for thermal power units based on fine-tuned large models provided by this invention has the following beneficial technical effects: 1. Significantly improved computational efficiency: By decoupling Patching lexicalization and Kronecker product attention, the computational complexity is reduced from O(L²V²) to O((L / P)²+V²), supporting long context modeling at the level of thousands of patches; 2. Optimal prediction accuracy: On the thermal power unit dataset, the MSE is as low as 0.322558 and the MAE is as low as 0.292739, which is 7.9% lower than the suboptimal model error, accurately capturing the coupling law between the unit's thermal inertia and variable physics; 3. Strong industrial adaptability: Unified prediction, filling, and conditional extrapolation tasks using the multivariate NTP paradigm, end-to-end fine-tuning to adapt to the data distribution of thermal power units, and solving the modal mismatch problem; 4. Complete interpretability: The visualization of integral gradient attribution reveals the temporal nearest neighbor effect and variable asymmetric dependence, providing a basis for decision-making in industrial operation and maintenance. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 Comparison chart of different time series forecasting architectures; Figure 2 This is a performance comparison chart of the multivariate time series forecasting baseline models MSE and MAE; Figure 3 Heatmap of attribution analysis for multivariate prediction features of the Timer-XL model; Figure 4 Heatmap of the intervariate attribution matrix for the Timer-XL model. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] First embodiment: This invention provides a method for multivariate time series prediction of thermal power units based on fine-tuning large models, which includes the following steps: (1) Adaptive noise reduction is performed on the multivariate original time series data of thermal power units, and the historical observation matrix and prediction target sequence are formally defined to transform the prediction task into sequence-to-sequence regression modeling. The original time series data in step (1) includes, for example, temperature, pressure and flow rate. In other embodiments, the original time series data may be added to or subtracted from this data. This invention is not limited to the data listed above.

[0020] In step (1), for example, VMD-PE-wavelet adaptive noise reduction can be used. In other embodiments, other methods can also be used for adaptive noise reduction.

[0021] (2) The timing signal is processed by patching block technology to convert the continuous numerical signal into a discrete timing lexical sequence, thereby reducing the sequence length; In step (2), non-overlapping segmentation is used for lexicalization. The patch length is P=96. The patch is mapped to d_model dimensional lexicals through linear projection, and the sequence length is compressed from L to L / P.

[0022] (3) Based on the pure decoder architecture, a multivariate sub-term prediction (NTP) paradigm is constructed, which flattens the two-dimensional spatiotemporal data with a time-first strategy and ensures the unidirectional flow of information in the time dimension through causal masking; (4) Build the Timer-XL model, decouple global attention into temporal causal attention and variable fully connected attention through Kronecker product, and complete spatiotemporal dependency modeling by combining dual positional encoding; In step (4), the joint mask matrix is ​​calculated by the Kronecker product of the variable dependency matrix and the temporal causality mask matrix. The time dimension adopts rotation position encoding (RoPE), and the variable dimension introduces learnable scalar parameters u and v.

[0023] (5) An end-to-end full-sample fine-tuning strategy is adopted to adapt to the data distribution of thermal power units and optimize model parameters; In step (5), fine-tuning uses the AdamW optimizer and the mean squared error (MSE) loss function, combined with cosine annealing learning rate scheduling, mixed precision training and gradient clipping.

[0024] like Figure 2 The figure shown is a performance comparison chart of the multivariate time series prediction baseline models MSE and MAE.

[0025] (6) The prediction reasoning is completed in an autoregressive manner, and the interpretability analysis of the model decision is achieved through the integral gradient method.

[0026] In step (6), the integral gradient is based on the all-zero matrix to calculate the attribution matrix of the input features to the prediction results, and to visualize the contribution of time and variable dimensions.

[0027] This invention also provides a multivariate time series prediction system for thermal power units based on a fine-tuned large model, which includes: The data preprocessing module is used to perform noise reduction and formal definition of time-series data of thermal power units; The lexicalization module is used to implement patching and linear projection of time-series signals; The model building module is used to build the Timer-XL model, achieving attention decoupling and dual positional encoding; The fine-tuning module is used to perform end-to-end full-sample fine-tuning; The reasoning and interpretation module is used for autoregressive prediction and integral gradient interpretability analysis.

[0028] In one embodiment of the present invention, optionally, the Timer-XL model is an 8-layer pure decoder architecture, the hidden layer dimension of the Timer-XL model is 1024, and the feedforward network dimension is 2048. Figure 3 Heatmap of attribution analysis for multivariate prediction features of the Timer-XL model. Figure 4 Heatmap of the intervariate attribution matrix for the Timer-XL model.

[0029] Second embodiment: Step 1: Data Preprocessing and Formal Definition Time series data of five key variables, including temperature, pressure, and flow rate, of thermal power units were collected. After VMD-PE-wavelet adaptive denoising, a two-dimensional observation matrix X∈R^(L×V) was constructed. The historical window was set to L=2880 time steps and the prediction horizon was set to H=96 time steps. The prediction task was defined as: predicting Y_(L+1:L+H) based on X_(1:L).

[0030] Step 2: Temporal word dichotomy Set the patch length P=96 and the step size S=96 (non-overlapping segmentation) to divide the univariate sequence into N_P=30 patches; map each patch to a d_model=1024-dimensional word through linear projection to complete the conversion of continuous signal to discrete word.

[0031] Step 3: Multivariate NTP Modeling The two-dimensional word sequence is flattened in a time-first manner, with word index k=(i-1)V+m; causal masking constraints are adopted, and the model only relies on historical words to predict all variable words in the next time step. The optimization objective is to minimize the log-likelihood error between the predicted words and the true words.

[0032] Step 4: Timer-XL Model Setup The model uses an 8-layer decoder, with a feedforward network dimension D_ff=2048; 1. Construct a temporal causal mask matrix T∈R^(T×T) (all 1s in the lower triangular region) and a variable dependency matrix C∈R^(V×V) (all 1s). 2. Calculate the joint mask M=C using the Kronecker product. T, to achieve spatiotemporal attention decoupling; 3. RoPE encoding is used in the time dimension, and learnable parameters u and v are introduced in the variable dimension to complete dual positional encoding.

[0033] Step 5: Model fine-tuning Initialize pre-trained weights using the AdamW optimizer with an initial learning rate η = 5 × 10^-6 and training epochs E = 10. Use cosine annealing to decay the learning rate, with a batch size of 18. Combine mixed precision training and gradient clipping to complete the fine-tuning of data specific to thermal power units.

[0034] Step 6: Reasoning and Interpretability Analysis 1. Inference: Input the historical sequence of the test set, generate 96-step prediction results through autoregression, and output the predicted values ​​of physical quantities after inverse normalization; 2. Interpretability: Using the all-zero matrix as a baseline, the attribution matrix is ​​calculated through integral gradient, and the time steps and variable contributions are visualized to verify the model's high sensitivity to key features such as recent time series and inlet steam temperature.

[0035] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0036] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for multivariate time-series forecasting of thermal power units based on a fine-tuned large model, characterized in that, Includes the following steps: (1) Adaptive noise reduction is performed on the multivariate original time series data of thermal power units, and the historical observation matrix and prediction target sequence are formally defined to transform the prediction task into sequence-to-sequence regression modeling. (2) The timing signal is processed by patching block technology to convert the continuous numerical signal into a discrete timing lexical sequence, thereby reducing the sequence length; (3) Based on the pure decoder architecture, a multivariate sub-element prediction paradigm is constructed, and the two-dimensional spatiotemporal data is flattened with a time-first strategy. Causal masking is used to ensure the unidirectional flow of information in the time dimension. (4) Build the Timer-XL model, decouple global attention into temporal causal attention and variable fully connected attention through Kronecker product, and complete spatiotemporal dependency modeling by combining dual positional encoding; (5) An end-to-end full-sample fine-tuning strategy is adopted to adapt to the data distribution of thermal power units and optimize model parameters; (6) The prediction reasoning is completed in an autoregressive manner, and the interpretability analysis of the model decision is achieved through the integral gradient method.

2. The method according to claim 1, characterized in that, In step (1), the raw time series data includes temperature, pressure and flow rate.

3. The method according to claim 1, characterized in that, In step (1), VMD-PE-wavelet is used for adaptive noise reduction.

4. The method according to claim 1, characterized in that, In step (2), non-overlapping segmentation is used for lexicalization. The patch length is P=96. The patch is mapped to d_model dimensional lexicals through linear projection, and the sequence length is compressed from L to L / P.

5. The method according to claim 1, characterized in that, In step (4), the joint mask matrix is ​​calculated by the Kronecker product of the variable dependency matrix and the temporal causality mask matrix. The time dimension is encoded by rotational position, and the variable dimension introduces learnable scalar parameters u and v.

6. The method according to claim 1, characterized in that, In step (5), the fine-tuning uses the AdamW optimizer and the mean squared error (MSE) loss function, combined with cosine annealing learning rate scheduling, mixed precision training and gradient clipping.

7. The method according to claim 1, characterized in that, In step (6), the integral gradient is based on the all-zero matrix to calculate the attribution matrix of the input features to the prediction results, and to visualize the contribution of time and variable dimensions.

8. A multivariate time-series prediction system for thermal power units based on a fine-tuned large model, characterized in that, include: The data preprocessing module is used to perform noise reduction and formal definition of time-series data of thermal power units; The lexicalization module is used to implement patching and linear projection of time-series signals; The model building module is used to build the Timer-XL model, achieving attention decoupling and dual positional encoding; The fine-tuning module is used to perform end-to-end full-sample fine-tuning; The reasoning and interpretation module is used for autoregressive prediction and integral gradient interpretability analysis.

9. The system according to claim 8, characterized in that, The Timer-XL model is an 8-layer pure decoder architecture.

10. The system according to claim 9, characterized in that, The Timer-XL model has a hidden layer dimension of 1024 and a feedforward network dimension of 2048.

Images