Non-linear causal enhanced large model time sequence prediction method and non-linear causal enhanced large model time sequence prediction device

Abstract

The invention provides a nonlinear causal enhanced large model time sequence prediction method and device, and the method comprises the steps: obtaining multivariable historical time sequence data of a to-be-predicted object in a time window, and splicing the multivariable historical time sequence data with a prediction placeholder representing a future time step, thereby obtaining a prediction input sequence; inputting the input sequence into a joint representation generator comprising a convolution encoder and a multi-layer residual vector quantizer to obtain discrete joint representation adaptive to a pre-training large model; and sending the discrete joint representation into a pre-trained large model to obtain a first prediction result, learning nonlinear Granger causal weights between variables and lag steps from historical data based on a convolutional network, and performing residual correction on the first prediction result to obtain a multivariable time sequence prediction result. According to the technical scheme, collaborative modeling of time dependence and the nonlinear causal relationship is achieved, and the accuracy and interpretability of multivariable time sequence prediction are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method and apparatus for time series prediction of large models with nonlinear causal enhancement. Background Technology

[0002] Multivariate time series forecasting, as a core cornerstone of intelligent decision-making systems, plays an irreplaceable role in key areas such as industrial operation and maintenance, energy dispatching, traffic management, and financial risk control. This technology analyzes and models high-dimensional historical data to predict future evolution trends, providing crucial information for system optimization, resource allocation, and risk mitigation. Existing multivariate time series forecasting methods mainly fall into two categories: those based on statistical learning and those based on deep learning. Statistical learning-based methods struggle to effectively characterize complex nonlinear time series features, while deep learning-based methods, although possessing strong feature extraction capabilities, typically rely on large-scale data training from scratch, exhibiting limited few-sample learning and cross-domain generalization abilities, making them unsuitable for complex real-world scenarios with limited or varying data distributions.

[0003] Meanwhile, large-scale models excel in multimodal tasks, possessing rich general knowledge accumulation, powerful sequence modeling capabilities, and excellent few-shot and zero-shot generalization abilities, providing new technical approaches for multivariate time series prediction in complex scenarios. However, the pre-training of existing large-scale models is mainly based on one-dimensional natural language text, modeling only the dependencies between positions in the sequence, lacking the ability to explicitly model the relationships between multiple sequences or multiple variables. In multivariate time series, different variables often exhibit nonlinear, dynamic, and high-order coupling relationships; these inter-variable dependencies are important prediction clues beyond temporal information. Ignoring these dependencies in prediction will limit the model's comprehensive representation capabilities and prediction accuracy.

[0004] Therefore, there is an urgent need for a multivariate time series prediction method that can introduce nonlinear causal information within a unified framework and work in tandem with large-scale time series modeling capabilities, in order to improve the modeling quality of time dependence and the interaction between variables, and enhance the accuracy, stability, and causal interpretability of multivariate time series prediction. Summary of the Invention

[0005] This invention provides a nonlinear causal enhancement method for large-scale model time series prediction. The method acquires multivariate historical time series data of the object to be predicted within a target time window. This historical time series data is concatenated with prediction placeholders representing future time steps to be predicted to construct a multivariate time series prediction input sequence. This input sequence is then subjected to multi-scale convolutional encoding and layer-by-layer residual quantization by a multivariate time series joint representation generator, which includes a multi-layer causal convolutional encoder and a multi-layer residual vector quantizer, to generate a multivariate time series discrete joint representation adapted to a pre-trained large model. Finally, this discrete joint representation is input into... A lightweight, adaptable pre-trained large model obtains the first prediction result and uses a convolutional network to learn the nonlinear Granger causal weights between variables and their lag steps from multivariate historical time series data. The first prediction result is then corrected for causal constraint residuals to obtain multivariate time series prediction results. This achieves collaborative modeling of time-dependent features and nonlinear causal relationships between variables, improving the accuracy, stability, and causal interpretability of multivariate time series prediction under complex conditions. It also solves the problems of existing multivariate time series prediction methods, such as insufficient characterization of multi-scale nonlinear coupling relationships, weak cross-scenario generalization ability, and limited causal interpretability.

[0006] A first aspect of the present invention provides a large-scale time series prediction method for nonlinear causal enhancement, the method comprising: Obtain multivariate historical time series data of the object to be predicted within the target time window, and concatenate the historical time series data with prediction placeholders representing future time steps to be predicted to obtain a multivariate time series prediction input sequence. The multivariate temporal prediction input sequence is input into a pre-trained multivariate temporal joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer, and a multivariate temporal discrete joint representation sequence with a hierarchical structure is obtained through residual vector quantization. The multivariate time-series discrete joint representation sequence is input into a pre-trained large model to obtain the first multivariate time-series prediction result corresponding to the future time step to be predicted. Each variable in the multivariate historical time series data is taken as the target variable, and the target variables are reconstructed using multiple independent Granger discovery convolutional networks. The input of the convolutional network is the historical values ​​of each variable in the multivariate historical time series data at multiple lags. A causal weight matrix is ​​obtained based on the convolutional kernel weights during the reconstruction process to characterize the nonlinear influence of each variable on the target variable at multiple lags. The first prediction result is then corrected for residuals based on the causal weight matrix to obtain the multivariate time series prediction result.

[0007] By employing the above scheme, this invention provides a nonlinear causal enhancement method for large-scale model time series prediction. This method generates a multivariate joint representation adapted to the large model and fine-tunes the model to fully utilize its pre-training capabilities, achieving high-precision and generalizable multivariate time series prediction. Simultaneously, a nonlinear Granger causality correction module is introduced to mine causal relationships between key variables from historical data, optimizing the large model's prediction results with causal consistency constraints, further improving the accuracy, stability, and interpretability of the predictions. This invention enables two-dimensional modeling of both the time and variable dimensions, effectively enhancing the ability of large-scale model-based methods to characterize complex time series dependencies and variable coupling relationships, providing reliable support for intelligent optimization decision-making based on multivariate time series prediction.

[0008] In some embodiments of the present invention, the convolutional encoder in the multivariate temporal joint representation generator is a multi-layer causal convolutional structure. Each convolutional layer is connected to the input at the current time and the input at the historical time in the time dimension. By setting different convolutional kernel sizes and / or dilatation rates, the multivariate temporal prediction input sequence is subjected to multi-scale convolutional encoding, while extracting local temporal features and cross-scale temporal features.

[0009] In some embodiments of the present invention, the multilayer residual vector quantizer includes a first-layer codebook and at least one layer of residual codebook. The first-layer codebook is configured to perform vector quantization on the continuous joint representation output by the convolutional encoder to obtain a first-layer quantization vector and a first-layer residual vector. Each residual codebook sequentially performs vector quantization on the previous layer residual vector and accumulates it with the previous layer quantization vector to generate a multivariate temporal discrete joint representation sequence with a hierarchical structure composed of multilayer ordered codebook indices.

[0010] In some embodiments of the present invention, the multivariate temporal joint representation generator is pre-trained by introducing a decoder corresponding to the convolutional encoder. During pre-training, the multivariate temporal discrete joint representation sequence is input into the decoder to reconstruct the multivariate temporal series. Based on the reconstruction error between the reconstructed multivariate temporal series and the original multivariate historical temporal series data, as well as the weighted sum of the residual vector quantization errors of each layer, the parameters of the convolutional encoder and the multilayer residual vector quantizer are jointly optimized.

[0011] In some embodiments of the present invention, the step of inputting the multivariate time-series discrete joint representation sequence into a pre-trained large model to obtain a first multivariate time-series prediction result corresponding to a future time step to be predicted includes: The multivariate temporal discrete joint representation sequence is mapped to an embedding vector and then superimposed with a position embedding. This vector is then input into the pre-trained large model. A mask reconstruction task is performed on the embedding representing the predicted placeholder position, and the multivariate temporal first prediction result corresponding to the predicted placeholder position is output. When adapting the large model for the temporal prediction task, the parameters of the self-attention layer and the feedforward network layer remain unchanged, while the parameters of some embedding layers and normalization layers are updated.

[0012] In some embodiments of the present invention, the target variable is reconstructed univariate temporally using multiple independent Granger discovery convolutional networks, including: The multivariate historical time series data is regarded as being composed of multiple univariate time series. A corresponding single-layer convolutional network is constructed for each target variable. The historical values ​​of each input variable at multiple lag steps are stacked according to the channel dimension and input into the single-layer convolutional network to reconstruct the univariate time series of the target variable. Based on the convolution kernel weights in the reconstruction process, the input variables and lag steps are arranged and summarized to obtain a causal weight matrix that characterizes the nonlinear influence of each lagged input variable on the target variable.

[0013] In some embodiments of the present invention, the residual correction of the first prediction result based on the causal weight matrix includes: The weights of the convolution kernels of the residual correction convolutional network are initialized based on the causal weight matrix. In each convolution kernel, only the weights with larger absolute values ​​in the corresponding causal weight matrix are retained, and the remaining weights are reset to zero, resulting in a sparse residual correction convolutional network. The first prediction result of the multivariate time series is divided into multiple univariate prediction sequences according to the variables. The prediction values ​​of each univariate prediction sequence at multiple lag steps are input into the corresponding residual correction convolutional network to obtain the residual correction amount of each target variable. The residual correction amount is then added to the first prediction result of the multivariate time series to obtain the multivariate time series prediction result.

[0014] In some embodiments of the present invention, when jointly training the large model and the residual correction based on the causal weight matrix, the training target is the weighted sum of the prediction error between the multivariate time series prediction result and the true multivariate time series, the reconstruction error of the multivariate time series joint representation generator, and the regularization term of the causal weight matrix.

[0015] In some embodiments of the present invention, the object to be predicted is a wind turbine generator set, and the multivariate historical time series data includes at least one of the following: output power, wind speed, wind direction, ambient temperature, nacelle temperature, yaw angle, and blade twist angle of the wind turbine generator set under a preset sampling period. The multivariate time series prediction results are used to predict the output power of the wind turbine generator set within the target time window.

[0016] Compared with existing technologies, the advantages of this invention are as follows: This invention constructs a unified multivariate time-series prediction input sequence by concatenating multivariate historical time-series data of the object to be predicted within the target time window with prediction placeholders representing future prediction time steps. This achieves a target modeling method that normalizes different prediction tasks to a standardized mask prediction input space, facilitating integration and reuse with pre-trained large models. Furthermore, by employing a multivariate time-series joint representation generator containing multi-layer causal convolutional encoders and multi-layer residual vector quantizers, it achieves compressed expression and discrete embedding of multi-scale time dependencies and high-dimensional state information, reducing the computational complexity of subsequent modeling while preserving key temporal structural features. Finally, by inputting the discrete joint representation sequence into a lightweight adapted pre-trained large model... Based on mask reconstruction, the first prediction result of multivariate time series is obtained, which fully utilizes the modeling capabilities of large models for long sequences and the generalization ability with few samples, improving the basic prediction accuracy and cross-scenario stability. Furthermore, a one-dimensional convolutional network is used to reconstruct the univariate time series of each target variable from the multivariate historical time series data, thereby obtaining a Granger causal weight matrix characterizing the nonlinear influence of each input variable and its multiple lag steps on the target variable. Then, a residual correction convolutional network is constructed based on this causal weight matrix to correct the causal constraint residuals of the first prediction result. This achieves collaborative modeling of time-dependent features and nonlinear causal relationships between variables, and causal consistency correction of the prediction results, significantly improving the accuracy, stability, and causal interpretability of multivariate time series prediction under complex working conditions and changing scenarios. The technical solution of this invention realizes causal enhancement of large-model-driven prediction for multivariate time series, improving the reliability and usability of multivariate time series prediction in engineering applications.

[0017] A second aspect of the present invention provides a large-model time series prediction system with nonlinear causal enhancement, comprising: Data input and preprocessing module: used to acquire multivariate historical time series data of the object to be predicted within the target time window, and concatenate the historical time series data with prediction placeholders representing future time steps to be predicted to generate a multivariate time series prediction input sequence; Multivariate temporal joint representation generation module: used to receive the multivariate temporal prediction input sequence and generate a multivariate temporal discrete joint representation sequence with a hierarchical structure through a pre-trained joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer; Large Model Time Series Prediction Module: This module is used to input the multivariate time series discrete joint representation sequence into a pre-trained large model, perform temporal dependency modeling, and output the first multivariate time series prediction result corresponding to the future time step to be predicted. The nonlinear Granger causality correction module is used to take each variable in the multivariate historical time series data as a target variable, and reconstruct the target variable into a single variable time series using multiple independent Granger discovery convolutional networks. The input to each convolutional network is the historical value of each variable in the multivariate historical time series data at multiple lags. A causal weight matrix characterizing the nonlinear influence of each variable on the target variable at multiple lags is obtained based on the convolutional kernel weights during the reconstruction process. The residual of the first prediction result is corrected based on the causal weight matrix to obtain the multivariate time series prediction result.

[0018] A third aspect of the present invention provides a large-model time series prediction apparatus with nonlinear causal enhancement, characterized in that the apparatus includes a computer device, the computer device including a processor and a memory, the processor storing computer instructions, and when the computer instructions are executed, the apparatus implements the large-model time series prediction method with nonlinear causal enhancement.

[0019] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0020] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] In the attached diagram: Figure 1 This is a flowchart illustrating a large-model time series prediction method with nonlinear causal enhancement provided in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of training a multivariate temporal joint representation generator based on residual quantization, provided in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of a nonlinear Granger causality correction module provided in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the Granger causal enhancement fine-tuning large model prediction process provided in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of a large-model time series prediction system with nonlinear causal enhancement provided in an embodiment of the present invention.

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0029] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0030] Figure 1 This is a flowchart illustrating a large-model time series prediction method with nonlinear causal enhancement provided in an embodiment of the present invention.

[0031] Example 1, as Figure 1 As shown, this invention provides a nonlinear causal enhancement method for large model time series prediction, the method comprising the following steps: S1. Obtain multivariate historical time series data of the object to be predicted within the target time window, and concatenate the historical time series data with a prediction placeholder representing the future time step to be predicted to obtain a multivariate time series prediction input sequence. S2. Input the multivariate temporal prediction input sequence into a pre-trained multivariate temporal joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer, and obtain a multivariate temporal discrete joint representation sequence with a hierarchical structure through residual vector quantization. S3. Input the multivariate time series discrete joint representation sequence into the pre-trained large model to obtain the first multivariate time series prediction result corresponding to the future time step to be predicted. S4. Each variable in the multivariate historical time series data is taken as the target variable, and the target variable is reconstructed into a univariate time series using multiple independent Granger discovery convolutional networks; wherein, the input of the convolutional network is the historical value of each variable in the multivariate historical time series data at multiple lag steps; a causal weight matrix is ​​obtained based on the convolutional kernel weights in the reconstruction process to characterize the nonlinear influence of each variable on the target variable at multiple lag steps; the first prediction result is corrected for residuals based on the causal weight matrix to obtain the multivariate time series prediction result.

[0032] Using the above scheme, the nonlinear causal enhancement large-model time series prediction method of the present invention acquires multivariate historical time series data of the object to be predicted within the target time window, and concatenates it with prediction placeholders representing future time steps to be predicted to construct a multivariate time series prediction input sequence. This realizes the unified representation of future time step prediction tasks as a standardized mask prediction input form, which is convenient for docking and reuse with pre-trained large models. By inputting the prediction input sequence into a multivariate time series joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer, a multivariate time series discrete joint representation sequence with a hierarchical structure is generated. This achieves compressed expression and discrete embedding of multi-scale time dependencies and high-dimensional state information, reducing the computational complexity of subsequent modeling while retaining the main time series structural features. By inputting the multivariate time series discrete joint representation sequence into a pre-trained large model, the first prediction result of the multivariate time series corresponding to the future time step to be predicted is obtained, thereby improving the basic prediction level by utilizing the existing long sequence modeling and few-sample generalization capabilities of the large model. Furthermore, the univariate time series of each target variable are reconstructed through a convolutional network, and a causal weight matrix characterizing the nonlinear influence of each lagged input variable on the target variable is obtained based on the convolutional kernel weights in the reconstruction process. The first prediction result is then corrected for residuals based on the causal weight matrix, thereby realizing the explicit utilization of the multi-lagged nonlinear Granger causal relationship between variables and the causal consistency correction of the prediction result, significantly improving the accuracy, stability, and causal interpretability of multivariate time series prediction under complex working conditions and changing scenarios.

[0033] In some embodiments of the present invention, the convolutional encoder in the multivariate temporal joint representation generator is a multi-layer causal convolutional structure. Each convolutional layer is connected only to the input at the current time and the input at the historical time in the time dimension. By setting different convolutional kernel sizes and / or dilatation rates, the multivariate temporal prediction input sequence is subjected to multi-scale convolutional encoding to simultaneously extract local temporal features and cross-scale temporal features.

[0034] Specifically, in this embodiment, the multivariate temporal joint representation generator is constructed using residual vector quantization. It generates high-fidelity multivariate temporal discrete embedding representations adapted to large models through compression and quantization. The generator consists of a convolutional encoder and a residual vector quantizer. During the training phase, a reconstruction task is performed using a decoder with the opposite structure to the encoder to optimize the quality of the joint representation. The training process is as follows: Figure 2 As shown.

[0035] Convolutional encoder Composed of multiple layers of dilated causal convolutions and downsampling layers, this encoding method captures both local and cross-scale temporal features simultaneously through multi-scale convolutional structures, preserving key dynamic information while compressing the temporal length. Compared to the Patch embedding method, this encoding approach includes richer temporal context in each embedding. For a given set of... There are variables, with a length of . Multivariate timing , Map it to a preliminary multivariate joint representation that fits the large model. The output timing length is That is, corresponding generation Each embedding, the variable dimension is projected onto the hidden dimension of the large model. A consistent representation space.

[0036] In some embodiments of the present invention, the multilayer residual vector quantizer includes a first-layer codebook and at least one layer of residual codebook. The first-layer codebook is configured to perform vector quantization on the continuous joint representation output by the convolutional encoder to obtain a first-layer quantization vector and a first-layer residual vector. Each residual codebook sequentially performs vector quantization on the previous layer residual vector and accumulates it with the previous layer quantization vector to generate a multivariate temporal discrete joint representation sequence with a hierarchical structure composed of multilayer ordered codebook indices.

[0037] Specifically, in this embodiment, a Residual Vector Quantizer (RVQ) is used to discretize the multivariate time series representation. RVQ re-quantizes the quantized residuals and can iterate through multiple layers. Thus, the residual embedding at each layer is mapped to the nearest codeword in the codebook, forming a coarse-to-fine discrete approximation. Given a codebook... and quantization depth RVQ represents the encoded vector as Layer-ordered codeword sequence:

[0038] From the 0th layer residual embedding Initially, RVQ quantizes the residuals of each layer iteratively:

[0039] in It is the first Residual quantization embedding of a layer, by embedding each residual of the previous layer Mapped to its nearest codeword Obtain. Define the first The quantitative characterization of the layer is the first The cumulative sum of layer residual quantization embeddings, RVQ approximates the original vector representation layer by layer in a coarse-to-fine manner. As the quantization depth increases, the approximation becomes increasingly accurate, achieving high-fidelity multivariate time-series quantization representation adapted to large models. In some embodiments of the present invention, the multivariate temporal joint representation generator is pre-trained by introducing a decoder corresponding to the convolutional encoder. During pre-training, the multivariate temporal discrete joint representation sequence is input into the decoder to reconstruct the multivariate temporal series. Based on the reconstruction error between the reconstructed multivariate temporal series and the original multivariate historical temporal series data, as well as the weighted sum of the residual vector quantization errors of each layer, the parameters of the convolutional encoder and the multilayer residual vector quantizer are jointly optimized.

[0040] Specifically, in this embodiment, the decoder corresponds to the convolutional encoder. Follow with encoder The opposite structure uses transposed convolution to quantize the representation. Mapped to a reconstructed multivariate time series .

[0041] The overall loss function in the first stage consists of reconstruction loss. and RVQ constraint loss Composition, and joint optimization through weighted summation:

[0042]

[0043]

[0044] in, This represents the sum of quantization errors across all layers. This indicates that the gradient (stop-gradient) operation is stopped. These are the weighting coefficients.

[0045] In some embodiments of the present invention, the step of inputting the multivariate time-series discrete joint representation sequence into a pre-trained large model to obtain a first multivariate time-series prediction result corresponding to a future time step to be predicted includes: The multivariate temporal discrete joint representation sequence is mapped to an embedding vector and then superimposed with a position embedding. This vector is then input into the pre-trained large model. A mask reconstruction task is performed on the embedding representing the predicted placeholder position, and the multivariate temporal first prediction result corresponding to the predicted placeholder position is output. When adapting the large model for the temporal prediction task, the parameters of the self-attention layer and the feedforward network layer remain unchanged, and only some embedding layer and normalization layer parameters are updated.

[0046] Specifically, in this embodiment, a masked token reconstruction paradigm is used to achieve multivariate temporal prediction for large-scale models. First, a pre-trained multivariate temporal joint representation generator is used to generate a quantized representation of the sequence after concatenating the input temporal sequence with all-zero prediction placeholders, and this representation is then input into the large-scale model for temporal modeling. During fine-tuning, only the position embedding and layer normalization parameters are updated, while the attention layer and feedforward network layer are frozen. Finally, the large-scale model prediction output is obtained through the output linear layer. Where O represents the prediction length. Since multivariate information is mixed in large-scale temporal modeling and the dependencies between variables are not explicitly captured, a nonlinear Granger causality correction module is further proposed to clarify the Granger causal relationships between modeling variables and utilize this information to improve large-scale model predictions.

[0047] Granger causality is a classic causal discovery method used to quantify the contribution of historical information from one time series to predicting the future evolution of another time series. In traditional Granger causality estimation models, time... Multivariate time series, , can be represented as past A linear combination of lag steps:

[0048] in Describe the lag step Impact on future temporal evolution, The noise is zero-mean. However, these models assume that the time series follows linear dynamics and struggle to handle nonlinear dependencies between variables.

[0049] The nonlinear Granger causality correction module uses a convolutional neural network to reproduce the Granger causality principle, enabling nonlinear causal modeling of multivariate time series and providing interpretability. For example... Figure 3 As shown, this module consists of two steps: first, discovering nonlinear Granger causal relationships from historical data; and second, using the discovered causal information to correct large model predictions.

[0050] In some embodiments of the present invention, the step of performing univariate temporal reconstruction of the target variable through multiple independent Granger discovery convolutional networks includes: The multivariate historical time series data is regarded as being composed of multiple univariate time series. A corresponding single-layer convolutional network is constructed for each target variable. The historical values ​​of each input variable at multiple lag steps are stacked according to the channel dimension and input into the single-layer convolutional network to reconstruct the univariate time series of the target variable. Based on the convolution kernel weights in the reconstruction process, the input variables and lag steps are arranged and summarized to obtain a causal weight matrix that characterizes the nonlinear influence of each lagged input variable on the target variable.

[0051] Specifically, in this embodiment, to clarify the causal effect of different variable sequences on the target sequence at each lag step, this module will perform multivariate time series analysis. Treat as multiple univariate sequences And each univariate sequence is reconstructed through multiple independent single-layer Granger discovery convolutional networks. :

[0052] in Introducing nonlinearity, and These represent the kernel weights and biases, respectively. The reconstruction sequence is minimized. With the original sequence The mean square error can be optimized. To approximate the underlying Granger causal structure. The causal effects of each lagged input variable on the target sequence are summarized and can be viewed as a Granger causal weight matrix. The lagged input variables refer to the time-lagging variable observations extracted from multivariate historical time series data.

[0053] In some embodiments of the present invention, the residual correction of the first prediction result based on the causal weight matrix includes: The weights of the convolution kernels of the residual correction convolutional network are initialized based on the causal weight matrix. In each convolution kernel, only the weights with larger absolute values ​​in the corresponding causal weight matrix are retained, and the remaining weights are reset to zero to obtain a sparse residual correction convolutional network. The first prediction result of the multivariate time series is divided into multiple univariate prediction sequences according to the variables. The prediction values ​​of each univariate prediction sequence at multiple lag steps are input into the corresponding residual correction convolutional network to obtain the residual correction amount of each target variable. The residual correction amount is then added to the first prediction result of the multivariate time series to obtain the multivariate time series prediction result.

[0054] Specifically, in this embodiment, in the first stage, the... Causal information is used to correct the prediction output of large models. .Will Decomposed into multiple univariate sequences The corrected sequence is generated using a separate set of single-layer Granger correction convolutional networks in a residual manner. Granger causality correction of convolutional kernel weights in convolutional networks Depend on Derivatives, each Only keep The largest of them Only one value is set, and the rest are set to zero, thus ensuring that only the lagged sequences with the most significant causal impact on the target variable are used for correction, avoiding overfitting. Variable At any moment The corrected output is:

[0055] in This is a trainable bias. Concatenating all the univariate corrected outputs yields the Granger causality corrected prediction output. The final prediction result is obtained through truncation operation. .

[0056] The Granger causality correction module introduces interpretable Granger causality constraints into the prediction output of large models, thereby improving the accuracy and interpretability of multivariate time series predictions.

[0057] In some embodiments of the present invention, when jointly training the large model and the residual correction based on the causal weight matrix, the training objective is to use the prediction error between the multivariate time series prediction result and the true multivariate time series, the reconstruction error of the multivariate time series joint representation generator, and the weighted sum of the regularization terms of the causal weight matrix, so as to enhance the characterization of key causal paths and suppress unimportant causal weights.

[0058] The second stage involves Granger causality enhancement fine-tuning of the large model prediction process, as follows: Figure 4 As shown, its training objective function is:

[0059] The third term is a penalty term, which uses the Frobenius norm to highlight important Granger causal weights and suppress unimportant weights. and This is a hyperparameter.

[0060] In some embodiments of the present invention, the object to be predicted is a wind turbine generator set, and the multivariate historical time series data includes at least one of the following: output power, wind speed, wind direction, ambient temperature, nacelle temperature, yaw angle, and blade twist angle of the wind turbine generator set under a preset sampling period. The multivariate time series prediction results are used to predict the output power of the wind turbine generator set within the target time window.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows: The technical solution of this invention constructs a unified multivariate time series prediction input sequence by concatenating the multivariate historical time series data of the object to be predicted within the target time window with prediction placeholders representing future prediction time steps. This achieves a target modeling method that normalizes different prediction tasks to a standardized mask prediction input space, facilitating integration and reuse with pre-trained large models. By employing a multivariate time series joint representation generator containing multi-layer causal convolutional encoders and multi-layer residual vector quantizers, it achieves compressed expression and discrete embedding of multi-scale time dependencies and high-dimensional state information, reducing the computational complexity of subsequent modeling while preserving key time series structural features. Finally, by inputting the discrete joint representation sequence into a lightweight, adapted pre-trained large model and reconstructing it based on a mask, the solution achieves a unified multivariate time series prediction input sequence. Obtaining the first prediction result for multivariate time series data fully utilizes the modeling capabilities of large models for long sequences and the generalization ability with few samples, improving the basic prediction accuracy and cross-scenario stability. Furthermore, a one-dimensional convolutional network reconstructs the univariate time series of each target variable from historical multivariate time series data, thereby obtaining a Granger causal weight matrix characterizing the nonlinear influence of each input variable and its multiple lags on the target variable. Based on this causal weight matrix, a residual correction convolutional network is constructed to correct the causal constraints of the first prediction result. This achieves collaborative modeling of time-dependent features and nonlinear causal relationships between variables, and causal consistency correction of the prediction results, significantly improving the accuracy, stability, and causal interpretability of multivariate time series prediction under complex working conditions and changing scenarios. The technical solution of this invention realizes causal enhancement-driven large-model prediction for multivariate time series data, improving the reliability and usability of multivariate time series prediction in engineering applications.

[0062] To verify the effectiveness of the technical solution of this invention, actual operating and maintenance data of a 3.5 MW wind turbine generator set in a domestic wind farm were selected for model verification. The wind turbine generator set consists of a rotor, transmission mechanism, yaw and braking device, generator, nacelle, tower, etc. It captures wind energy through the rotor and transfers mechanical energy to the generator via the transmission system, achieving efficient conversion of wind energy into electrical energy. With the continuous increase in the proportion of wind power in the energy structure, the fluctuation of wind turbine generator set power output poses a challenge to the stable operation of the power grid. Therefore, accurate prediction of wind turbine generator set output power has become a crucial link in ensuring power grid security and optimizing dispatch.

[0063] The wind turbine's operational data is automatically collected and stored by the Supervisory Control and Data Acquisition (SCADA) system at a sampling period of 10 minutes. The data includes the turbine's output power, external meteorological parameters (such as wind speed, wind direction, and ambient temperature), and internal operating parameters (such as nacelle temperature, yaw angle, and blade twist angle). To ensure data quality, the raw data is first cleaned and preprocessed to remove abnormal and missing records, such as samples with wind speeds less than 0 m / s or when the turbine is shut down. Subsequently, polynomial interpolation is used to complete the few missing values, and a digital low-pass filter is used to smooth high-frequency noise, thereby obtaining a continuous, stable, and physically consistent multivariate time-series dataset.

[0064] On this dataset, a multivariate time series prediction task is constructed by using the unit output power as the prediction target and combining multidimensional operation and environmental characteristics.

[0065] The model is first pre-trained on a reconstruction task using a multivariate joint representation generator based on residual quantization, which is used to generate high-quality joint embedding representations that are adapted to large models.

[0066] In the prediction phase, the time step to be predicted is set as the mask position, and the multivariate time series with prediction placeholders is input to the representation generator to generate a discretized joint embedding representation.

[0067] The embedding is then fed into a finely tuned large model, which leverages its powerful sequence modeling capabilities to perform temporal dependency modeling and predict the output.

[0068] To further improve the accuracy and interpretability of predictions, the nonlinear Granger causality correction module mines key causal dependencies between variables from historical data and corrects the causal consistency of large model predictions, obtaining the final prediction results that integrate nonlinear causal knowledge.

[0069] The prediction results are evaluated using two metrics, MSE and MAE. Compared with classic methods based on statistical learning and deep learning, this invention significantly improves the prediction accuracy and stability in wind turbine power prediction tasks and can better capture the dynamic changes under complex wind conditions.

[0070] Table 1 Comparison of Multivariate Prediction Model Results for Wind Turbine Output Power and Influencing Factors

[0071] Figure 4 This is a flowchart illustrating a large-model time-series prediction system with nonlinear causal enhancement provided in an embodiment of the present invention.

[0072] Example 2, as Figure 5As shown, the present invention also provides a large model time series prediction system with nonlinear causal enhancement, comprising: a data input and preprocessing module S11, a multivariate time series joint representation generation module S12, a large model time series prediction module S13, and a nonlinear Granger causality correction module S14.

[0073] Data input and preprocessing module: used to acquire multivariate historical time series data of the object to be predicted within the target time window, and concatenate the historical time series data with prediction placeholders representing future time steps to be predicted to generate a multivariate time series prediction input sequence; Multivariate temporal joint representation generation module: used to receive the multivariate temporal prediction input sequence and generate a multivariate temporal discrete joint representation sequence with a hierarchical structure through a pre-trained joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer; Large Model Time Series Prediction Module: This module is used to input the multivariate time series discrete joint representation sequence into a pre-trained large model, perform temporal dependency modeling, and output the first multivariate time series prediction result corresponding to the future time step to be predicted. The nonlinear Granger causality correction module is used to take each variable in the multivariate historical time series data as a target variable, and reconstruct the target variable into a single variable time series using multiple independent Granger discovery convolutional networks. The input to each convolutional network is the historical value of each variable in the multivariate historical time series data at multiple lags. A causal weight matrix characterizing the nonlinear influence of each variable on the target variable at multiple lags is obtained based on the convolutional kernel weights during the reconstruction process. The residual of the first prediction result is corrected based on the causal weight matrix to obtain the multivariate time series prediction result.

[0074] Example 3: The present invention also provides a large model time series prediction device with nonlinear causal enhancement. The device includes a computer device, which includes a processor and a memory. The processor stores computer instructions. When the computer instructions are executed, the device implements the large model time series prediction method with nonlinear causal enhancement.

[0075] Example 4, as Figure 6 As shown, the present invention also provides an electronic device 100 for implementing a large model time series prediction method for nonlinear causal enhancement.

[0076] The electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on at least one processor 102, and at least one communication bus 104.

[0077] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the nonlinear causal enhancement large model time series prediction method described in the first aspect of the present invention by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101.

[0078] The memory 101 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0079] At least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 102 may be a microprocessor or any conventional processor. Processor 102 is the control center of electronic device 100, connecting various parts of electronic device 100 via various interfaces and lines.

[0080] The memory 101 in the electronic device 100 stores multiple instructions to implement a large model time series prediction method with nonlinear causal enhancement, and the processor 102 can execute multiple instructions to achieve the following: Obtain multivariate historical time series data of the object to be predicted within the target time window, and concatenate the historical time series data with prediction placeholders representing future time steps to be predicted to obtain a multivariate time series prediction input sequence. The multivariate temporal prediction input sequence is input into a pre-trained multivariate temporal joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer, and a multivariate temporal discrete joint representation sequence with a hierarchical structure is obtained through residual vector quantization. The multivariate time-series discrete joint representation sequence is input into a pre-trained large model to obtain the first multivariate time-series prediction result corresponding to the future time step to be predicted. Each variable in the multivariate historical time series data is taken as the target variable, and the target variables are reconstructed using multiple independent Granger discovery convolutional networks. The input of the convolutional network is the historical values ​​of each variable in the multivariate historical time series data at multiple lags. A causal weight matrix is ​​obtained based on the convolutional kernel weights during the reconstruction process to characterize the nonlinear influence of each variable on the target variable at multiple lags. The first prediction result is then corrected for residuals based on the causal weight matrix to obtain the multivariate time series prediction result.

[0081] Example 5: If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, and read-only memory (ROM).

[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A large-scale time series prediction method with nonlinear causal enhancement, characterized in that, The method includes: Obtain multivariate historical time series data of the object to be predicted within the target time window, and concatenate the historical time series data with prediction placeholders representing future time steps to be predicted to obtain a multivariate time series prediction input sequence. The multivariate temporal prediction input sequence is input into a pre-trained multivariate temporal joint representation generator including a convolutional encoder and a multi-layer residual vector quantizer, and a multivariate temporal discrete joint representation sequence with a hierarchical structure is obtained through residual vector quantization. The multivariate time-series discrete joint representation sequence is input into a pre-trained large model to obtain the first multivariate time-series prediction result corresponding to the future time step to be predicted. Each variable in the multivariate historical time series data is taken as the target variable, and the target variables are reconstructed using multiple independent Granger discovery convolutional networks. The input of the convolutional network is the historical values ​​of each variable in the multivariate historical time series data at multiple lags. A causal weight matrix is ​​obtained based on the convolutional kernel weights during the reconstruction process to characterize the nonlinear influence of each variable on the target variable at multiple lags. The first prediction result is then corrected for residuals based on the causal weight matrix to obtain the multivariate time series prediction result.

2. The large-scale time series prediction method for nonlinear causal enhancement according to claim 1, characterized in that, The convolutional encoder in the multivariate temporal joint representation generator is a multi-layer causal convolutional structure. Each convolutional layer is connected to the input at the current time and historical time in the time dimension. By setting different convolutional kernel sizes and / or dilatation rates, the multivariate temporal prediction input sequence is encoded using multi-scale convolution, while extracting local temporal features and cross-scale temporal features.

3. The large-scale time series prediction method for nonlinear causal enhancement according to claim 1, characterized in that, The multi-layer residual vector quantizer includes a first-layer codebook and at least one layer of residual codebook. The first-layer codebook is configured to perform vector quantization on the continuous joint representation output by the convolutional encoder to obtain a first-layer quantization vector and a first-layer residual vector. Each residual codebook sequentially performs vector quantization on the previous layer residual vector and accumulates it with the previous layer quantization vector to generate a multivariate temporal discrete joint representation sequence with a hierarchical structure composed of multi-layer ordered codebook indices.

4. The large-scale time series prediction method with nonlinear causal enhancement according to claim 1, characterized in that, The multivariate temporal joint representation generator is pre-trained by introducing a decoder corresponding to the convolutional encoder. During pre-training, the multivariate temporal discrete joint representation sequence is input into the decoder to reconstruct the multivariate temporal series. Based on the reconstruction error between the reconstructed multivariate temporal series and the original multivariate historical temporal series data, as well as the weighted sum of the residual vector quantization errors of each layer, the parameters of the convolutional encoder and the multilayer residual vector quantizer are jointly optimized.

5. The nonlinear causal enhancement method for large-scale model time series prediction according to any one of claims 1 to 4, characterized in that, The step of inputting the multivariate time-series discrete joint representation sequence into a pre-trained large model to obtain the first multivariate time-series prediction result corresponding to the future time step to be predicted includes: The multivariate temporal discrete joint representation sequence is mapped to an embedding vector and then superimposed with a position embedding. This vector is then input into the pre-trained large model. A mask reconstruction task is performed on the embedding representing the predicted placeholder position, and the multivariate temporal first prediction result corresponding to the predicted placeholder position is output. When adapting the large model for the temporal prediction task, the parameters of the self-attention layer and the feedforward network layer remain unchanged, while the parameters of some embedding layers and normalization layers are updated.

6. The large-scale time series prediction method with nonlinear causal enhancement according to claim 5, characterized in that, The step of reconstructing the target variable using multiple independent Granger discovery convolutional networks includes: The multivariate historical time series data is regarded as being composed of multiple univariate time series. A corresponding single-layer convolutional network is constructed for each target variable. The historical values ​​of each input variable at multiple lag steps are stacked according to the channel dimension and input into the single-layer convolutional network to reconstruct the univariate time series of the target variable. Based on the convolution kernel weights in the reconstruction process, the input variables and lag steps are arranged and summarized to obtain a causal weight matrix that characterizes the nonlinear influence of each lagged input variable on the target variable.

7. The large-model time series prediction method with nonlinear causal enhancement according to claim 6, characterized in that, The residual correction of the first prediction result based on the causal weight matrix includes: The weights of the convolution kernels of the residual correction convolutional network are initialized based on the causal weight matrix. In each convolution kernel, only the weights with larger absolute values ​​in the corresponding causal weight matrix are retained, and the remaining weights are reset to zero, resulting in a sparse residual correction convolutional network. The first prediction result of the multivariate time series is divided into multiple univariate prediction sequences according to the variables. The prediction values ​​of each univariate prediction sequence at multiple lag steps are input into the corresponding residual correction convolutional network to obtain the residual correction amount of each target variable. The residual correction amount is then added to the first prediction result of the multivariate time series to obtain the multivariate time series prediction result.

8. The large-model time series prediction method with nonlinear causal enhancement according to claim 7, characterized in that, When jointly training the large model and the residual correction based on the causal weight matrix, the training objective is the weighted sum of the prediction error between the multivariate time series prediction results and the true multivariate time series, the reconstruction error of the multivariate time series joint representation generator, and the regularization term of the causal weight matrix.

9. The large-scale time series prediction method for nonlinear causal enhancement according to claim 1, characterized in that, The object to be predicted is a wind turbine generator set. The multivariate historical time series data includes at least one of the following: output power, wind speed, wind direction, ambient temperature, nacelle temperature, yaw angle, and blade twist angle of the wind turbine generator set under a preset sampling period. The multivariate time series prediction results are used to predict the output power of the wind turbine generator set within the target time window.

10. A large-scale model time series prediction device with nonlinear causal enhancement, characterized in that, The apparatus includes a computer device, which includes a processor and a memory. The processor stores computer instructions, and when the computer instructions are executed, the apparatus implements the nonlinear causal enhancement large model time series prediction method as described in any one of claims 1 to 9.

Images