Multivariable time series prediction method and system based on hypergraph causal modeling

Abstract

The invention discloses a multivariable time sequence prediction method and system based on hypergraph causal modeling. The method comprises the steps that a historical multivariable time sequence of a target object is acquired and preprocessed; constructing a candidate hypergraph containing high-order dependency based on the preprocessed historical multivariable time sequence; based on the candidate hypergraph containing the high-order dependence, utilizing a differential causal pruning mechanism to adaptively screen effective hyperedges; based on the effective hyperedge, utilizing a hierarchical sparse aggregation mechanism to generate aggregation features fused with multi-level dependence; and performing time sequence prediction by using a bidirectional prediction decoder according to the aggregation feature fused with multi-level dependence. According to the method, accurate description and dynamic screening of inter-channel high-order dependence are realized, future time step interactive modeling is optimized, the core problems that prediction precision and expandability are difficult to consider at the same time in a high-dimensional scene, false dependence interferes prediction reliability and the like are solved, and the prediction efficiency is improved. And the accuracy and robustness of the multivariable time sequence in the long and short term prediction task are obviously improved.

Description

Technical Field

[0001] This invention relates to the field of time series forecasting technology, specifically to a multivariate time series forecasting method and system based on hypergraph causal modeling. Background Technology

[0002] In key areas such as traffic management, energy dispatch, weather forecasting, and financial analysis, multivariate time series forecasting is a core technology supporting decision-making and resource optimization. Its forecasting accuracy directly affects the operational efficiency and stability of systems within these fields. For example, traffic flow forecasting needs to consider the correlation of traffic flow changes across multiple road segments, while electricity consumption forecasting requires integrating the dynamic interaction of electricity consumption data from multiple regions. In these scenarios, the dependencies between variables (i.e., channels) are crucial for improving forecasting performance.

[0003] Traditional multivariate time series forecasting methods mainly revolve around channel modeling, forming three major paradigms: channel independence (CI), channel dependence (CD), and channel partial dependence (CP). The channel independence paradigm (such as DLinear and PatchTST) models each channel individually to pursue computational efficiency, but completely ignores the synergistic effects between channels, resulting in severe lack of robustness in scenarios with tightly coupled variables (such as meteorological data). The channel dependence paradigm (such as iTransformer and CrossFormer) treats all channels as a whole to capture global correlations, but as the number of channels increases, the computational complexity increases exponentially, facing serious scalability challenges. The channel partial dependence paradigm (such as DUET and TimeFilter) balances efficiency and expressiveness through sparse interactions or fixed groupings, but due to its reliance on static structures or coarse-grained similarity estimation, it is difficult to adapt to dynamically evolving channel relationships and cannot effectively capture higher-order dependencies and causal associations, leading to limited prediction accuracy in complex scenarios.

[0004] With the development of deep learning technology, graph-based extension methods (such as GNNs and traditional hypergraph models) have been introduced into multivariate time series prediction, attempting to characterize the structural relationships between channels through graph topology. However, traditional graph models can only capture pairwise variable interactions and cannot represent higher-order dependencies. While existing hypergraph models support multivariate association modeling, they still implicitly follow traditional channel paradigms, either indirectly connecting independent channels through common hyperedges (such as CI), or constructing fully connected hyperedges leading to computational overload (such as CD), or relying on fixed channel groupings lacking dynamic adaptability (such as CP), failing to overcome the inherent defects of traditional paradigms. At the same time, these methods generally ignore the causal relationships between channels, easily incorporating noisy associations or spurious dependencies into the modeling, leading to a decrease in the reliability of prediction results, especially in long-term predictions or high-noise datasets (such as the traffic PEMS series), where performance degradation is significant.

[0005] Furthermore, existing prediction models often employ a one-way "history → future" reasoning mode in their decoders, neglecting potential dependencies between future time steps and making it difficult to guarantee the temporal consistency of the predicted sequence. Moreover, as the input window length increases, most models are prone to attention dilution due to historical information redundancy, hindering sustained improvement in prediction performance. Therefore, designing a channel modeling paradigm that combines high-order dependency capture capabilities, causal reasoning abilities, and scalability, while optimizing future time step interaction modeling, has become a core problem urgently needing to be solved in the field of multivariate time series prediction. Summary of the Invention

[0006] To address the aforementioned shortcomings in existing technologies, this invention provides a multivariate time series prediction method and system based on hypergraph causal modeling. This solves the problems of existing technologies being unable to effectively capture higher-order dependencies and causal associations, easily incorporating noisy associations or spurious dependencies into the modeling, having insufficient reliability of prediction results, and being unable to guarantee the temporal consistency of the predicted series.

[0007] To achieve the aforementioned objectives, the present invention employs the following technical solution: a scalable multivariate time series forecasting method based on hypergraph causal modeling, comprising: Obtain the historical multivariate time series of the target object and perform preprocessing; Construct candidate hypergraphs containing higher-order dependencies based on preprocessed historical multivariate time series; Effective hyperedges are adaptively selected based on a differentiable causal pruning mechanism using candidate hypergraphs containing higher-order dependencies. Based on the effective hyperedge utilization hierarchical sparse aggregation mechanism, aggregated features that integrate multi-level dependencies are generated. Time series prediction is performed using a bidirectional predictive decoder based on the aggregated features that integrate multi-level dependencies. The historical multivariate time series of the target object is: traffic flow, speed, and traffic density of a certain road segment at multiple time steps; or electricity consumption, electricity price, power generation, and power flow data of a certain region at multiple time steps; or wind speed, precipitation, humidity, temperature, light intensity, and cloud cover of a certain region at multiple time steps.

[0008] Secondly, this invention also provides a scalable multivariate time series forecasting system based on hypergraph causal modeling, comprising: The preprocessing module is used to obtain the historical multivariate time series of the target object and perform preprocessing. The higher-order hypergraph construction module is used to construct candidate hypergraphs containing higher-order dependencies based on preprocessed historical multivariate time series. The differentiable causal pruning mechanism module is used to adaptively filter effective hyperedges based on candidate hypergraphs containing higher-order dependencies using the differentiable causal pruning mechanism. The hierarchical sparse aggregation mechanism module is used to generate aggregated features that integrate multi-level dependencies based on the effective superedge utilization of the hierarchical sparse aggregation mechanism. A bidirectional predictive decoder for time series prediction based on aggregated features that fuse multi-level dependencies.

[0009] The beneficial effects of this invention are as follows: 1. By constructing an integrated system of "hypergraph construction - causal pruning - hierarchical aggregation - bidirectional decoding", we can accurately characterize and dynamically filter high-order dependencies between channels. At the same time, we can optimize the interactive modeling of future time steps, solve the core problems of difficulty in balancing prediction accuracy and scalability in high-dimensional scenarios and the interference of spurious dependencies on prediction reliability, and significantly improve the accuracy and robustness of multivariate time series in long-term and short-term prediction tasks.

[0010] 2. By utilizing hierarchical sparse aggregation to achieve efficient integration of multi-level information, and introducing bidirectional predictive decoding to optimize future time step interaction, the prediction accuracy and causal interpretability of multivariate time series are taken into account, significantly improving the accuracy and generalizability of long-term and short-term prediction tasks in multiple fields such as traffic flow, power consumption, and meteorological monitoring. Attached Figure Description

[0011] Figure 1 A flowchart of a scalable multivariate time series forecasting method based on hypergraph causal modeling is provided for an embodiment. Figure 2 This is a structural diagram of a scalable multivariate time series prediction system based on hypergraph causal modeling, provided for an embodiment. Detailed Implementation

[0012] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0013] like Figure 1 As shown, in one embodiment of the present invention, a scalable multivariate time series prediction method based on hypergraph causal modeling includes the following steps: S1. Use the preprocessing module to obtain the historical multivariate time series of the target object and perform preprocessing.

[0014] Obtain the historical multivariate time series corresponding to the time series to be predicted, such as traffic flow, electricity consumption, and weather monitoring. The historical multivariate time series of the target object includes, but is not limited to, traffic flow, speed, and traffic density of a road segment at multiple time steps; it also includes electricity consumption, electricity price, power generation, and power flow data of a region at multiple time steps; and wind speed, precipitation, humidity, temperature, light intensity, and cloud cover of a region at multiple time steps. Input the obtained historical multivariate time series as follows: Figure 2 The scalable multivariate time series forecasting system based on hypergraph causal modeling shown uses a preprocessing module to preprocess the historical multivariate time series of the target object.

[0015] The preprocessing includes: Use a length of Step size is (generally The sliding window divides historical multivariate time series into local segments, where the first segment... The segments are as follows:

[0016] in, Indicates the first Each segment Display window Inner Multivariate observations at each time step Indicates the number of channels is Length is A real matrix. The total number of windows is ,in .

[0017] S2. Using the higher-order hypergraph construction module, construct candidate hypergraphs containing higher-order dependencies based on preprocessed historical multivariate time series.

[0018] Specifically, applying the Fast Fourier Transform to each segment of the historical multivariate time series yields the complex numerical spectrum, expressed as follows: Frequency domain feature extraction can effectively reveal the signal periodicity and spectral components in multi-channel time series, providing reliable input for constructing high-order hypergraphs.

[0019] The spectrum simultaneously encodes the amplitude and phase information of each frequency component. Furthermore, effective frequency domain features are extracted by calculating the spectral amplitude vector, which highlights the amplitude of each frequency component and reflects the distribution of signal energy at different frequencies. Its expression is:

[0020] in, Indicates channel The spectral magnitude vector, Display window inner passage In the The amplitude of each frequency interval; the superscript T denotes the transpose of the matrix. This representation provides a more interpretable description of the signal frequency content for subsequent analysis.

[0021] Furthermore, to quantize channel similarity in the frequency domain, the cosine similarity of the spectral amplitude vectors between channels is calculated, and its expression is:

[0022] in, Display window inner passage With channel The similarity matrix, Indicates channel The spectral magnitude vector, Denotes the Euclidean norm; where, ; Thresholds are introduced based on the similarity matrix. Identifying pairwise relationships and constructing candidate hyperedges, specifically: Define an undirected graph, its expression is:

[0023] in, Represents an undirected graph. For channel set, ; For edge set, ; By identifying undirected graphs Connected components are used to generate hyperedges, and the expression for the set of hyperedges is:

[0024] in, Denotes the set of superedges. For window The Middle One super-edge, For superedge index; Indicates the superedge The base number, i.e., the number of channels; Indicates the first w In the sliding window, the first k a super-edge The included first The index of a channel; Card(·) counts the number of elements in a set. That is, to represent the super-edge How many channel nodes are included? Importantly, due to the characteristics of the clustering process, any two hyperedges are disjoint, that is, for... ,have And the union of all superedges covers all or some of the nodes, i.e. .

[0025] Based on the generated hyperedges, a candidate hypergraph containing higher-order dependencies is obtained, and its expression is:

[0026] in, This represents a candidate hypergraph.

[0027] S3. The differentiable causal pruning mechanism module adaptively filters effective hyperedges based on candidate hypergraphs containing higher-order dependencies.

[0028] The specific method is as follows: To evaluate the causal contribution of each hyperedge to the prediction task, we first extract the joint features of the channels contained in the hyperedge based on local segments of historical multivariate time series, expressed as:

[0029] in, Represents the joint characteristic matrix of hyperedges. Indicates the superedge The Middle Time vectors of each channel; The causal scoring calculation for the joint feature matrix of hyperedges is expressed as follows:

[0030] in, Indicates the superedge In the window In the causal score, a higher score indicates a greater impact on multivariate time series prediction; a lower score indicates a smaller contribution. The sigmoid activation function is represented by sigmoid, and MLP represents the corresponding multilayer perceptron processing. Represents the learnable parameters of the MLP; Define hyperedge The initial feature vector is ; Utilizing hyperedge In the window Causal rating in Adaptive modulation is performed, and its expression is:

[0031] in, To extend beyond the edge after pruning Corresponding features.

[0032] To encourage the system to assign lower scores to irrelevant hyperedges while retaining hyperedges with strong causal contributions, a pruning loss is introduced, expressed as follows:

[0033] in, This represents the pruning loss value. Display window The causal rating vector in ; Represents the regularization coefficient; Represents the Manhattan norm; Super-edge features with pruning loss values ​​less than or equal to a preset threshold are retained, while super-edge features with pruning loss values ​​greater than the preset threshold are removed.

[0034] S4. Based on the effective superedge, the hierarchical sparse aggregation mechanism module generates aggregated features that integrate multi-level dependencies.

[0035] Traditional aggregation methods (such as pooling or averaging) struggle to capture the dynamic high-order dependencies between hyperedges. To address this, this invention adapts a multi-head attention mechanism to the hyperedge level, enabling flexible and structure-aware information integration between sparse channel groups.

[0036] The specific method is as follows: The hyperedge feature matrix after causal pruning is constructed as follows:

[0037] in, Represents the hyperedge feature matrix; Display window The number of hyperedge features retained in the middle. Indicates the first Features corresponding to each hyperedge; Hyperedge aggregation is performed using a multi-head attention mechanism to obtain global-level aggregated features, the expression of which is:

[0038] in, Represents the global representation. Indicates splicing, The weight matrix is ​​used to perform linear transformation and fusion on the hyperedge features obtained by the multi-head attention mechanism to generate a unified global representation. This matrix is ​​adaptively updated during model training using the gradient descent algorithm. Indicates the first The channel interaction subspace captured by each attention head through scaling dot product attention is expressed as follows:

[0039] in, Represents the query vector. Represents the key vector. Represents a value vector. , , For the first Each attention head has a unique learnable projection matrix. Let be the dimension of the key vector. This global attention mechanism enables the system to integrate cross-hyperedge dependencies and capture diverse structural relationships between channel groups within the current window.

[0040] Next, local attention is used within a single hyperedge to capture channel associations, resulting in a subgroup-level representation, expressed as:

[0041] in, This represents the subgroup level representation. for Activation function For query vector, For key vectors, It is a value vector; , , A learnable projection matrix unique to each hyperedge; The global-level aggregated features are concatenated with all subgroup-level representations to obtain the fused representation, which is expressed as follows:

[0042] in, For fusion characterization; This represents a learnable linear projection matrix used to map the concatenated representation to a dimension of 1. A shared semantic fusion space.

[0043] S5. Time series prediction is performed using a bidirectional predictive decoder based on the aggregated features of multi-level dependencies.

[0044] Standard decoders rely solely on historical inputs, often ignoring potential dependencies between future time steps. To address this limitation, this invention proposes a bidirectional decoding strategy with future-aware capabilities, enabling mutual optimization between prediction steps.

[0045] The specific method is as follows: Initial predictions of future values ​​are generated using a standard Transformer decoder to fuse representations. and historical multivariate time series within the current window As input to the Transformer decoder, its expression is:

[0046] in, Indicates the initial prediction. Indicates the predicted duration. Indicates a decoding operation; To further improve the temporal consistency of future forecasts, an auxiliary Transformer module is used to optimize the initial forecast, the expression of which is:

[0047] in, This is an optimized sequence that includes future bidirectional interstep interactions; This indicates the corresponding processing handled by the auxiliary Transformer module; By combining learnable weighted predictions with the advantages of initial predictions and bidirectional optimized predictions, the expression is as follows:

[0048] in, For the final predicted multivariate time series, These are learnable fusion coefficients used to adaptively balance the contributions of the two prediction branches.

[0049] In another feasible embodiment of the present invention, taking the prediction of precipitation and temperature in a certain area over the next 24 hours as an example: Historical precipitation and temperature sequences for the region are obtained and preprocessed accordingly. Based on the preprocessed historical multivariate sequences (including precipitation and temperature), candidate hypergraphs containing higher-order dependencies are constructed for precipitation and temperature. Effective hyperedges are adaptively filtered using a differentiable causal pruning mechanism based on the obtained candidate hypergraphs containing higher-order dependencies. Based on the effective hyperedges, a hierarchical sparse aggregation mechanism is used to generate aggregated features that fuse multi-level dependencies. Time series prediction is performed using a bidirectional predictive decoder based on the aggregated features that fuse multi-level dependencies to obtain the precipitation and temperature data sequences for the next 24 hours. It should be noted that the specific implementation steps in this embodiment are logically identical to the aforementioned steps S1-S5, and will not be repeated here.

[0050] In summary, this invention constructs an integrated system of "hypergraph construction - causal pruning - hierarchical aggregation - bidirectional decoding" to achieve accurate characterization and dynamic filtering of high-order dependencies between channels. Simultaneously, it optimizes future time step interaction modeling, addressing core issues such as the difficulty in balancing prediction accuracy and scalability in high-dimensional scenarios and the interference of spurious dependencies on prediction reliability. This significantly improves the accuracy and robustness of multivariate time series forecasting in both short and long-term tasks. Furthermore, by utilizing hierarchical sparse aggregation to achieve efficient integration of multi-level information and introducing bidirectional prediction decoding to optimize future time step interactions, it balances the prediction accuracy and causal interpretability of multivariate time series, significantly improving the accuracy and generalizability of short and long-term forecasting tasks in multiple fields such as traffic flow, electricity consumption, and meteorological monitoring.

Claims

1. A scalable multivariate time series forecasting method based on hypergraph causal modeling, characterized in that, include: Obtain the historical multivariate time series of the target object and perform preprocessing; Construct candidate hypergraphs containing higher-order dependencies based on preprocessed historical multivariate time series; Effective hyperedges are adaptively selected based on a differentiable causal pruning mechanism using candidate hypergraphs containing higher-order dependencies. Based on the effective hyperedge utilization hierarchical sparse aggregation mechanism, aggregated features that integrate multi-level dependencies are generated. Time series prediction is performed using a bidirectional predictive decoder based on the aggregated features that integrate multi-level dependencies. The historical multivariate time series of the target object is: traffic flow, speed, and traffic density of a certain road segment at multiple time steps; or electricity consumption, electricity price, power generation, and power flow data of a certain region at multiple time steps; or wind speed, precipitation, humidity, temperature, light intensity, and cloud cover of a certain region at multiple time steps.

2. The method according to claim 1, characterized in that, The preprocessing includes: Use a length of Step size is A sliding window divides historical multivariate time series into local segments, where the first segment is the first segment. The segments are as follows: in, Indicates the first Each segment Display window Inner Multivariate observations at each time step Indicates the number of channels is Length is A real matrix.

3. The method according to claim 2, characterized in that, The specific method for constructing a candidate hypergraph containing higher-order dependencies is as follows: Applying the Fast Fourier Transform to each segment of the historical multivariate time series yields the complex numerical spectrum; The expression for calculating the spectral amplitude vector based on the complex numerical spectrum is as follows: in, Indicates channel The spectral magnitude vector, Display window inner passage In the The amplitude of each frequency range; the superscript T indicates the transpose of the matrix; The cosine similarity of the spectral amplitude vectors between channels is calculated using the following expression: in, Display window inner passage With channel The similarity matrix, Indicates channel The spectral magnitude vector, Denotes the Euclidean norm; where ; Thresholds are introduced based on the similarity matrix. Identifying pairwise relationships and constructing candidate hyperedges, specifically: Define an undirected graph, its expression is: in, Represents an undirected graph. For channel set, ; For edge set, ; By identifying undirected graphs Connected components are used to generate hyperedges, and the expression for the set of hyperedges is: ， in, Denotes the set of superedges. For window The Middle One super edge, For superedge index; Indicates the superedge The base number, i.e., the number of channels; Indicates the first w In the sliding window, the first k a super-edge The included first The index of a channel; Card(·) counts the number of elements in a set. That is, to represent the super edge How many channel nodes are included? Based on the generated hyperedges, a candidate hypergraph containing higher-order dependencies is obtained, and its expression is: in, This represents a candidate hypergraph.

4. The method according to claim 3, characterized in that, The specific method for adaptively selecting effective superedges using the differentiable causal pruning mechanism is as follows: The joint features of channels contained in hyperedges are extracted from local segments based on historical multivariate time series, and their expression is as follows: in, Represents the joint characteristic matrix of hyperedges. Indicates the superedge The Middle Time vectors of each channel; The causal scoring calculation for the joint feature matrix of hyperedges is expressed as follows: in, Indicates the superedge In the window Causal rating in The sigmoid activation function is represented by sigmoid, and MLP represents the corresponding multilayer perceptron processing. Represents the learnable parameters of the MLP; Define hyperedge The initial feature vector is ; Utilizing hyperedge In the window Causal rating in Adaptive modulation is performed, and its expression is: in, For the edge after pruning The corresponding features are denoted as hyperedge features; Introducing the pruning loss, its expression is: in, This represents the pruning loss value. Display window The causal rating vector in ; Represents the regularization coefficient; Represents the Manhattan norm; Super-edge features with pruning loss values ​​less than or equal to a preset threshold are retained, while super-edge features with pruning loss values ​​greater than the preset threshold are removed.

5. The method according to claim 4, characterized in that, The specific method for generating aggregated features that integrate multi-level dependencies based on the hierarchical sparse aggregation mechanism using effective hyperedges is as follows: The hyperedge feature matrix after causal pruning is constructed as follows: in, Represents the hyperedge feature matrix; Display window The number of hyperedge features retained in the middle. Indicates the first Features corresponding to each hyperedge; Hyperedge aggregation is performed using a multi-head attention mechanism to obtain global-level aggregated features, the expression of which is: in, Represents the global representation. Indicates splicing, This is the weight matrix; Indicates the first The channel interaction subspace captured by each attention head through scaling dot product attention is expressed as follows: in, Represents the query vector. Represents the key vector. Represents a value vector. , , For the first Each attention head has a unique learnable projection matrix. The dimension of the key vector; Within a single hyperedge, local attention is used to capture channel associations, resulting in a subgroup-level representation, the expression of which is: in, This represents the subgroup level representation. for Activation function For query vector, For key vectors, It is a value vector; , , A learnable projection matrix unique to each hyperedge; The global-level aggregated features are concatenated with all subgroup-level representations to obtain the fused representation, which is expressed as follows: in, For fusion characterization; This represents a learnable linear projection matrix used to map the concatenated representation to a dimension of 1. A shared semantic fusion space.

6. The method according to claim 5, characterized in that, The specific method for time series prediction using a bidirectional predictive decoder based on aggregated features that fuse multi-level dependencies is as follows: Initial predictions of future values ​​are generated using a standard Transformer decoder to fuse representations. and historical multivariate time series within the current window As input to the Transformer decoder, its expression is: in, Indicates the initial prediction. Indicates the predicted duration. Indicates a decoding operation; The initial prediction is optimized using the auxiliary Transformer module, and its expression is: in, This is an optimized sequence that includes future bidirectional interstep interactions; This indicates the corresponding processing handled by the auxiliary Transformer module; By combining learnable weighted predictions with the advantages of initial predictions and bidirectional optimized predictions, the expression is as follows: in, For the final predicted multivariate time series, These are learnable fusion coefficients used to adaptively balance the contributions of the two prediction branches.

7. A system for implementing the scalable multivariate time series forecasting method based on hypergraph causal modeling as described in any one of claims 1 to 6, characterized in that, include: The preprocessing module is used to obtain the historical multivariate time series of the target object and perform preprocessing. The higher-order hypergraph construction module is used to construct candidate hypergraphs containing higher-order dependencies based on preprocessed historical multivariate time series. The differentiable causal pruning mechanism module is used to adaptively filter effective hyperedges based on candidate hypergraphs containing higher-order dependencies using the differentiable causal pruning mechanism. The hierarchical sparse aggregation mechanism module is used to generate aggregated features that integrate multi-level dependencies based on the effective superedge utilization of the hierarchical sparse aggregation mechanism. A bidirectional predictive decoder for time series prediction based on aggregated features that fuse multi-level dependencies.

Images