Multivariate time series prediction method and system based on structural representation learning

Abstract

The application provides a multivariate time series prediction method and system based on structure representation learning, and belongs to the technical field of multivariate time series prediction. The method comprises the following steps: constructing a similarity matrix between variables according to multivariate time series data, obtaining a variable community label through community detection, and obtaining a binary community perception adjacency matrix through weighted binary coding; inputting an initial variable feature matrix and the adjacency matrix into a graph autoencoder to obtain a variable structure embedding matrix; obtaining a unified input matrix by fusing variable value embedding, time step position embedding and structure embedding; inputting a time series prediction model based on structure information guided attention mechanism modeling to obtain a prediction result; wherein the structure embedding serves as an auxiliary feature for modeling without applying an explicit structure constraint. The application mines variable correlation through structure representation learning, and improves the precision and generalization ability of multivariate time series prediction.

Description

Technical Field

[0001] This invention relates to the intersection of structural representation learning and time series modeling in multivariate time series forecasting, specifically to a multivariate time series forecasting method and system based on structural representation learning. Background Technology

[0002] In numerous real-world scenarios such as intelligent transportation, financial markets, industrial monitoring, and power energy management, it is often necessary to predict future values ​​based on historical observation data of multiple interdependent variables. The core task corresponding to this technical requirement is Multivariate Time Series Forecasting (MTSF). With the widespread adoption of IoT devices and sensors, time series data generated by modern systems exhibit significant characteristics of multi-source heterogeneity, multi-scale variation, and high coupling between variables. How to accurately model both the spatial dependencies between variables and the temporal dynamic evolution of the data itself has become a core research challenge that urgently needs to be addressed in the field of MTSF. Traditional statistical methods often focus on univariate modeling or assume that variables are independent, failing to capture the complex structural relationships hidden between variables. Early deep learning models, such as convolutional neural networks and recurrent neural networks, while improving nonlinear fitting capabilities, still do not fully consider the correlation between variables. Models relying on static prior graphs such as geographic adjacency graphs or mutual information graphs are difficult to adapt to the actual situation of dynamic changes in system topology over time. Against this backdrop, Graph Neural Networks (GNNs) have become one of the mainstream technologies in multivariate time series prediction tasks due to their unique advantages in modeling non-Euclidean structures and learning potential dependencies between nodes.

[0003] To address the core challenges of multivariate time series prediction, three main solutions have emerged in the field: GNN-based, Transformer-based, and multi-task learning-based. However, all of them have significant drawbacks: ① GNN-based methods: Although these methods can utilize the implicit structural correlations between nodes, and some solutions also combine modules such as convolution and attention mechanisms to capture spatiotemporal dependencies, most of them only rely on pairwise similarity estimation or adjacency reconstruction. It is difficult to explicitly model higher-order topological structures such as modularity, group consistency, and hierarchical clustering. These higher-order topological structures include, but are not limited to, community structures, functional grouping relationships, or hierarchical association patterns among variables. (e.g., functional zoning and sensor sub-clusters in transportation networks) lack high-order structural priors that can improve model interpretability and efficiency, making the model prone to learning spurious or noisy connections caused by random fluctuations; ② Transformer-type methods: With the advantage of self-attention mechanism in capturing long-range time series dependencies, this type of method has become another mainstream choice for multivariate time series prediction. Some schemes optimize the time series modeling effect through trend decomposition, patch modeling, cross-period attention, etc., but most models still treat each variable as an independent channel, ignoring the evolution law of dynamic structure between variables, and lack robust structural prior constraints, which limits the generalization ability on heterogeneous datasets; ③ Multi-task learning methods: This type of method uses structure learning as an auxiliary task, and improves model performance by jointly optimizing the reconstruction and prediction objectives, introducing frequency or structure-assisted supervision, etc., but usually lacks explicit and interpretable community-level prior information, resulting in insufficient structural interpretability and prediction robustness of the model, making it difficult to cope with the prediction needs of complex scenarios.

[0004] In summary, the two most prominent core problems of existing multivariate time series prediction technologies are: first, insufficient high-order topological features, making it difficult to effectively capture collective behavior and functional grouping among variables; and second, limited model generalization ability, making it difficult to adapt to pattern changes caused by data distribution shifts and time dependencies. Summary of the Invention

[0005] In view of this, it is necessary to provide a multivariate time series prediction method and system based on structural representation learning to solve the technical problems of insufficient high-order topological features and limited model generalization ability in the existing technology.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a multivariate time series prediction method based on structural representation learning, comprising:

[0007] A similarity matrix between variables is constructed based on multivariate time series data, and a community label for each variable is obtained through a community detection algorithm. The connection relationships between variables in the similarity matrix are weighted by combining the community labels, and the weighted similarity matrix is ​​binarized to obtain a binarized community-aware adjacency matrix.

[0008] Construct an initial variable feature matrix for multivariate time series prediction, and input the initial variable feature matrix and the binarized community-aware adjacency matrix into a graph autoencoder to obtain a variable structure embedding matrix and a reconstructed community-aware adjacency matrix;

[0009] The variable value embeddings, time step position embeddings, and variable structure embedding matrices obtained from feature extraction of the multivariate time series data are fused after linear projection to obtain a unified input matrix. The unified input matrix is ​​then input into a time series prediction model based on a structure information-guided attention mechanism. The multivariate time series is modeled using the structure information-guided attention mechanism to obtain multivariate time series prediction results. The variable structure embedding matrix serves as an auxiliary feature describing the global structural relationship of variables in the time series modeling process, without imposing explicit structural constraints on the prediction stage.

[0010] In one possible implementation, the multivariate time series data is obtained from the original multivariate time series data through preprocessing; the preprocessing includes:

[0011] Missing values ​​in the original multivariate time series data are imputed to obtain continuous multivariate time series data; wherein, the imputation process includes forward imputation.

[0012] Continuous multivariate time series data are normalized and converted into tensor format to obtain multivariate time series data.

[0013] In one possible implementation, constructing a similarity matrix between variables based on multivariate time series data includes:

[0014] Independent observation units are determined based on independent observation objects and independent observation indicators of multivariate time series data, and the independent observation units are used as the total number of variables in multivariate time series data.

[0015] Determine the strength of the correlation between the variables;

[0016] By combining the total number of variables and the correlation strength between each variable, a preset algorithm is used to traverse all variable pairs to obtain a similarity matrix.

[0017] In one possible implementation, obtaining the community label for each variable through a community detection algorithm includes:

[0018] Each element in the similarity matrix is ​​filtered; the filtering is based on the value of each element in the similarity matrix, retaining the connection relationships between variables corresponding to elements with similarity values ​​greater than 0, and removing connection relationships with similarity values ​​less than or equal to 0; the filtered similarity matrix is ​​converted into an undirected graph adjacency matrix based on its connection relationships between variables;

[0019] The Leiden community detection algorithm is used to partition the adjacency matrix of the undirected graph into communities. At the same time, a modularity function is configured as a measure of the quality of community partitioning. The community partitioning is performed iteratively until a preset number of iterations is reached or the modularity function tends to stabilize.

[0020] The community labels corresponding to each variable are extracted from the iterative output of the Leiden community detection algorithm, and arranged in the original sorting order of the variables to form a community label set, so that each community label corresponds one-to-one with the corresponding variable.

[0021] In one possible implementation, the weighting of the connection relationships between variables in the similarity matrix based on the community tags, followed by binarization of the weighted similarity matrix to obtain a binarized community-aware adjacency matrix, includes:

[0022] For any two elements corresponding to variables in the similarity matrix, determine whether their community labels are consistent. If the community labels are the same, the element is weighted using a preset intra-community connectivity enhancement factor. If the community labels are different, the element is weighted using a preset inter-community connectivity inhibition factor to obtain a weighted similarity matrix.

[0023] The numerical distribution of all elements in the weighted similarity matrix is ​​statistically analyzed, and a preset percentage percentile is selected as the binarization threshold.

[0024] Traverse all elements in the weighted similarity matrix, set elements greater than or equal to the binarization threshold to 1, and set elements less than the binarization threshold to 0, to obtain the binarized community-aware adjacency matrix.

[0025] In one possible implementation, the graph autoencoder includes an encoder and a decoder; the step of inputting the initial variable feature matrix and the binarized community-aware adjacency matrix into the graph autoencoder to obtain a variable structure embedding matrix and a reconstructed community-aware adjacency matrix includes:

[0026] The initial variable feature matrix and the binarized community-aware adjacency matrix are input into the encoder to obtain the variable structure embedding matrix; the encoder consists of a graph convolutional network with a preset number of layers;

[0027] The variable structure embedding matrix is ​​input into the decoder to reconstruct the variable structure embedding matrix, thereby obtaining the reconstructed community-aware adjacency matrix.

[0028] In one possible implementation, the step of fusing the variable value embeddings, time step position embeddings, and variable structure embedding matrices obtained from feature extraction of the multivariate time series data through linear projection to obtain a unified input matrix includes:

[0029] The multivariate time series data is mapped to a preset dimension to obtain variable value embeddings;

[0030] Encode the time step number of the multivariate time series data to obtain the time step position embedding that is consistent with the embedding dimension of the variable value;

[0031] The variable structure embedding matrix is ​​linearly mapped and the time dimension is extended to obtain a structure embedding that is consistent with the variable value embedding dimension.

[0032] The variable value embedding, time step position embedding, and structure embedding are summed element by element to obtain a unified input matrix.

[0033] In one possible implementation, the step of inputting the unified input matrix into a time series prediction model based on a structure information-guided attention mechanism, and performing feature modeling of the multivariate time series through the structure information-guided attention mechanism to obtain multivariate time series prediction results includes:

[0034] Based on the variable structure embedding matrix or the community tag information, construct structural guidance information to characterize the potential structural relationships between variables;

[0035] The structural guidance information is introduced as additional bias information or weight adjustment factor into the attention score calculation process to guide and adjust the attention allocation among different variables;

[0036] The attention scores after introducing structural guidance information are normalized to obtain the attention weight matrix guided by structural information.

[0037] The unified input matrix is ​​weighted and aggregated based on the attention weight matrix to obtain a temporal feature representation of fused structural information;

[0038] The temporal feature representation of the fused structural information is nonlinearly enhanced by a feedforward neural network, and the enhanced temporal feature representation is linearly predicted to obtain multivariate temporal prediction results.

[0039] In one possible implementation, the multivariate time series prediction method further includes model training, wherein the model training optimizes model parameters by minimizing the multi-task joint optimization loss, specifically including:

[0040] The multi-task joint optimization loss is obtained by weighting the reconstruction loss and the temporal prediction loss of the graph autoencoder according to a preset weight; the preset weight includes the reconstruction loss weight coefficient and the temporal prediction loss weight coefficient, both of which are positive numbers and their sum is 1;

[0041] The reconstruction loss is calculated based on the difference between the binarized community-aware adjacency matrix and the reconstructed community-aware adjacency matrix, and the temporal prediction loss is calculated based on the difference between the multivariate temporal prediction result and the true value.

[0042] The parameters of the graph autoencoder, the structural information-guided attention mechanism, and the feedforward neural network are iteratively updated using gradient descent-like algorithms until the multi-task joint optimization loss stabilizes or reaches the preset number of training rounds, thus completing model training.

[0043] Secondly, the present invention also provides a multivariate time series prediction system based on structural representation learning, comprising:

[0044] The community-aware adjacency matrix construction unit is used to construct a similarity matrix between variables based on multivariate time series data, obtain the community label of each variable through a community detection algorithm, perform weighted processing on the connection relationship between variables in the similarity matrix in combination with the community label, and perform binarization processing on the weighted similarity matrix to obtain a binarized community-aware adjacency matrix.

[0045] The variable structure embedding extraction unit is used to construct an initial variable feature matrix for multivariate time series prediction. The initial variable feature matrix and the binarized community-aware adjacency matrix are input into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix.

[0046] The multi-feature fusion and temporal prediction unit is used to fuse the variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of the multivariate time series data after linear projection to obtain a unified input matrix; the unified input matrix is ​​input into a temporal prediction model based on a structure information-guided attention mechanism, and the multivariate time series is modeled by the structure information-guided attention mechanism to obtain multivariate temporal prediction results; wherein, the variable structure embedding matrix participates in the temporal modeling process as an auxiliary feature describing the global structural relationship of variables, without imposing explicit structural constraints on the prediction stage.

[0047] The beneficial effects of this invention are as follows: The multivariate time series prediction method based on structural representation learning provided by this invention firstly constructs a similarity matrix between variables based on multivariate time series data, obtains community labels by combining a community detection algorithm, and performs weighted and binarized processing on the similarity matrix to obtain a binarized community-aware adjacency matrix. This effectively mines the potential association structure between multiple variables. The weighted adjustment of community labels strengthens the association strength of variables within the same community and suppresses the interference of variables from different communities. The binarization processing further simplifies the structural information and removes redundant noise, providing high-quality structured input for subsequent structural representation learning. Secondly, by constructing an initial variable feature matrix and inputting it together with the binarized community-aware adjacency matrix into a graph autoencoder, a variable structure embedding matrix and a reconstructed community-aware adjacency matrix are obtained. The matrix can transform high-dimensional sparse structural information into a low-dimensional dense variable structure embedding matrix, accurately preserving the global structural relationships of variables. At the same time, the reconstruction process can verify the effectiveness of structure learning and ensure the reliability of structural embedding features. Finally, a unified input matrix is ​​obtained by fusing variable value embedding, time step position embedding, and variable structure embedding matrix after linear projection. The structural information is used to guide the attention mechanism of the time series prediction model to perform feature modeling and obtain the prediction results. The variable structure embedding matrix is ​​only used as an auxiliary feature without imposing explicit structural constraints, realizing the deep fusion of time series features and structural features. This allows the model to accurately capture the temporal correlation between multiple variables, while avoiding the problem of reduced generalization ability caused by explicit constraints. Ultimately, it significantly improves the accuracy and stability of multivariate time series prediction. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A schematic flowchart of an embodiment of the multivariate time series prediction method based on structural representation learning provided by the present invention;

[0050] Figure 2 This is a schematic flowchart of another embodiment of the multivariate time series prediction method based on structural representation learning provided by the present invention;

[0051] Figure 3 This is a schematic flowchart of an embodiment of the present invention for preprocessing raw multivariate time series data;

[0052] Figure 4 This is a schematic diagram of an embodiment of the process for constructing a similarity matrix provided by the present invention;

[0053] Figure 5 This is a schematic diagram of an embodiment of the present invention for obtaining community labels for each variable using a community detection algorithm;

[0054] Figure 6 A schematic diagram of an embodiment of the present invention for obtaining a binarized community-aware adjacency matrix;

[0055] Figure 7 A schematic diagram of an embodiment of the present invention for obtaining the variable structure embedding matrix and the reconstructed community-aware adjacency matrix;

[0056] Figure 8 This is a schematic diagram of an embodiment of the present invention for obtaining a unified input matrix;

[0057] Figure 9 This is a schematic flowchart of an embodiment of the present invention, which obtains multivariate time series prediction results by inputting a unified input matrix into a time series prediction model based on a structure information-guided attention mechanism.

[0058] Figure 10 This is a schematic diagram of an embodiment of the present invention that optimizes model parameters by minimizing the joint optimization loss of multiple tasks;

[0059] Figure 11 This is a schematic diagram of an embodiment of the performance comparison chart of MSE and MAE in ablation experiments on the Traffic dataset provided by the present invention.

[0060] Figure 12 This is a schematic diagram of an embodiment comparing the performance of MSE and MAE in ablation experiments on the ETTh1 dataset provided by the present invention.

[0061] Figure 13 A schematic diagram illustrating an embodiment of hyperparameter sensitivity analysis on the Traffic dataset provided by the present invention;

[0062] Figure 14 This is a schematic diagram of an embodiment of a multivariate time series prediction system based on structural representation learning provided by the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0064] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0065] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0066] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0067] Before demonstrating the embodiments, the following terms will be explained.

[0068] Multivariate time series data: A collection of data consisting of observations of multiple interrelated variables at consecutive time points, arranged in an ordered manner over time.

[0069] Community awareness: In model building and training, this is a technical approach that uses community affiliation information of variables to strengthen community associations, optimize feature interactions, and allocate attention.

[0070] Community Tag: A label obtained through a community detection algorithm, used to identify the community cluster to which each variable belongs.

[0071] This invention provides a multivariate time series prediction method and system based on structural representation learning, which will be described below.

[0072] Figure 1 This is a schematic flowchart of an embodiment of the multivariate time series prediction method based on structural representation learning provided by the present invention, as shown below. Figure 1 As shown, a multivariate time series prediction method based on structural representation learning includes:

[0073] S101. Construct a similarity matrix between variables based on multivariate time series data, and obtain the community label of each variable through a community detection algorithm. Combine the community labels to perform weighted processing on the connection relationship between variables in the similarity matrix, and perform binarization processing on the weighted similarity matrix to obtain a binarized community-aware adjacency matrix.

[0074] S102. Construct the initial variable feature matrix for multivariate time series prediction, input the initial variable feature matrix and the binarized community-aware adjacency matrix into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix.

[0075] S103. The variable value embedding, time step position embedding, and variable structure embedding matrices obtained from feature extraction of multivariate time series data are fused after linear projection to obtain a unified input matrix. The unified input matrix is ​​then input into a time series prediction model based on a structure information-guided attention mechanism. The multivariate time series is modeled using the structure information-guided attention mechanism to obtain multivariate time series prediction results. Among these, the variable structure embedding matrix serves as an auxiliary feature describing the global structural relationship of variables in the time series modeling process, without imposing explicit structural constraints on the prediction stage.

[0076] In some embodiments of the present invention, the hardware configuration used in the experiments was as follows: an Intel Xeon Gold 6330 @2.00GHz CPU (60GB of memory) and an NVIDIA RTX 3090 GPU (24GB of video memory) were employed to ensure the efficiency of graph construction and model training for large-scale datasets (such as the Traffic dataset with 862 variables); the software environment was as follows: the operating system was Ubuntu 20.04, CUDA version 11.8, Python version 3.8, the deep learning framework was PyTorch 2.0.0, and the dependent libraries included numpy (for matrix calculation), pandas (for data preprocessing), scikit-learn (for similarity calculation and data partitioning), and matplotlib (for result visualization).

[0077] In some embodiments of the present invention, the datasets selected are nine mainstream MTSF benchmark datasets, covering different application scenarios and data scales, specifically including:

[0078] (1) Small-scale highly relevant data: ETT series datasets (ETTh1, ETTh2, ETTm1, ETTm2), Exchange dataset, ILI dataset;

[0079] (2) Mesoscale data: Weather dataset;

[0080] (3) Large-scale data: Electricity dataset, Traffic dataset.

[0081] The detailed information on the dataset type, number of variables, time steps, and number of communities for the above 9 datasets is shown in Table 1 below:

[0082] Table 1: Dataset Details

[0083]

[0084] The variables for the nine datasets are described below:

[0085] 1. ETTh1&ETTh2 (Electricity Transformer Temperature - Hourly)

[0086] Type: Power transformer temperature data (hourly level)

[0087] Source: Transformer monitoring data from a power company

[0088] Variables (7):

[0089] - HUFL: High Useful Load

[0090] - HULL: High Useless Load

[0091] - MUFL: Middle Useful Load

[0092] - MULL: Middle Useless Load

[0093] - LUFL: Low Useful Load

[0094] - LULL: Low Useless Load

[0095] - OT: Oil Temperature

[0096] 2. ETTm1&ETTm2 (Electricity Transformer Temperature - Minutes)

[0097] Type: Power transformer temperature data (minute-level)

[0098] Variables: Same as ETTh (7 variables)

[0099] 3. Exchange-Rate (Exchange)

[0100] Type: Exchange Rate Data

[0101] Source: International Currency Exchange Market

[0102] Variables (8 currencies)

[0103] 4. ILI (Influenza-Like Illness)

[0104] Type: Influenza disease surveillance data

[0105] Source: A certain influenza surveillance system

[0106] Variables (7):

[0107] - Influenza case rates in 7 different regions of a certain country

[0108] - For example: Northeast, Midwest, South, West, and other regions.

[0109] 5. Weather

[0110] Type: Meteorological observation data

[0111] Source: A biogeochemical research institute

[0112] Variables (21):

[0113] - p: air pressure

[0114] - T: Temperature

[0115] - Tpot: Potential temperature

[0116] - Tdew: Dew point temperature

[0117] - rh: relative humidity

[0118] - VPmax: Saturated vapor pressure

[0119] - VPact: Actual vapor pressure

[0120] - VPdef: Water vapor pressure difference

[0121] - sh: specific humidity

[0122] - H2OC: Water vapor concentration

[0123] - rho: air density

[0124] - wv: wind speed

[0125] - max_wv: Maximum wind speed

[0126] - wd: wind direction

[0127] - rain: rainfall amount

[0128] - raining: Rainfall indicator

[0129] - SWDR: Shortwave Radiation

[0130] - PAR: Photosynthetically Active Radiation

[0131] - and other meteorological indicators

[0132] 6. Electricity

[0133] Type: Power load data

[0134] Source: A machine learning library

[0135] Variables (321):

[0136] - Electricity consumption data from 321 customers

[0137] - Each variable represents an electricity customer.

[0138] 7. Traffic

[0139] Type: Traffic flow data

[0140] Source: A regional transportation department (Caltrans)

[0141] Variables (862):

[0142] - 862 sensors on a highway in a certain region

[0143] Each sensor records road occupancy.

[0144] It should be noted that, to verify the predictive effectiveness and generalization ability of the method of this invention, the preprocessed multivariate time series dataset needs to be divided into a training set and a test set according to a preset ratio. The training set is used for training and optimizing the model parameters, while the test set is used to independently evaluate the final predictive performance of the model. The temporal order of the data is maintained during the partitioning process to avoid disrupting temporal dependencies. Model training will combine the loss function described later to complete iterative parameter optimization. After training, the prediction results are obtained by inputting the test set into the model, and the prediction results are compared with the true values ​​in the test set to verify the feasibility of the method.

[0145] In some embodiments of the present invention, each type of dataset is divided as follows:

[0146] (1) For the datasets ETTh1, ETTh2, ETTm1, ETTm2, Weather, Electricity, and Traffic: the input window length T=96 is used, and the prediction time step is... The data is divided into "training set: validation set: test set = 7:2:1" settings.

[0147] (2) For the ILI dataset: the input window length T=36 and the prediction time step are set to 36. The settings are also configured to divide the data according to the ratio of "7:2:1".

[0148] It should be noted that "time interval" and "time step" are synonymous. Both refer to a fixed time unit for continuous observation in multivariate time series data. That is, the original data is observed and recorded once according to a preset uniform time span (such as hourly or daily). The time unit corresponding to each observation is called a "time step" or a "time interval". The time span between two adjacent observation records is consistent with this time unit.

[0149] In summary, the multivariate time series prediction method based on structural representation learning provided in this invention firstly constructs a similarity matrix between variables based on multivariate time series data, obtains community labels by combining a community detection algorithm, and performs weighted and binarized processing on the similarity matrix to obtain a binarized community-aware adjacency matrix. This effectively uncovers the potential association structures between multiple variables. The weighted adjustment of community labels strengthens the association strength of variables within the same community and suppresses interference from variables outside the community. The binarization process further simplifies structural information and removes redundant noise, providing high-quality structured input for subsequent structural representation learning. Secondly, by constructing an initial variable feature matrix and inputting it together with the binarized community-aware adjacency matrix into a graph autoencoder, a variable structure embedding matrix and a reconstructed community-aware adjacency matrix are obtained. The matrix can transform high-dimensional sparse structural information into a low-dimensional dense variable structure embedding matrix, accurately preserving the global structural relationships of variables. At the same time, the reconstruction process can verify the effectiveness of structure learning and ensure the reliability of structural embedding features. Finally, a unified input matrix is ​​obtained by fusing variable value embedding, time step position embedding, and variable structure embedding matrix after linear projection. The structural information is used to guide the attention mechanism of the time series prediction model to perform feature modeling and obtain the prediction results. The variable structure embedding matrix is ​​only used as an auxiliary feature without imposing explicit structural constraints, realizing the deep fusion of time series features and structural features. This allows the model to accurately capture the temporal correlation between multiple variables, while avoiding the problem of reduced generalization ability caused by explicit constraints. Ultimately, it significantly improves the accuracy and stability of multivariate time series prediction.

[0150] In some embodiments of the present invention Figure 2 This is a schematic flowchart of another embodiment of the multivariate time series prediction method based on structural representation learning provided by the present invention.

[0151] In some embodiments of the present invention, multivariate time series data are obtained from raw multivariate time series data through preprocessing; such as... Figure 3 As shown, the preprocessing includes:

[0152] S301. Impute missing values ​​in the original multivariate time series data to obtain continuous multivariate time series data; the imputation method includes forward imputation.

[0153] S302. Normalize the continuous multivariate time series data and convert it into tensor format to obtain multivariate time series data.

[0154] In some embodiments of the present invention, Z-score normalization is performed on continuous multivariate time series data according to the following formula 1:

[0155] (1)

[0156] in, These are the time series data values ​​obtained after Z-score normalization. The mean of the training set data. The standard deviation of the training set data, and and Calculations are performed only on the training set to avoid leaking validation and test set information into the training process.

[0157] In some embodiments of the present invention, the normalized data is converted into PyTorch tensor format. Where T is the input window length, which is 36 or 96, N is the number of variables, and F=1 indicates that the dataset is a single-feature time series, which is used as input for subsequent modules.

[0158] It should be noted that missing value imputation reduces noise interference from invalid data on model training, ensuring the integrity and reliability of input data; normalization and format conversion unify data units, adapt to the input requirements of the model, and effectively improve the stability of model training.

[0159] In some embodiments of the present invention, a similarity matrix between variables is constructed based on multivariate time series data, such as... Figure 4 As shown, it includes:

[0160] S401. Determine independent observation units based on independent observation objects and independent observation indicators of multivariate time series data, and use the independent observation units as the total number of variables of the multivariate time series data;

[0161] S402. Determine the correlation strength between each variable;

[0162] S403. Combine the total number of variables and the correlation strength between each variable, and use a preset algorithm to traverse all variable pairs to obtain a similarity matrix.

[0163] In some embodiments of the present invention, by calculating the absolute value of the pairwise Pearson correlation coefficient |r| between variables, the correlation strength is judged (high correlation: |r|≥0.6; medium correlation: 0.3≤|r|<0.6; low correlation: |r|<0.3); the data scale is judged by the total number of variables N (small scale: N≤20; medium scale: 20<N≤300; large scale: N>300).

[0164] In some embodiments of the present invention, when combining the total number of variables and the correlation strength between each variable, and using a preset algorithm to traverse all variable pairs to obtain a similarity matrix, the selection of the preset algorithm needs to be flexibly determined in combination with the scale of the data and the correlation characteristics between variables. Specifically, different calculation methods can be adapted according to the data type:

[0165] ① Small-scale high-correlation data (such as ETTh1): Call the Pearson similarity function of the scikit-learn library, and calculate the similarity matrix as shown in the following formula (2):

[0166] (2)

[0167] Where, represents the Pearson similarity between variable i and variable j; t represents the index of the time step (indicating the t-th time point); T represents the total number of time steps (i.e., the time length of the sequence); represents the observed value of variable i at the t-th time step; represents the mean value of the observed values of variable i over all T time steps; represents the observed value of variable j at the t-th time step; the mean value of the observed values of variable j over all T time steps; is the finally obtained similarity matrix, where N is the total number of variables, and the matrix is a real matrix with N rows and N columns.

[0168] ② Medium-scale medium-correlation data (such as Weather) or large-scale medium-correlation data (such as Electricity, Traffic): Call the Cosine similarity function, and calculate the similarity matrix as shown in the following formula (3):

[0169] (3)

[0170] in, The cosine similarity between variables i and j is represented by t; the index of the time step is represented by t; and the total number of time steps is represented by T. The table shows the observed values ​​of variable i at time step t; The table shows the observed values ​​of variable i at time step t.

[0171] ③ Small-scale, low-relevance data (such as Exchange, ILI): Call the mutual_info_score (Mutual Information, MI) similarity function and calculate the similarity matrix as shown in Formula 4 below:

[0172] (4)

[0173] in, This represents one possible value of variable i; This represents one possible value of variable j; Represents all possible values ​​of variable i. Sum all possible values ​​b of variable j; Indicates that variable i takes Value, variable j takes The joint probability of the values ​​is solved by the kernel density estimation method; Indicates that variable i takes The marginal probability of the value; This represents the marginal probability that variable j takes the value b.

[0174] ④ Supplementary validation (e.g., for the small-scale, low-correlation ILI dataset): Use the dtw library to calculate the DTW distance, and calculate the similarity matrix as shown in Formula 5 below:

[0175] (5)

[0176] in, This represents the similarity between variables i and j based on DTW distance (normalized to the [0,1] interval). This represents the dynamic time warping (DTW) distance between the time series of variable i (composed of all its time step observations) and the time series of variable j.

[0177] In some embodiments of the present invention, the community label of each variable is obtained through a community detection algorithm, such as... Figure 5 As shown, it includes:

[0178] S501. Filter each element in the similarity matrix; the filtering is based on the value of each element in the similarity matrix, retaining the connection relationships between variables corresponding to elements with similarity values ​​greater than 0, and removing connection relationships with similarity values ​​less than or equal to 0; convert the filtered similarity matrix into an undirected graph adjacency matrix based on its connection relationships between variables.

[0179] S502. The Leiden community detection algorithm is used to partition the adjacency matrix of the undirected graph into communities. At the same time, a modularity function is configured as the criterion for measuring the quality of community partitioning. The community partitioning is iterated until the preset number of iterations is reached or the modularity function tends to stabilize.

[0180] S503. Extract the community labels corresponding to each variable from the iterative output of the Leiden community detection algorithm, arrange them in the original order of the variables to form a community label set, so that each community label corresponds one-to-one with the corresponding variable.

[0181] Filtering each element in the similarity matrix can filter out meaningless negative associations, simplifying the complexity of the subsequent graph structure and improving computational efficiency.

[0182] In some embodiments of the present invention, the Leiden community detection algorithm is used, and the find_partition function of the leidenalg library is called.

[0183] In some embodiments of the present invention, the similarity matrix S is converted into an adjacency matrix of an undirected graph (the threshold is set to...). >0, meaning elements with similarity greater than 0 are selected from the similarity matrix. The modularity function Modularity VertexPartition is set with 100 iterations to ensure the stability of community partitioning.

[0184] In some embodiments of the present invention, the community label of each variable is obtained after running the Leiden community detection algorithm. ( (The community number to which the i-th variable belongs). For example, the Traffic dataset (862 variables) is eventually divided into 33 communities, and each community corresponds to a sensor cluster in a geographical area.

[0185] The Leiden algorithm's community partitioning process, combined with a modularity function, ensures the quality of the partitioning, achieving accurate and stable community partitioning and clarifying the functional attribution of each variable.

[0186] In some embodiments of the present invention, the connection relationships between variables in the similarity matrix are weighted by combining community tags, and the weighted similarity matrix is ​​binarized to obtain a binarized community-aware adjacency matrix, including, for example... Figure 6As shown, it includes:

[0187] S601. For any two variables in the similarity matrix, determine whether their community labels are the same. If the community labels are the same, use a preset intra-community connection enhancement factor to weight the element. If the community labels are different, use a preset inter-community connection inhibition factor to weight the element, and obtain a weighted similarity matrix.

[0188] S602. The numerical distribution of all elements in the statistical weighted similarity matrix is ​​used to select a preset percentage percentile as the binarization threshold.

[0189] S603. Traverse all elements in the weighted similarity matrix, set elements greater than or equal to the binarization threshold to 1, and set elements less than the binarization threshold to 0, to obtain the binarized community-aware adjacency matrix.

[0190] In some embodiments of the present invention, the weighted similarity matrix is ​​calculated according to the following formula 6:

[0191] (6)

[0192] Among them, >1 represents the community connectivity enhancement factor, 0< A value less than 1 indicates an inter-community connectivity inhibition factor; preferably, an intra-community connectivity enhancement factor can be set. , ; This represents the element in the i-th row and j-th column of the weighted adjacency matrix A; This represents the similarity value between variable i and variable j in the previously calculated similarity matrix; These are the community tags corresponding to variables i and j, respectively. This can be understood as, when variables i and j belong to the same community (… When using an enhancing factor Multiplying by the similarity between the two yields a weighted association strength, when variables i and j belong to different communities ( When using inhibitory factors Multiplying the two by their similarity score highlights the connection within the same community and weakens interference from different communities, ultimately generating a weighted similarity matrix.

[0193] In some embodiments of the present invention, the weighted similarity matrix is ​​binarized and calculated according to the following formula 7:

[0194] (7)

[0195] in, Represents the binarized adjacency matrix The element in the i-th row and j-th column, Let represent the element in the i-th row and j-th column of the weighted adjacency matrix A. This represents the adaptive threshold, calculated using the preset percentage percentiles of all elements in the weighted matrix A as the binarization ratio. For example, the preset percentage percentile for the Traffic dataset is 90. = 0.32, the sparsity of the image after binarization is maintained at 0.15 (balancing structural integrity and computational efficiency).

[0196] It should be noted that the preset percentile, taking the 90th percentile as an example, is the threshold value that allows 90% of the weight values ​​to be less than or equal to it and only 10% of the weight values ​​to be greater than or equal to it after sorting the connection weights between all variables in the weighted adjacency matrix from smallest to largest. Using it as a threshold can filter out the 10% of core connections with the highest weights, while maintaining the sparsity of the balanced graph and preserving effective information.

[0197] Binarization clarifies the effective connections between variables, further optimizes the quality of the graph structure, and reduces the computational complexity of the model.

[0198] In some embodiments of the present invention, the graph autoencoder includes an encoder and a decoder; the initial variable feature matrix and the binarized community-aware adjacency matrix are input into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix, such as... Figure 7 As shown, it includes:

[0199] S701. Input the initial variable feature matrix and the binarized community-aware adjacency matrix into the encoder to obtain the variable structure embedding matrix; the encoder consists of a graph convolutional network with a preset number of layers;

[0200] S702. Input the variable structure embedding matrix into the decoder, reconstruct the variable structure embedding matrix, and obtain the reconstructed community-aware adjacency matrix.

[0201] In some embodiments of the present invention, the initial variable feature matrix is ​​used as the node feature input to the graph autoencoder, and the binarized community-aware adjacency matrix is ​​used as the graph structure information to participate in the graph convolution operation, thereby realizing the input of the initial variable feature matrix and the binarized community-aware adjacency matrix into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix.

[0202] It should be noted that the initial variable feature moments are usually identity matrices or learnable embedding tables. When set to an identity matrix, it indicates that the initial features of each variable are its own unique identity identifiers, and there is no initial feature association between variables. This allows the graph autoencoder to focus on the topological relationships between variables represented by the binarized community-aware adjacency matrix. Through the encoding-decoding process, the initial identity identifiers are optimized into variable structure embedding matrices containing high-order topological information. This is suitable for multivariate time series prediction scenarios without clear prior variable features.

[0203] In some embodiments of the present invention, preferably, such as Figure 2 As shown, the graph autoencoder uses a 2-layer graph convolutional network (GCN) as the encoder, and the input is a binarized community-aware adjacency matrix. With the initial node feature matrix (Preferably, it is set as an identity matrix) ):

[0204] ① First layer GCN: Input dimension N, output dimension 2d (d = 64), activation function is ReLU;

[0205] ② Second-layer GCN: Input dimension 2d, output dimension d, no activation function, output node structure embedding .

[0206] In some embodiments of the present invention, preferably, the decoder employs an inner product decoder to reconstruct the binarized community-aware adjacency matrix according to the following formula 8:

[0207] (8)

[0208] in, Represents a node structure embedding vector with uniform dimensions. (·) is the sigmoid function, which maps the calculation result to the range [0,1] so that the reconstructed value conforms to the value logic of the adjacency matrix; The embedding vector of node i transpose; Let represent the d-dimensional embedding vector corresponding to the j-th node in the node structure embedding matrix Z output by the encoder. By using the inner product of the node embedding vectors output by the encoder, combined with the sigmoid function, the connection relationships between nodes are restored, thus reconstructing the adjacency matrix.

[0209] The feature fusion process of graph autoencoders integrates the variable's own features with topological association information, generating a low-dimensional, dense variable structure embedding matrix, which improves the effectiveness of feature representation.

[0210] In some embodiments of the present invention, the variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of multivariate time series data are fused after linear projection to obtain a unified input matrix, such as... Figure 8 As shown, it includes:

[0211] S801. Map multivariate time series data to a preset dimension to obtain variable value embeddings;

[0212] S802. Encode the time step number of the multivariate time series data to obtain the time step position embedding that is consistent with the variable value embedding dimension.

[0213] S803. Linearly map the variable structure embedding matrix and extend the time dimension to obtain a structure embedding consistent with the variable value embedding dimension.

[0214] S804. The variable value embedding, time step position embedding, and structure embedding are summed element by element to obtain a unified input matrix.

[0215] In some embodiments of the present invention, the variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of multivariate time series data are fused after linear projection to obtain a unified input matrix, which is obtained as follows:

[0216] ① Variable value embedding ( ): Input time series Mapping to dimensions via linear layers =128, thus obtaining ;

[0217] ② Time step position embedding ( The position is generated using sinusoidal position encoding, as shown in Formula 9 below:

[0218] (9)

[0219] Where t represents the time step index (corresponding to the t-th time step of the input, t∈[1, T]); d represents the embedding dimension index (corresponding to d model The d-th dimension in the dimension, d∈[1,d model ]), dimensions and Consistency. The even-numbered positions in the embedding dimension are calculated using a sine function, and the odd-numbered positions using a cosine function, to avoid overlapping encoded values ​​at different positions and ensure the distinguishability of the time sequence.

[0220] ③Structural embedding ( ): Embed the variable structure output by the graph autoencoder into a matrix Mapped to via linear layer Dimension, then expand the time dimension to T, to obtain ;

[0221] ④ Finally, the variable value embedding, time step position embedding, and structure embedding are summed element by element, as shown in Formula 10 below:

[0222] (10)

[0223] After summing, E becomes the unified input matrix, which serves as the input for the Transformer encoder to guide subsequent structural information.

[0224] The multi-dimensional embedding fusion method integrates three key types of information: data value, time location, and community structure, providing the model with comprehensive and suitable input and strongly supporting subsequent accurate predictions.

[0225] In some embodiments of the present invention, a unified input matrix is ​​input into a time series prediction model based on a structure information-guided attention mechanism. The structure information-guided attention mechanism is used to model the features of multivariate time series, resulting in multivariate time series prediction results, such as... Figure 9 As shown, it includes:

[0226] S901. Based on variable structure embedding matrix or community tag information, construct structural guidance information to characterize the potential structural relationships between variables;

[0227] S902. Introduce structural guidance information as additional bias information or weight adjustment factor into the attention score calculation process to guide and adjust the attention allocation among different variables;

[0228] S903. Normalize the attention score after introducing structural guidance information to obtain the attention weight matrix guided by structural information.

[0229] S904. Weighted aggregation of the unified input matrix is ​​performed based on the attention weight matrix to obtain the temporal feature representation of the fused structural information;

[0230] S905. The temporal feature representation of the fused structural information is nonlinearly enhanced by a feedforward neural network, and the enhanced temporal feature representation is linearly predicted to obtain the multivariate temporal prediction result.

[0231] It should be noted that the time series prediction model used is based on Transformer, but its original self-attention mechanism has been improved by introducing structural information as guidance.

[0232] In some embodiments of the present invention, the structural information-guided attention mechanism is configured as follows:

[0233] ① Number of attention heads: Set to 8, per head dimension , It is the embedding dimension of the Transformer. It is the dimension of each attention head, obtained by splitting the total embedding dimension, and used for parallel computation of multi-head attention;

[0234] ② Additional bias information (additional bias matrix B): Executed according to the following formula 11:

[0235] set up (11)

[0236] This community-awareness bias matrix is ​​used to enhance attention weights within the community, where... Let be the element in the i-th row and j-th column of the bias matrix B; For the community tags corresponding to variables i and j; preferably, the bias value = 0.5;

[0237] ③ Attention calculation: Perform according to the following formula 12:

[0238] V(12)

[0239] Among them, Q (query matrix), K (key matrix) and V (value matrix) are attention core matrices obtained by linear transformation of a unified input vector; To calculate the initial attention scores among the variables; This is a scaling factor to prevent the attention score from becoming too large and causing gradient vanishing. B is used to normalize the attention score; B is the additional bias matrix.

[0240] The subsequent feedforward network consists of two fully connected layers with four intermediate dimensions. The activation function is ReLU, which is used to perform feature transformation on the attention output;

[0241] ④ Number of encoder layers: Set to 3 layers, that is, the Transformer encoder is stacked 3 layers to ensure long-range timing dependency capture capability, while avoiding overfitting.

[0242] The execution process of the community-aware attention mechanism strengthens the attention weight of variables within the same community through bias adjustment, allowing the model to more accurately capture the correlation between variables within the same functional group.

[0243] In some embodiments of the present invention, a linear prediction head is added after the Transformer encoder output, and T×N The output is mapped to the prediction result of H×N×1. , where H is the prediction time step (96 / 192 / 336 / 720).

[0244] In some embodiments of the present invention, the multivariate time series prediction method further includes model training, wherein model training optimizes model parameters by minimizing the multi-task joint optimization loss, specifically as follows: Figure 10 As shown, it includes:

[0245] S1001. The multi-task joint optimization loss is obtained by weighting the reconstruction loss and the temporal prediction loss of the graph autoencoder according to preset weights. The preset weights include the reconstruction loss weight coefficient and the temporal prediction loss weight coefficient, both of which are positive numbers and their sum is 1.

[0246] S1002, The reconstruction loss is calculated based on the difference between the binarized community-aware adjacency matrix and the reconstructed community-aware adjacency matrix, and the temporal prediction loss is calculated based on the difference between the multivariate temporal prediction result and the true value.

[0247] S1003. Use gradient descent-like algorithms to iteratively update the parameters of the graph autoencoder, the structural information-guided attention mechanism, and the feedforward neural network until the multi-task joint optimization loss stabilizes or reaches the preset training rounds, thus completing model training.

[0248] In some embodiments of the present invention, the time-series prediction loss is the primary task, and the mean squared error (MSE) is used, calculated as shown in Formula 13 below:

[0249] (13)

[0250] in, For predicting loss; H is the number of time steps in the input window (i.e., the length of the historical data time steps the model depends on); N is the total number of variables; F = 1 indicates a single feature; t is the time step index of the input window; i is the variable index; f As a characteristic; This represents the true value of the i-th variable in the f-th feature at the t-th historical time step; This represents the predicted value at the corresponding location output by the model. The error of the time series prediction is quantified by calculating the mean of the squared difference between the actual value and the predicted value for all historical time steps, all variables, and all predicted time steps.

[0251] The reconstruction loss uses the binary cross-entropy loss, and the calculation method is shown in Formula 14 below:

[0252] (14)

[0253] in, Let N be the graph autoencoder reconstruction loss (also known as the community classification loss), where N is the total number of variables and i and j are the variable indices. For binary community-aware adjacency matrix; Let represent the connection values ​​of variables i and j in the adjacency matrix reconstructed by the decoder in the graph autoencoder. By using binary cross-entropy loss, the deviation between the "true binary adjacency matrix" and the "adjacency matrix reconstructed by the graph autoencoder" is quantified, forcing the model to learn node embeddings that conform to the real community structure.

[0254] The total loss (multi-task joint optimization loss) is calculated as shown in Formula 15 below:

[0255] (15)

[0256] calculate, The total loss is represented by the tradeoff coefficient. = 0.1 (determined through cross-validation on the validation set; this value was verified on the ETTh1 dataset to optimally balance prediction and reconstruction performance).

[0257] The construction of the multi-task loss function improves the stability and robustness of model training and enhances the reliability of feature learning through the synergistic optimization of temporal prediction loss and community classification loss.

[0258] In some embodiments of the present invention, the optimizer uses the AdamW optimizer, with a learning rate lr = 5e-4, weight decay = 1e-4, and 300 training epochs. Regarding learning rate scheduling, cosine annealing is used, with an initial learning rate of 5e-4, decaying to 0.8 of the current value every 50 epochs to avoid oscillations in later training stages. An early stopping strategy is employed during training: if the validation set MSE does not decrease for 20 consecutive epochs, training is stopped, and the current optimal model parameters are saved. During training, each epoch first calculates the optimal model parameters using a graph autoencoder. Then, the structural information guides the Transformer's calculations. The parameters of all modules are updated by joint backpropagation (the graph autoencoder and structural information guide the Transformer to share the optimizer).

[0259] In some embodiments of the present invention, a trained time series prediction model is obtained by training the time series prediction model based on multi-task joint optimization loss; historical data of the test set to be predicted is then used. The prediction results are output sequentially through the "graph construction module based on community detection and graph autoencoder" and the "MTSF prediction module based on Transformer". .

[0260] In some embodiments of the present invention, in order to evaluate the performance of the method of the present invention, the MSE and MAE indices of the test set (compared to the true values) are calculated in the ablation experiments and hyperparameter sensitivity analysis experiments. contrast).

[0261] It should be noted that in the multivariate time series forecasting task of this invention, MSE (mean squared error) and MAE (mean absolute error) are the core indicators for evaluating forecast accuracy. MSE is the average of the squared differences between the true and predicted values ​​across all historical time steps, variables, and forecast time steps within the input window. It is more sensitive to large forecast deviations and reflects the average squared level of the deviation. MAE is the average of the absolute values ​​of the differences between the true and predicted values ​​within the aforementioned range. It reflects the average level of forecast deviation and, together with MSE, evaluates the robustness of the model's forecasts.

[0262] In some embodiments of this invention, ablation experiments were conducted on the Traffic and ETTh1 datasets to better demonstrate the performance of the method. The experiments compared the prediction performance of four ablation configurations—"community-aware structure modeling without community awareness," "unstructured representation learning," "structured embedding not involved in prediction," and "attention guided by unstructured information"—with the complete method of this invention.

[0263] Among them, "community-aware structure modeling" refers to not performing the steps of constructing a similarity matrix, community detection, and weighted binarization processing based on multivariate time series data in the ablation experiment, thus not introducing a community-aware adjacency matrix; "structure-free representation learning" refers to not introducing a graph autoencoder for variable structure embedding learning in the ablation experiment.

[0264] Figure 11 The figure shows a comparison of MSE and MAE performance in the ablation experiments of this invention on the Traffic dataset. The results show that the MSE (0.361~0.379) and MAE (0.310~0.332) of each ablation configuration are higher than those of the complete method of this invention. The complete method of this invention has an MSE of 0.331 and an MAE of 0.284, which is the best among all configurations.

[0265] Figure 12 The figure shows a comparison of MSE and MAE performance in the ablation experiments of this invention on the ETTh1 dataset. The results show that the MSE (0.272~0.275) and MAE (0.316~0.327) of each ablation configuration are higher than those of the complete method of this invention. The complete method of this invention has an MSE of 0.251 and an MAE of 0.299, which is the best among all configurations.

[0266] The results above indicate that "community detection", "graph autoencoder", "structure embedding" and "structure information-guided attention mechanism" are all key modules in this invention to improve the accuracy of multivariate time series prediction. After the modules work together, the prediction error of the model is significantly reduced.

[0267] Figure 13This is a hyperparameter sensitivity analysis plot on the Traffic dataset, presented as a heatmap, showing the intra-community connectivity enhancement factors. (Value range 1.0~4.0) and inter-community connectivity inhibitory factor (Value range 0.01~0.25) The variation trend of MSE and MAE of the present invention under different combinations.

[0268] To better implement the multivariate time series prediction method based on structural representation learning in the embodiments of the present invention, based on the multivariate time series prediction method based on structural representation learning, the corresponding method is as follows: Figure 14 As shown, this embodiment of the invention also provides a multivariate time series prediction system based on structural representation learning. The multivariate time series prediction system 1400 based on structural representation learning includes:

[0269] The community-aware adjacency matrix construction unit 1401 is used to construct a similarity matrix between variables based on multivariate time series data, obtain the community label of each variable through a community detection algorithm, combine the community label to perform weighted processing on the connection relationship between variables in the similarity matrix, and perform binarization processing on the weighted similarity matrix to obtain a binarized community-aware adjacency matrix.

[0270] The variable structure embedding extraction unit 1402 is used to construct the initial variable feature matrix for multivariate time series prediction. The initial variable feature matrix and the binarized community-aware adjacency matrix are input into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix.

[0271] The multi-feature fusion and temporal prediction unit 1403 is used to fuse the variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of multivariate time series data through linear projection to obtain a unified input matrix. The unified input matrix is ​​then input into a temporal prediction model based on a structure information-guided attention mechanism. The multivariate time series is modeled using the structure information-guided attention mechanism to obtain multivariate temporal prediction results. The variable structure embedding matrix serves as an auxiliary feature describing the global structural relationship of variables in the temporal modeling process, without imposing explicit structural constraints on the prediction stage.

[0272] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0273] The above provides a detailed description of the multivariate time series prediction method and system based on structural representation learning provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A multivariate time series prediction method based on structural representation learning, characterized in that, include: A similarity matrix between variables is constructed based on multivariate time series data of road occupancy collected by multiple sensors in the traffic network. The community label of each road occupancy variable is obtained through a community detection algorithm. The connection relationship between variables in the similarity matrix is ​​weighted by the community label and the weighted similarity matrix is ​​binarized to obtain a binarized community-aware adjacency matrix. Construct an initial variable feature matrix for multivariate time-series prediction of road occupancy, and input the initial variable feature matrix and the binarized community-aware adjacency matrix into a graph autoencoder to obtain a variable structure embedding matrix and a reconstructed community-aware adjacency matrix. The variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of the multivariate time series data are fused after linear projection to obtain a unified input matrix. The unified input matrix is ​​then input into a time series prediction model based on a structure information-guided attention mechanism. The multivariate time series is modeled using the structure information-guided attention mechanism to obtain the multivariate time series prediction result of road occupancy. The variable structure embedding matrix participates in the time series modeling process as an auxiliary feature describing the global structural relationship of variables, without imposing explicit structural constraints on the prediction stage.

2. The method according to claim 1, characterized in that, The multivariate time series data is obtained from the original multivariate time series data through preprocessing; the preprocessing includes: Missing values ​​in the original multivariate time series data are imputed to obtain continuous multivariate time series data; wherein, the imputation process includes forward imputation. Continuous multivariate time series data are normalized and converted into tensor format to obtain multivariate time series data.

3. The method according to claim 1, characterized in that, The construction of a similarity matrix between variables based on multivariate time series data includes: Independent observation units are determined based on independent observation objects and independent observation indicators of multivariate time series data, and the independent observation units are used as the total number of variables in multivariate time series data. Determine the strength of the correlation between the variables; By combining the total number of variables and the correlation strength between each variable, a preset algorithm is used to traverse all variable pairs to obtain a similarity matrix.

4. The method according to claim 1, characterized in that, The process of obtaining the community label for each variable using a community detection algorithm includes: Each element in the similarity matrix is ​​filtered; the filtering is based on the value of each element in the similarity matrix, retaining the connection relationships between variables corresponding to elements with similarity values ​​greater than 0, and removing connection relationships with similarity values ​​less than or equal to 0; the filtered similarity matrix is ​​converted into an undirected graph adjacency matrix based on its connection relationships between variables; The Leiden community detection algorithm is used to partition the adjacency matrix of the undirected graph into communities. At the same time, a modularity function is configured as a measure of the quality of community partitioning. The community partitioning is performed iteratively until a preset number of iterations is reached or the modularity function tends to stabilize. The community labels corresponding to each variable are extracted from the iterative output of the Leiden community detection algorithm, and arranged in the original sorting order of the variables to form a community label set, so that each community label corresponds one-to-one with the corresponding variable.

5. The method according to claim 1, characterized in that, The process of weighting the connections between variables in the similarity matrix using the community tags, and then binarizing the weighted similarity matrix to obtain a binarized community-aware adjacency matrix, includes: For any two elements corresponding to variables in the similarity matrix, determine whether their community labels are consistent. If the community labels are the same, the element is weighted using a preset intra-community connectivity enhancement factor. If the community labels are different, the element is weighted using a preset inter-community connectivity inhibition factor to obtain a weighted similarity matrix. The numerical distribution of all elements in the weighted similarity matrix is ​​statistically analyzed, and a preset percentage percentile is selected as the binarization threshold. Traverse all elements in the weighted similarity matrix, set elements greater than or equal to the binarization threshold to 1, and set elements less than the binarization threshold to 0, to obtain the binarized community-aware adjacency matrix.

6. The method according to claim 1, characterized in that, The graph autoencoder includes an encoder and a decoder; the step of inputting the initial variable feature matrix and the binarized community-aware adjacency matrix into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix includes: The initial variable feature matrix and the binarized community-aware adjacency matrix are input into the encoder to obtain the variable structure embedding matrix; the encoder consists of a graph convolutional network with a preset number of layers; The variable structure embedding matrix is ​​input into the decoder to reconstruct the variable structure embedding matrix, thereby obtaining the reconstructed community-aware adjacency matrix.

7. The method according to claim 1, characterized in that, The process of fusing the variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of the multivariate time series data through linear projection to obtain a unified input matrix includes: The multivariate time series data is mapped to a preset dimension to obtain variable value embeddings; Encode the time step number of the multivariate time series data to obtain the time step position embedding that is consistent with the embedding dimension of the variable value; The variable structure embedding matrix is ​​linearly mapped and the time dimension is extended to obtain a structure embedding consistent with the variable value embedding dimension. The variable value embedding, time step position embedding, and structure embedding are summed element by element to obtain a unified input matrix.

8. The method according to claim 1, characterized in that, The step of inputting the unified input matrix into a time-series prediction model based on a structure-information-guided attention mechanism, and using this mechanism to model the features of the multivariate time series to obtain the multivariate time-series prediction result for road occupancy includes: Based on the variable structure embedding matrix or the community tag information, construct structural guidance information to characterize the potential structural relationships between variables; The structural guidance information is introduced as additional bias information or weight adjustment factor into the attention score calculation process to guide and adjust the attention allocation among different variables; The attention scores after introducing structural guidance information are normalized to obtain the attention weight matrix guided by structural information. The unified input matrix is ​​weighted and aggregated based on the attention weight matrix to obtain a temporal feature representation of fused structural information; The temporal feature representation of the fused structural information is nonlinearly enhanced by a feedforward neural network, and the enhanced temporal feature representation is linearly predicted to obtain the multivariate temporal prediction result of road occupancy.

9. The method according to claim 1, characterized in that, The multivariate time series prediction method also includes model training, which optimizes model parameters by minimizing the multi-task joint optimization loss, specifically including: The multi-task joint optimization loss is obtained by weighting the reconstruction loss and the temporal prediction loss of the graph autoencoder according to a preset weight; the preset weight includes the reconstruction loss weight coefficient and the temporal prediction loss weight coefficient, both of which are positive numbers and their sum is 1; The reconstruction loss is calculated based on the difference between the binarized community-aware adjacency matrix and the reconstructed community-aware adjacency matrix, and the temporal prediction loss is calculated based on the difference between the multivariate temporal prediction result and the true value. The parameters of the graph autoencoder, the structural information-guided attention mechanism, and the feedforward neural network are iteratively updated using gradient descent-like algorithms until the multi-task joint optimization loss stabilizes or reaches the preset number of training rounds, thus completing model training.

10. A multivariate time series prediction system based on structural representation learning, characterized in that, include: The community-aware adjacency matrix construction unit is used to construct a similarity matrix between variables based on multivariate time series data of road occupancy collected by multiple sensors in the traffic network, obtain the community label of each road occupancy variable through a community detection algorithm, perform weighted processing on the connection relationship between variables in the similarity matrix in combination with the community label, and perform binarization processing on the weighted similarity matrix to obtain a binarized community-aware adjacency matrix. The variable structure embedding extraction unit is used to construct the initial variable feature matrix for multivariate time-series prediction of road occupancy. The initial variable feature matrix and the binarized community-aware adjacency matrix are input into the graph autoencoder to obtain the variable structure embedding matrix and the reconstructed community-aware adjacency matrix. The multi-feature fusion and temporal prediction unit is used to fuse the variable value embedding, time step position embedding, and variable structure embedding matrix obtained by feature extraction of the multivariate time series data after linear projection to obtain a unified input matrix. The unified input matrix is ​​then input into a temporal prediction model based on a structure information-guided attention mechanism. The multivariate time series is modeled using the structure information-guided attention mechanism to obtain the multivariate temporal prediction result of road occupancy. The variable structure embedding matrix participates in the temporal modeling process as an auxiliary feature describing the global structural relationship of variables, without imposing explicit structural constraints on the prediction stage.

Images