Space-time sequence prediction method
By combining sliding window sampling, multi-head attention mechanism, hypergraph network and gated recurrent unit, the problem of difficult modeling of complex correlation relationships between nodes in spatiotemporal sequence prediction is solved, and high-precision spatiotemporal sequence prediction is achieved.
Patent Information
- Application Number
- CN202510690542.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-27
- Publication Date
- 2025-10-03
AI Technical Summary
Existing spatiotemporal series prediction methods find it difficult to effectively capture the complex correlation relationships between nodes, and rarely consider the joint modeling of long-term and short-term time series, resulting in insufficient accuracy and robustness of the prediction results, making it difficult to meet high-precision requirements.
Sliding window sampling is used to generate long-time subsequence samples, and the multi-head attention mechanism is used to capture hidden representations. A learnable hypergraph network is constructed to update node and hyperedge information. The gated recursive unit is combined to process short-time subsequences, and K-means clustering is used to assign independent feedforward neural layers to different clusters for prediction.
It significantly improves the accuracy and robustness of spatiotemporal series predictions, comprehensively considers the complex correlation information of long and short time series, and improves the accuracy and reliability of predictions.
Smart Images
Figure CN120744345A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence time series prediction and relates to a spatiotemporal sequence prediction method. Background Art
[0002] Spatiotemporal data is a type of data that contains temporal and spatial information and can describe how things change at different times and places. The prediction of spatiotemporal data has many application scenarios and can be used in many fields such as weather forecasting and traffic flow prediction. However, many spatiotemporal series prediction scenarios in the real world involve a large amount of node data, and there are complex and diverse interactions and influence relationships between nodes. Traditional methods have difficulty accurately capturing and modeling the complex correlations between nodes, which limits the accuracy of the prediction results. On the other hand, although existing deep learning methods have improved the performance of time series prediction to a certain extent, they rarely consider the joint modeling of long and short time series, and cannot meet the requirements of high precision and strong robustness for time series prediction in actual application scenarios. Summary of the Invention
[0003] In view of this, the purpose of the present invention is to provide a spatiotemporal series prediction method, which, by integrating multiple technologies such as sliding window sampling, multi-head attention mechanism, hypergraph network and gated recurrent unit, performs deep modeling and joint analysis on multi-node long time series and short time series data, thereby overcoming the difficulties faced by existing technologies in processing complex spatiotemporal series data, significantly improving the accuracy and reliability of spatiotemporal series prediction, and effectively meeting the growing high-precision demand for spatiotemporal series prediction.
[0004] In order to achieve the above object, the present invention provides the following technical solutions:
[0005] A spatiotemporal sequence prediction method, characterized in that the method comprises the following steps:
[0006] S1, using the sliding window sampling method to process the original time series of each node to generate long time subsequence samples;
[0007] S2, randomly mask the sample and obtain the sample encoding vector through the fully connected layer;
[0008] S3. Capture the hidden representation of long-time subsequence samples based on the multi-head attention mechanism to obtain a pre-trained model;
[0009] S4. Obtain the incidence matrix of the hypergraph based on the node representation matrix and the hyperedge representation matrix;
[0010] S5. Update node information;
[0011] S6. Update the hyperedge information;
[0012] S7. Define hyperedge constraint function and node constraint function;
[0013] S8, inputting the short-time subsequence sample into the gated recurrent unit to obtain the hidden representation of the short-time subsequence sample;
[0014] S9. Cluster the nodes based on the K-means algorithm, input the hidden representations of the long-time subsequence samples and the short-time subsequence samples into the feedforward neural layer to obtain the prediction results of the time series.
[0015] Furthermore, in step S1, the sliding window sampling method is used to process the original time series of each node to generate long time subsequence samples. Specifically, define S∈R T×N×C is the original time series, where T is the total number of time steps, N is the number of nodes, and C is the number of channels for each node; define S i ∈R T×C is the original time series of node i, 1≤i≤N, for S i The sliding window sampling method is used for processing. The window length is set to L and the sliding step is K. Starting from the starting point of the time series, a subsequence of length L is intercepted as a sample, and the window is slid backward for K time steps. The process is repeated until the complete subsequence cannot be intercepted. The long time subsequence sample of node i can be obtained [S i,1 ,…,S i,p ,…,S i,P ], where S i,p represents the p-th long time subsequence sample of node i, 1≤p≤P, where P represents the number of long time subsequence samples.
[0016] Furthermore, in step S2, the sample is randomly masked and the encoding vector of the sample is obtained through the fully connected layer. Specifically, let M i,p is the matrix S i,p The mask matrix is defined as follows:
[0017]
[0018] Among them, 1 L×C represents an L×C dimensional matrix of all 1s, 0 L×C represents an L×C dimensional all-zero matrix, p′ is a given constant, 0≤p′≤1;
[0019] Perform mask processing on the long time subsequence samples of node i, let Represents the long time subsequence sample after masking, the formula is as follows:
[0020]
[0021] Where ⊙ represents element-by-element multiplication;
[0022] Will Flattened into a one-dimensional vector of 1×(L×C), let Represents the flattened one-dimensional vector, The input fully connected layer is converted into a 1×d dimensional embedding vector, is the output of the fully connected layer, and the formula is as follows:
[0023]
[0024] where W1∈R (L×C)×d is a learnable weight matrix, b1∈R 1×d is a learnable bias vector;
[0025] PE i,p for Position encoding, PE i,p,j for The encoding of the j-th position is as follows:
[0026]
[0027] in Definition U i,p is the encoding vector, and the formula is as follows:
[0028]
[0029] Furthermore, in step S3, the hidden representation of long-time subsequence samples is captured based on the multi-head attention mechanism to obtain a pre-training model. Specifically, let U i Represents the input matrix of the multi-head attention mechanism of node i, and the formula is as follows:
[0030] U i =[U i,1 ,…,U i,p ,…,U i,P ]∈R P×d
[0031] definition are the query matrix, key matrix, and value matrix of the n-th head in the multi-head attention mechanism, respectively. The formula is as follows:
[0032]
[0033] in Denotes the learnable weight matrix of the nth head in the multi-head attention mechanism, N′ denotes the number of attention heads; define H i,n The hidden representation of the long subsequence sample output by the n-th attention head is as follows:
[0034]
[0035] The hidden representations of each attention head output are concatenated and linearly transformed to obtain the output of the fusion of N attention head hidden representations. The formula is as follows:
[0036] H i =Concat(H i,1 ,…,H i,n ,…,H i,N )W 0
[0037] Where Concat(·) represents the matrix concatenation operation, W 0 is a learnable weight matrix;
[0038] H i Enter the fully connected layer, let Y i represents the output of the fully connected layer, and the formula is as follows:
[0039] Y i =H i W2+b2
[0040] The learnable parameters are updated based on the optimization of the loss function to obtain a pre-trained model. Specifically, the loss function of the pre-trained model is defined as follows:
[0041]
[0042] where Y i,p Y i The p-th subsequence sample in , 1≤p≤P, is minimized by the loss function and the learnable variables are updated until the model converges.
[0043] Further, in step S4, the incidence matrix of the hypergraph is obtained based on the node representation matrix and the hyperedge representation matrix, specifically, defining E n ∈R N×C and E g ∈R M×C are the learnable node representation matrix and hyperedge representation matrix respectively, where M is the number of hyperedges;
[0044] Define A∈R N×M is the incidence matrix of the hypergraph, and the calculation formula is as follows:
[0045]
[0046] Let A′ denote the sparsified incidence matrix, and let a′ i,j Represents the element in row i and column j in A′, and the formula is as follows:
[0047]
[0048] where a i,j represents the element in row i and column j in A, and β is the sparsification threshold;
[0049] Furthermore, in step S5, the node information is updated, specifically, defining D v,i and D e,j They represent the degree of node i and the degree of hyperedge j respectively, and the formulas are as follows:
[0050]
[0051] Definition D v ∈R N×N and D e ∈R M×M are the degree matrices of nodes and hyperedges, respectively, and are defined as follows:
[0052] D v =diag(D v,1 ,…,D v,i ,…,D v,N ),1≤i≤N
[0053] D e =diag(D e,1 ,…,D e,j ,…,D e,M ),1≤j≤M
[0054] Define S′ P =[S 1,P ,…,S i,P ,…,S N,P ] is the input matrix of the hypergraph convolutional network, and S′ is calculated based on the hypergraph convolution formula. P To update, the formula is as follows:
[0055]
[0056] where S′ P,k represents the node embedding representation of the kth layer, and S′ P,0 =S′ P , σ(·) is the activation function, Θ k-1 is the k-1th layer learnable variable, K is the number of layers of the hypergraph convolutional network, let X P,K =[X 1,P,K ,…,X i,P,K ,…,X N,P,K ] represents the output of the hypergraph convolutional network, 1≤i≤N.
[0057] Furthermore, in step S6, the hyperedge information is updated, specifically, defining For X i,P,K The flattened one-dimensional vector, e jis the embedded representation of hyperedge j, and the formula is as follows:
[0058]
[0059] Further, in step S7, the hyperedge constraint function and the node constraint function are defined, specifically, is the cosine similarity between hyperedge i and hyperedge j, and the formula is as follows:
[0060]
[0061] Define L g is the hyperedge constraint function, and its formula is as follows:
[0062]
[0063] Definition D j is the sum of the Euclidean distances between hyperedge j and the nodes it connects to, and the formula is as follows:
[0064]
[0065] Define L n is the node constraint function, and the formula is as follows:
[0066]
[0067] Furthermore, in step S8, the short-time subsequence samples are input into the gated recursive unit to obtain the hidden representation of the short-time subsequence samples. Specifically, define X i,P,K is the short time subsequence sample of node i, let h i Represents the hidden representation of the short-time subsequence sample of node i, and X i,P,K The data of each time step in the process is sequentially input into the gated recurrent unit to obtain h i , the formula is as follows:
[0068] r i,j =σ(W r ·[h i,j-1 ,X′ i,j ])
[0069] z i,j =σ(W z ·[h i,j-1 ,X′ i,j ])
[0070]
[0071] h i =h i,L
[0072] where X′ i,j Represents Xi,P,K The data of the jth time step in , 1≤j≤L, r i,j represents the state of the reset gate of node i at the jth time step, z i,j represents the state of the update gate of node i at the jth time step, represents the candidate hidden state of node i at the jth time step, h i,j represents the hidden state of node i at the jth time step, W r 、W z , W0 is a learnable parameter.
[0073] Furthermore, in step S9, the nodes are clustered based on the K-means algorithm, and the hidden representations of the long-time subsequence samples and the short-time subsequence samples are input into the feedforward neural layer to obtain the prediction results of the time series. Specifically, the nodes are divided into Q clusters by the K-means algorithm, and a separate feedforward neural layer is assigned to each cluster to predict the time series. Denotes the feedforward neural layer used by the qth cluster, 1≤q≤Q, let represents the output of the feedforward neural layer, and the formula is as follows:
[0074]
[0075] where H′ i,P Indicates H i The vector of the Pth row in ;
[0076] Define L s is the prediction loss function, defined as follows:
[0077]
[0078] Among them S i,P+1 is the true value, and is defined is the final loss function, defined as follows:
[0079]
[0080] Among them, k1 and k2 are the weight coefficients of node loss and hyperedge loss. The model is trained based on minimizing the loss function. After the model converges, the time series can be predicted.
[0081] The beneficial effects of the present invention are as follows: the present invention is oriented to the spatiotemporal sequence prediction task, and utilizes multi-node historical time series with spatiotemporal information to predict future spatiotemporal sequence conditions, and proposes a spatiotemporal sequence prediction method based on sliding window sampling, multi-head attention mechanism, hypergraph network, gated recursive unit and K-means clustering. The present invention generates long-time subsequence samples through sliding window sampling, and utilizes multi-head attention mechanism to capture the hidden representation of long-time subsequence samples, constructs a learnable hypergraph, and realizes the update of node and hyperedge information based on the hypergraph convolutional network, combines the gated recursive unit to process short-time subsequence samples, captures the hidden representation of short-time subsequence samples, and allocates independent feedforward neural layers to nodes of different clusters for prediction through K-means clustering to improve the accuracy of the prediction. The present invention comprehensively considers the historical patterns of long-time series and the dynamic changes of short-time series, effectively integrates the complex correlation information between nodes, and significantly improves the robustness and accuracy of spatiotemporal sequence prediction.
[0082] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0083] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:
[0084] Figure 1 This is a schematic diagram of the spatiotemporal sequence prediction proposed by the present invention;
[0085] Figure 2 Schematic diagram of the flow of spatiotemporal series prediction method. DETAILED DESCRIPTION
[0086] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0087] See also Figures 1 and 2 , Figure 1 This is a schematic diagram of the spatiotemporal sequence prediction proposed by the present invention, as shown in Figure 1As shown in the figure, this method generates long-time subsequence samples through sliding window sampling, and uses a multi-head attention mechanism to capture the hidden representation of long-time subsequence samples, constructs a learnable hypergraph, and updates the node and hyperedge information based on the hypergraph convolutional network. It combines the gated recursive unit to process short-time subsequence samples, captures the hidden representation of short-time subsequence samples, and assigns independent feedforward neural layers to nodes in different clusters for prediction through K-means clustering.
[0088] Figure 2 This is a flow chart of the spatiotemporal series prediction method, as shown in Figure 2 As shown, the method specifically includes the following steps:
[0089] Step 1: Use the sliding window sampling method to process the original time series of each node to generate long time subsequence samples;
[0090] The sliding window sampling method is used to process the original time series of each node to generate long time subsequence samples. Specifically, define S∈R T×N×C is the original time series, where T is the total number of time steps, N is the number of nodes, and C is the number of channels for each node; define S i ∈R T×C is the original time series of node i, 1≤i≤N, for S i The sliding window sampling method is used for processing. The window length is set to L and the sliding step is K. Starting from the starting point of the time series, a subsequence of length L is intercepted as a sample, and the window is slid backward for K time steps. The process is repeated until the complete subsequence cannot be intercepted. The long time subsequence sample of node i can be obtained [S i,1 ,…,S i,p ,…,S i,P ], where S i,p represents the p-th long time subsequence sample of node i, 1≤p≤P, where P represents the number of long time subsequence samples.
[0091] Step 2: Randomly mask the sample and obtain the sample encoding vector through the fully connected layer;
[0092] The sample is randomly masked and the encoding vector of the sample is obtained through the fully connected layer. Specifically, let M i,p is the matrix S i,p The mask matrix is defined as follows:
[0093]
[0094] Among them, 1 L×C represents an L×C dimensional matrix of all 1s, 0 L×C represents an L×C dimensional all-zero matrix, p′ is a given constant, 0≤p′≤1;
[0095] Perform mask processing on the long time subsequence samples of node i, let Represents the long time subsequence sample after masking, the formula is as follows:
[0096]
[0097] Where ⊙ represents element-by-element multiplication;
[0098] Will Flattened into a one-dimensional vector of 1×(L×C), let Represents the flattened one-dimensional vector, The input fully connected layer is converted into a 1×d dimensional embedding vector, is the output of the fully connected layer, and the formula is as follows:
[0099]
[0100] where W1∈R (L×C)×d is a learnable weight matrix, b1∈R 1×d is a learnable bias vector;
[0101] PE i,p for Position encoding, PE i,p,j for The encoding of the j-th position is as follows:
[0102]
[0103] in Definition U i,p is the encoding vector, and the formula is as follows:
[0104]
[0105] Step 3: Capture the hidden representation of long subsequence samples based on the multi-head attention mechanism to obtain a pre-trained model;
[0106] Based on the multi-head attention mechanism to capture the hidden representation of long-time subsequence samples, the pre-training model is obtained. Specifically, let U i Represents the input matrix of the multi-head attention mechanism of node i, and the formula is as follows:
[0107] U i =[U i,1 ,…,U i,p ,…,U i,P ]∈R P×d
[0108] definition are the query matrix, key matrix, and value matrix of the n-th head in the multi-head attention mechanism, respectively. The formula is as follows:
[0109]
[0110] in Denotes the learnable weight matrix of the nth head in the multi-head attention mechanism, N′ denotes the number of attention heads; define H i,n The hidden representation of the long subsequence sample output by the n-th attention head is as follows:
[0111]
[0112] The hidden representations of each attention head output are concatenated and linearly transformed to obtain the output of the fusion of N attention head hidden representations. The formula is as follows:
[0113] H i =Concat(H i,1 ,…,H i,n ,…,H i,N )W 0
[0114] Where Concat(·) represents the matrix concatenation operation, W 0 is a learnable weight matrix;
[0115] H i Enter the fully connected layer, let Y i represents the output of the fully connected layer, and the formula is as follows:
[0116] Y i =H i W2+b2
[0117] The learnable parameters are updated based on the optimization of the loss function to obtain a pre-trained model. Specifically, the loss function of the pre-trained model is defined as follows:
[0118]
[0119] where Y i,p Y i The p-th subsequence sample in , 1≤p≤P, is minimized by the loss function and the learnable variables are updated until the model converges.
[0120] Step 4: Obtain the incidence matrix of the hypergraph based on the node representation matrix and the hyperedge representation matrix;
[0121] Based on the node representation matrix and the hyperedge representation matrix, the incidence matrix of the hypergraph is obtained, specifically: Define E n ∈R N×C and Eg ∈R M×C are the learnable node representation matrix and hyperedge representation matrix respectively, where M is the number of hyperedges;
[0122] Define A∈R N×M is the incidence matrix of the hypergraph, and the calculation formula is as follows:
[0123]
[0124] Let A′ denote the sparsified incidence matrix, and let a′ i,j Represents the element in row i and column j in A′, and the formula is as follows:
[0125]
[0126] where a i,j represents the element in row i and column j in A, and β is the sparsification threshold.
[0127] Step 5: Update the node information;
[0128] Update the node information, specifically: define D v,i and D e,j They represent the degree of node i and the degree of hyperedge j respectively, and the formulas are as follows:
[0129]
[0130] Definition D v ∈R N×N and D e ∈R M×M are the degree matrices of nodes and hyperedges, respectively, and are defined as follows:
[0131] D v =diag(D v,1 ,…,D v,i ,…,D v,N ),1≤i≤N
[0132] D e =diag(D e,1 ,…,D e,j ,…,D e,M ),1≤j≤M
[0133] Define S′ P =[S 1,P ,…,S i,P ,…,S N,P ] is the input matrix of the hypergraph convolutional network, and S′ is calculated based on the hypergraph convolution formula. P To update, the formula is as follows:
[0134]
[0135] where S′ P,k represents the node embedding representation of the kth layer, and S′ P,0 =S′ P , σ(·) is the activation function, Θ k-1 is the k-1th layer learnable variable, K is the number of layers of the hypergraph convolutional network, let X P,K =[X 1,P,K ,…,X i,P,K ,…,X N,P,K ] represents the output of the hypergraph convolutional network, 1≤i≤N.
[0136] Step 6: Update the hyperedge information;
[0137] Update the hyperedge information, specifically: define For X i,P,K The flattened one-dimensional vector, e j is the embedded representation of hyperedge j, and the formula is as follows:
[0138]
[0139] Step 7: Define the hyperedge constraint function and node constraint function;
[0140] Define the hyperedge constraint function and node constraint function, specifically: define is the cosine similarity between hyperedge i and hyperedge j, and the formula is as follows:
[0141]
[0142] Define L g is the hyperedge constraint function, and its formula is as follows:
[0143]
[0144] Definition D j is the sum of the Euclidean distances between hyperedge j and the nodes it connects to, and the formula is as follows:
[0145]
[0146] Define L n is the node constraint function, and the formula is as follows:
[0147]
[0148] Step 8: Input the short-time subsequence sample into the gated recurrent unit to obtain the hidden representation of the short-time subsequence sample;
[0149] The short-time subsequence samples are input into the gated recurrent unit to obtain the hidden representation of the short-time subsequence samples. Specifically, define Xi,P,K is the short time subsequence sample of node i, let h i Represents the hidden representation of the short-time subsequence sample of node i, and X i,P,K The data of each time step in the process is sequentially input into the gated recurrent unit to obtain h i , the formula is as follows:
[0150] r i,j =σ(W r ·[h i,j-1 ,X′ i,j ])
[0151] z i,j =σ(W z ·[h i,j-1 ,X′ i,j ])
[0152]
[0153] h i =h i,L
[0154] where X′ i,j Represents X i,P,K The data of the jth time step in , 1≤j≤L, r i,j represents the state of the reset gate of node i at the jth time step, z i,j represents the state of the update gate of node i at the jth time step, represents the candidate hidden state of node i at the jth time step, h i,j represents the hidden state of node i at the jth time step, W r 、W z , W0 is a learnable parameter.
[0155] Step 9: Cluster the nodes based on the K-means algorithm, input the hidden representations of the long-time subsequence samples and the short-time subsequence samples into the feedforward neural layer to obtain the prediction results of the time series;
[0156] Based on the K-means algorithm, the nodes are clustered and the hidden representations of long-time subsequence samples and short-time subsequence samples are input into the feedforward neural layer to obtain the prediction results of the time series. Specifically, the nodes are divided into Q clusters by the K-means algorithm, and a separate feedforward neural layer is assigned to each cluster to predict the time series. Denotes the feedforward neural layer used by the qth cluster, 1≤q≤Q, let represents the output of the feedforward neural layer, and the formula is as follows:
[0157]
[0158] where H′ i,PIndicates H i The vector of the Pth row in ;
[0159] Define L s is the prediction loss function, defined as follows:
[0160]
[0161] Among them S i,P+1 is the true value, and is defined is the final loss function, defined as follows:
[0162]
[0163] Among them, k1 and k2 are the weight coefficients of node loss and hyperedge loss. The model is trained based on minimizing the loss function. After the model converges, the time series can be predicted.
[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.
Claims
1. A spatiotemporal sequence prediction method, characterized by: The method comprises the following steps: S1, using the sliding window sampling method to process the original time series of each node to generate long time subsequence samples; S2, randomly mask the sample and obtain the sample encoding vector through the fully connected layer; S3. Capture the hidden representation of long-time subsequence samples based on the multi-head attention mechanism to obtain a pre-trained model; S4. Obtain the incidence matrix of the hypergraph based on the node representation matrix and the hyperedge representation matrix; S5. Update node information; S6. Update the hyperedge information; S7. Define hyperedge constraint function and node constraint function; S8, inputting the short-time subsequence sample into the gated recurrent unit to obtain the hidden representation of the short-time subsequence sample; S9. Cluster the nodes based on the K-means algorithm, input the hidden representations of the long-time subsequence samples and the short-time subsequence samples into the feedforward neural layer to obtain the prediction results of the time series.
2. The method for time series prediction according to claim 1, wherein: In step S1, the sliding window sampling method is used to process the original time series of each node to generate long time subsequence samples. Specifically, define S∈R T×N×C is the original time series, where T is the total number of time steps, N is the number of nodes, and C is the number of channels for each node; define S i ∈R T×C is the original time series of node i, 1≤i≤N, for S i The sliding window sampling method is used for processing. The window length is set to L and the sliding step is K. Starting from the starting point of the time series, a subsequence of length L is intercepted as a sample, and the window is slid backward for K time steps. The process is repeated until the complete subsequence cannot be intercepted. The long time subsequence sample of node i can be obtained [S i,1 ,...,S i,p ,...,S i,P ], where S i,p represents the p-th long time subsequence sample of node i, 1≤p≤P, where P represents the number of long time subsequence samples.
3. The method for time series prediction according to claim 1, wherein: In step S2, the sample is randomly masked and the encoding vector of the sample is obtained through the fully connected layer. Specifically, let M i,p is the matrix S i,p The mask matrix is defined as follows: Among them, 1 L×C represents an L×C dimensional matrix of all 1s, 0 L×C represents an L×C dimensional all-zero matrix, p′ is a given constant, 0≤p′≤1; Perform mask processing on the long time subsequence samples of node i, let Represents the long time subsequence sample after masking, the formula is as follows: Where ⊙ represents element-by-element multiplication; Will Flattened into a one-dimensional vector of 1×(L×C), let Represents the flattened one-dimensional vector, The input fully connected layer is converted into a 1×d dimensional embedding vector, is the output of the fully connected layer, and the formula is as follows: where W1∈R (L×C)×d is a learnable weight matrix, b1∈R 1×d is a learnable bias vector; PE i,p for Position encoding, PE i,p,j for The encoding of the j-th position is as follows: in Definition U i,p is the encoding vector, and the formula is as follows:
4. The method for time series prediction according to claim 1, wherein: In step S3, the hidden representation of long-time subsequence samples is captured based on the multi-head attention mechanism to obtain a pre-training model. Specifically, let U i Represents the input matrix of the multi-head attention mechanism of node i, and the formula is as follows: IN i =[U i,1 ,…,IN i,p ,…,IN i,P ]∈R P×d definition are the query matrix, key matrix, and value matrix of the n-th head in the multi-head attention mechanism, respectively. The formula is as follows: in Denotes the learnable weight matrix of the nth head in the multi-head attention mechanism, N′ denotes the number of attention heads; define H i,n The hidden representation of the long subsequence sample output by the n-th attention head is as follows: The hidden representations of each attention head output are concatenated and linearly transformed to obtain the output of the fusion of N attention head hidden representations. The formula is as follows: H i =Concat(H i,1 ,...,H i,n ,...,H i,N )W 0 Where Concat(·) represents the matrix concatenation operation, W 0 is a learnable weight matrix; H i Enter the fully connected layer, let Y i represents the output of the fully connected layer, and the formula is as follows: <h2 style=";text-align:left;direction:ltr">Y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =H<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> W2+b2 The learnable parameters are updated based on the optimization of the loss function to obtain a pre-trained model. Specifically, the loss function of the pre-trained model is defined as follows: where Y i,p Y i The p-th subsequence sample in , 1≤p≤P, is minimized by the loss function and the learnable variables are updated until the model converges.
5. The method for time series prediction according to claim 1, wherein: In step S4, the incidence matrix of the hypergraph is obtained based on the node representation matrix and the hyperedge representation matrix. Specifically, define E n ∈R N×C and E g ∈R M×C are the learnable node representation matrix and hyperedge representation matrix respectively, where M is the number of hyperedges; Define A∈R N×M is the incidence matrix of the hypergraph, and the calculation formula is as follows: Let A′ denote the sparsified incidence matrix, and let a′ i,j Represents the element in row i and column j in A′, and the formula is as follows: where a i,j represents the element in row i and column j in A, and β is the sparsification threshold.
6. The method for time series prediction according to claim 1, wherein: In step S5, the node information is updated, specifically, defining D v,i and D e,j They represent the degree of node i and the degree of hyperedge j respectively, and the formulas are as follows: Definition D v ∈R N×N and D e ∈R M×M are the degree matrices of nodes and hyperedges, respectively, and are defined as follows: D v =diag(D v,1 ,...,D v,i ,...,D v,N ),1≤i≤N D e =diag(D e,1 ,...,D e,j ,...,D e,M ),1≤j≤M Define S′ P =[S 1,P ,...,S i,P ,...,S N,P ] is the input matrix of the hypergraph convolutional network, and S′ is calculated based on the hypergraph convolution formula. P To update, the formula is as follows: where S′ P,k represents the node embedding representation of the kth layer, and S′ P,0 =S′ P , σ(·) is the activation function, Θ k-1 is the k-1th layer learnable variable, K is the number of layers of the hypergraph convolutional network, let X P,K =[X 1,P,K ,...,X i,P,K ,...,X N,P,K ] represents the output of the hypergraph convolutional network, 1≤i≤N.
7. The method for time series prediction according to claim 1, wherein: In step S6, the hyperedge information is updated, specifically, defining For X i,P,K The flattened one-dimensional vector, e j is the embedded representation of hyperedge j, and the formula is as follows:
8. The method for time series prediction according to claim 1, wherein: In step S7, define the hyperedge constraint function and node constraint function, specifically, define is the cosine similarity between hyperedge i and hyperedge j, and the formula is as follows: Define L g is the hyperedge constraint function, and its formula is as follows: Definition D j is the sum of the Euclidean distances between hyperedge j and the nodes it connects to, and the formula is as follows: Define L n is the node constraint function, and the formula is as follows:
9. The method for time series prediction according to claim 1, wherein: In step S8, the short-time subsequence samples are input into the gated recursive unit to obtain the hidden representation of the short-time subsequence samples. Specifically, define X i,P,K is the short time subsequence sample of node i, let h i Represents the hidden representation of the short-time subsequence sample of node i, and X i,P,K The data of each time step in the process is sequentially input into the gated recurrent unit to obtain h i , the formula is as follows: r i,j =σ(W r ·[h i,j-1 ,X′ i,j ]) z i,j =σ(W z ·[h i,j-1 ,X′ i,j ]) h i =h i,L where X′ i,j Represents X i,P,K The data of the jth time step in , 1≤j≤L, r i,j represents the state of the reset gate of node i at the jth time step, z i,j represents the state of the update gate of node i at the jth time step, represents the candidate hidden state of node i at the jth time step, h i,j represents the hidden state of node i at the jth time step, W r 、W z , W0 is a learnable parameter.
10. The method for time series prediction according to claim 1, wherein: In step S9, the nodes are clustered based on the K-means algorithm, and the hidden representations of the long-time subsequence samples and the short-time subsequence samples are input into the feedforward neural layer to obtain the prediction results of the time series. Specifically, the nodes are divided into Q clusters by the K-means algorithm, and a separate feedforward neural layer is assigned to each cluster to predict the time series. Denotes the feedforward neural layer used by the qth cluster, 1≤q≤Q, let represents the output of the feedforward neural layer, and the formula is as follows: where H′ i,P Indicates H i The vector of the Pth row in ; Define L s is the prediction loss function, defined as follows: Among them S i,P+1 is the true value, and is defined is the final loss function, defined as follows: Among them, k1 and k2 are the weight coefficients of node loss and hyperedge loss. The model is trained based on minimizing the loss function. After the model converges, the time series can be predicted.