Multivariate time series prediction method and system for multi-device federated dynamic graph

By using dynamic graph structure and federated learning methods in multi-device federated scenarios, the problem of difficulty in identifying variable dependency changes is solved, and the accuracy of multivariate time series prediction and model generalization ability are improved.

CN118427562BActive Publication Date: 2024-11-08INFORMATION COMM COMPANY STATE GRID SHANDONG ELECTRIC POWER +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410888175.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-08
Estimated Expiration
2044-07-04

AI Technical Summary

Technical Problem

The existing multivariate time series prediction methods cannot effectively identify the dynamic changes in the dependencies between variables, resulting in a reduction in prediction accuracy, especially in the case of uneven data distribution and missing edge device data in multi-device federal scenarios.

Method used

A multi-device federated dynamic graph method is adopted to generate dynamic graph structures through global graphs and local information, capture the dependence changes between time series, and update model parameters using federated learning and weighted averaging technology to improve prediction accuracy.

Benefits of technology

The dynamic change modeling of time series patterns is realized, the prediction accuracy of multiple time series is improved, and the generalization ability and prediction performance of the model are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118427562B_ABST
    Figure CN118427562B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of time series prediction of IoT devices, and provides a multivariate time series prediction method and system for multi-device federated dynamic graphs. In order to solve the problem that the fixed graph structure cannot recognize the change of the dependency relationship between variables, resulting in the model being unable to learn the correct pattern of the time series, the multivariate time series prediction method for multi-device federated dynamic graphs uses the information of the global graph and the local information of the current time series to generate a graph structure to model the dynamic changes of the dependencies between time series, which can improve the prediction accuracy of multivariate time series.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of time series prediction of Internet of Things devices, and in particular relates to a multivariate time series prediction method and system for a multi-device federated dynamic graph. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The purpose of multivariate time series forecasting technology is to accurately predict future trends of multidimensional time series data. It is widely used in fields such as supply chain material control and environmental monitoring. In the process of multivariate time series forecasting, historical data is first used to build a model, and then the established model is used to predict the values ​​of multiple variables or dimensions at future time points. The data source for building the model includes multivariate time series data. These data usually contain complex relationships between multiple variables, such as temperature, humidity, and wind speed in meteorological data, which increases its challenge.

[0004] Existing time series analysis techniques include ARIMA (autoregressive integrated moving average model), modern machine learning methods such as recurrent neural network (RNN) and long short-term memory network (LSTM). Although these methods can capture patterns and trends in data, they have the defect of being unable to explicitly model the complex relationships between multiple variables.

[0005] In order to reflect the complex relationship between variables, methods based on graph neural networks (GNNs), such as graph deviation networks (GDNs), model multiple variables, i.e., each time series, as nodes in a graph, and model the relationship between variables as edges between nodes. The graph neural network uses its graph structure to update features and fully capture the dependencies between variables. This method is based on a complete centralized data set for training, but in real scenarios, there is often a data set distributed on multiple edge devices, and each device only has part of the time series. In this way, the graph structure obtained by using this method on each edge device has only some nodes, and there are missing edges, lacking the correct dependencies between variables, resulting in reduced prediction performance. In addition, the graph structure obtained by this method is fixed, and the dependencies between variables do not change. However, in real scenarios, the dependencies between variables may change due to situations such as the opening of valves on the device, and the fixed graph structure cannot recognize the change, resulting in the model being unable to learn the correct pattern of the time series, ultimately reducing the accuracy of multivariate time series prediction. Summary of the invention

[0006] In order to solve the problem that the above-mentioned fixed graph structure cannot recognize the change of the dependency relationship between variables, resulting in the model being unable to learn the correct pattern of the time series, and ultimately reducing the prediction accuracy of the multivariate time series, the present invention provides a multivariate time series prediction method and system for a multi-device federated dynamic graph, which utilizes the information of the global graph and the local information of the current time series to generate a graph structure to model the dynamic changes of the dependencies between time series, thereby improving the prediction accuracy of the multivariate time series.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a multivariate time series prediction method for a multi-device federated dynamic graph.

[0009] In one or more embodiments, a multivariate time series prediction method for a multi-device federated dynamic graph is provided, and the following steps are performed in a client:

[0010] Capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain the local feature vector;

[0011] Extract the global features of the local multivariate time series, form a global feature vector and transform it into a global graph; transform the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph;

[0012] Process the forecast graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features;

[0013] Use the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters;

[0014] Upload the global graph, global features and prediction model parameters to the federated server;

[0015] Receive the updated global graph and weighted average prediction model parameters sent by the federation server for subsequent training of the prediction model;

[0016] The trained prediction model is used to process the input multivariate time series to obtain the prediction results.

[0017] In one or more embodiments, a multivariate time series prediction method for a multi-device federated dynamic graph is provided, and the following steps are performed jointly in a client and a federated server:

[0018] The client captures the potential time series representation from the local multivariate time series, extracts the time dependency within the multivariate time series, and obtains the local feature vector;

[0019] The client extracts the global features of the local multivariate time series, forms a global feature vector and converts it into a global graph; converts the local feature vector into a local graph and combines it with the global graph to obtain a prediction graph;

[0020] The client processes the prediction graph and its corresponding local feature vector, extracts the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtains the time series features;

[0021] The client uses the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters;

[0022] The client uploads the global graph, global features, and prediction model parameters to the federated server;

[0023] The federated server updates the global graph based on the global graphs and global features received from all clients, and performs weighted average of all prediction model parameters and sends them to each client for subsequent training of the prediction model.

[0024] The client uses the trained prediction model to process the input multivariate time series to obtain the prediction results.

[0025] A second aspect of the present invention provides a multivariate time series prediction system for multi-device federated dynamic graphs.

[0026] In one or more embodiments, a multivariate time series prediction system for a multi-device federated dynamic graph includes:

[0027] A local feature extraction module is used to capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain a local feature vector;

[0028] A prediction graph calculation module is used to extract the global features of the local multivariate time series, form a global feature vector and convert it into a global graph; convert the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph;

[0029] The time series feature extraction module is used to process the prediction graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features;

[0030] The prediction model parameter update module is used to process the time series features using the prediction model to obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters;

[0031] A federated information upload module, which is used to upload the global graph, global features and prediction model parameters to the federated server;

[0032] A federated information receiving module, which is used to receive the updated global graph and the weighted average prediction model parameters sent by the federated server for subsequent training of the prediction model;

[0033] The time series prediction module is used to process the input multivariate time series using the trained prediction model to obtain the prediction results.

[0034] In other embodiments, a multivariate time series prediction system for a multi-device federated dynamic graph includes: a federated server and a client communicating with the federated server;

[0035] The client is used to:

[0036] Capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain the local feature vector;

[0037] Extract the global features of the local multivariate time series, form a global feature vector and transform it into a global graph; transform the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph;

[0038] Process the forecast graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features;

[0039] Use the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters;

[0040] Upload the global graph, global features and prediction model parameters to the federated server;

[0041] The federated server is used to:

[0042] The global graph and global features of all clients are received to update the global graph, and all prediction model parameters are weighted averaged and sent to each client for subsequent training of the prediction model;

[0043] The client is also used to process the input multivariate time series using the trained prediction model to obtain a prediction result.

[0044] A third aspect of the present invention provides a computer-readable storage medium.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the multivariate time series prediction method for a multi-device federated dynamic graph as described above.

[0046] A fourth aspect of the present invention provides a client.

[0047] A client includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the multivariate time series prediction method for a multi-device federated dynamic graph as described above are implemented.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) The present invention extracts global features from local multivariate time series, forms a global feature vector and converts it into a global graph, and captures its potential time series representation from the local multivariate time series to obtain a local feature vector, converts the local feature vector into a local graph and combines it with the global graph to obtain a prediction graph that can change over time, thereby achieving dynamic change modeling of dependencies between time series, better learning of time series patterns, and improving the prediction accuracy of multivariate time series.

[0050] (2) The present invention adopts a federated learning method to aggregate the local global graph information uploaded by each client, reconstructs the graph according to the node features, generates a federated global graph, and sends it to the client to help the client capture deeper dependencies; the federated averaging method is used to perform weighted averaging of node features and model parameters, which is sent to the client for updating, thereby improving the generalization ability of the prediction model and ultimately improving the prediction accuracy of multivariate time series. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0052] Figure 1 is a schematic diagram of a client according to an embodiment of the present invention;

[0053] Figure 2 It is a flow chart of a multivariate time series prediction method for a multi-device federated dynamic graph described from the client side in an embodiment of the present invention;

[0054] Figure 3It is a flowchart of a multivariate time series prediction method for a multi-device federated dynamic graph described from a mixed side of a client and a federated server in an embodiment of the present invention;

[0055] Figure 4 2 is a schematic diagram of the structure of a multivariate time series prediction system for a multi-device federated dynamic graph corresponding to the embodiment of the present invention;

[0056] Figure 5 The embodiment of the present invention and Figure 3 Corresponding schematic diagram of the multivariate time series prediction system structure for multi-device federated dynamic graphs. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0058] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0059] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0060] Reference Figure 1 , a schematic diagram of a client is given. It should be noted that Figure 1 The client 100 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0061] like Figure 1 As shown, the client 100 includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage part 108 to a random access memory (RAM) 103. In the RAM 103, various programs and data required for system operation are also stored. The central processing unit 101, the ROM 102, and the RAM 103 are connected to each other through a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0062] The following components are connected to the I / O interface 105: an input section 106 including a keyboard, a mouse, etc.; an output section 107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section 109 performs communication processing via a network such as the Internet. A drive 110 is also connected to the I / O interface 105 as needed. A removable medium 111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 110 as needed, so that a computer program read therefrom is installed into the storage section 108 as needed.

[0063] The client 100 mentioned in the embodiment of the present invention may include an Internet device running an APP application, such as an electronic device of a power system, and communicates with a federal server through the Internet.

[0064] The central processing unit 101 in the client of this embodiment implements the following when executing the program: Figure 2 The steps in the multivariate time series prediction method for multi-device federated dynamic graphs are shown.

[0065] Figure 2 is a flow chart of a multivariate time series prediction method for a multi-device federated dynamic graph in an embodiment of the present invention. This embodiment is mainly described from the client side. Figure 2 The multivariate time series prediction method for a multi-device federated dynamic graph in the embodiment shown may include:

[0066] S201, capturing the potential time series representation from the local multivariate time series, extracting the time dependency within the multivariate time series, and obtaining a local feature vector.

[0067] In the specific implementation, a temporal encoder is used to capture the latent time series representation from the local multivariate time series.

[0068] Wherein, the temporal encoder comprises a one-dimensional two-layer convolutional structure and a linear network;

[0069] The convolutional structure can effectively capture the local features of the time series through a multi-level learning process. In this process, the first layer of convolution mainly learns the basic features of the time series, such as trends and mutations. As the number of layers increases, the second layer of convolution further learns more advanced features, such as periodicity and seasonality, through an expanded receptive field. The linear network is used to integrate and map the features extracted by the convolutional layer, converting them into the dimensions and representations required by the model for subsequent processing and prediction.

[0070] The one-dimensional two-layer convolution structure is used to extract the internal time dependency of the multivariate time series and obtain the corresponding vector;

[0071] The linear network is used to map the vector obtained by the one-dimensional two-layer convolution structure to a high-dimensional space to extract high-dimensional temporal features and obtain local feature vectors.

[0072] Specifically, two layers of convolution are used to extract the features of the time series, namely ,in . Indicates The input of a time series, is the current timestamp, and the input of each timestamp contains the past length Historical data, represents the convolution operation, Represents the time from No. The feature vector extracted from a time series is extracted from the original sequence by convolution, so it implies the time dependency between the previous time and the current time in the original sequence and contains the time feature;

[0073] After using the convolutional network to extract the feature vector of the time series, a linear network is used to map the feature vector to a high-dimensional space to extract high-dimensional time features. ,Right now ,in, is the learnable parameter matrix, that is . represents a linear network, Representing time series exist The time corresponds to the node in the graph The local feature vector represents the local features extracted at a certain moment. The feature vector will change dynamically according to the input data at different moments.

[0074] S202, extracting the global features of the local multivariate time series, forming a global feature vector and converting it into a global graph; converting the local feature vector into a local graph and combining it with the global graph to obtain a prediction graph.

[0075] In the specific implementation, a graph structure generator is used to convert the global feature vector into a global graph and the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph.

[0076] In an optional embodiment, other models may be used to convert the global feature vector into a global graph and convert the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph, which will not be described in detail here.

[0077] Specifically, in step S202, the process of extracting the global features of the local multivariate time series, forming a global feature vector and converting it into a global graph is as follows:

[0078] For each time series, generate an embedding vector ,in is the number of time series, Represents the dimension of the embedding space. This embedding variable is initialized by the federated server and represents the time series Corresponding to the nodes in the graph The global feature vector of , which will be continuously optimized and updated through the subsequent federated learning and back propagation process. Therefore, before the client completes a training with a local dataset for federated collaboration and back propagation, the embedding vector will not change. It is therefore called a global vector, through which the global graph of each client is generated.

[0079] It should be noted here that generating a global graph requires calculating the similarity between nodes through the global feature vectors of the nodes to determine whether there is a potential dependency relationship between the nodes, and using the graph structure to explicitly model the dependency relationship. Since the global graph is generated by calculating the global vector, the structure of the graph does not change during a training process of the client's data set, so it is called a global graph.

[0080] The global eigenvectors of all nodes form a matrix , the dependency matrix between nodes .in and For two activation functions, the one-way dependency between nodes can be captured by the antisymmetric matrix. Convert negative results to 0 and use By converting the calculation results into probability form, the dependency relationship between nodes can be expressed as , that is, select node dependencies Middle The row with the largest value Nodes and nodes Establish a one-way relationship. This one-way relationship will be used as a node in the global graph structure Candidate nodes for dependency relationships, .in As a hyperparameter, if Too large may lead to unnecessary connections between nodes, thus generating a lot of noise when the graph neural network learns; if If the value is too small, only a few nodes may be connected, which may cause important dependencies to be missed. The choice needs to be made according to different scenarios. In addition, if the client receives the global graph issued by federated learning, the issued global graph is used as the local global graph, and the local generated global graph dependency Only used for subsequent federated learning.

[0081] In S202, generating a local graph requires calculation based on the local feature vector of the node, and using the time encoder to extract the input of any two time series and The local eigenvector of and Finally, the graph structure generator uses a multi-head attention mechanism to capture the dependencies of local feature variables. and Mapping to different subspace vectors , and , ,Right now , ,in, and are all learnable parameter matrices. Then the final attention score is:

[0082] .in is the number of nodes in the graph, which is also the number of time series. is the score threshold. The part below the threshold will be set to 0, and the part above the threshold will be set to 1. Adding a threshold limit can effectively reduce the noise of the graph structure. That is The adjacency matrix of the local graph structure corresponding to the time series. Since the local vector will change dynamically according to the current time series input, the generated graph structure will also change dynamically with the time series data, so this graph structure is called a local graph;

[0083] The process of converting the local feature vector into a local graph and combining it with the global graph to obtain a prediction graph includes:

[0084] After generating the local graph of the current input, the graph structure generator combines the global graph and the local graph to generate the final prediction graph. Based on the knowledge of the global graph, potential nodes that may have dependencies with the node can be screened out. However, the dependencies between nodes change dynamically, so the global graph cannot be used directly as a prediction graph. By combining the knowledge of the local graph and the global graph, the dependencies between nodes are explicitly modeled according to the dynamic changes of the time series to form a prediction graph. The prediction graph can capture the dynamic changes of the dependencies between nodes and remove the time series dependencies with low correlation, thereby avoiding the noise brought by the nodes with low correlation in the information propagation process during the subsequent graph attention network training. The resulting dynamic graph structure is used to model the spatial dependencies between time series for prediction.

[0085] S203, processing the prediction graph and its corresponding local feature vector, extracting the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtaining the time series features.

[0086] In this embodiment, a graph attention neural network is used to process the prediction graph and its corresponding local feature vector.

[0087] Specifically, the process of using the graph attention neural network to process the prediction graph and its corresponding local feature vector is as follows:

[0088] The graph attention neural network learns the potential spatial dependency from the prediction graph through the graph structure, aggregates the time series feature information of the neighboring nodes corresponding to the local feature vector, obtains the updated node feature information, and forms the time series feature;

[0089] Among them, the time series features integrate the spatial dependencies between multivariate time series and the temporal dependencies of the multivariate time series themselves.

[0090] Specifically, the client generates the graph structure generated by the graph structure generator And all the corresponding time series correspond to the local feature vector The graph attention neural network is sent to the graph attention neural network, and the features are updated through the network. The specific steps are as follows: the graph attention neural network obtains the updated node feature information by aggregating the time series feature information of neighbor nodes through the potential spatial dependency learned by the graph structure ,in is the activation function, is the learnable weight matrix, Representative Node The neighbor nodes used for information aggregation are For Node With Node The attention score is calculated by the formula Calculate, where Representation Node and nodes The attention score, exp represents the exponential function with the natural logarithm e as the base, , and are all learnable weight parameters, is the activation function. After this step, the new time series features fully integrate the spatial dependencies between time series and the temporal dependencies of the time series themselves. The updated features will be used to predict future series through the time series prediction model.

[0091] S204, using the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters.

[0092] Specifically, the prediction model predicts the future time series based on the obtained time series features, and uses the gradient descent method to perform back propagation optimization of the model. The specific steps are as follows:

[0093] The new node time series feature representation generated after the graph attention network Input into the prediction model for prediction, , is the time series forecast value, is the time series step to be predicted, is the number of time series, is the current timestamp, For the updated The time series characteristics of the nodes. As the loss function, ,in For the future time series The gradient is calculated through the loss function to perform back propagation optimization to train a model that can correctly capture the time series pattern.

[0094] S205, uploading the global graph, global features and prediction model parameters to the federation server.

[0095] S206, receiving the updated global graph and the weighted average prediction model parameters sent by the federal server for subsequent training of the prediction model.

[0096] The updated global map sent by the federated server is:

[0097] The global graph is completed by the federated server based on the global graphs uploaded by all clients.

[0098] Among them, the weighted average prediction model parameters sent by the federated server are:

[0099] The federation server uses the federation averaging collaborative method to perform weighted averaging on the prediction model parameters uploaded by all clients.

[0100] After back propagation updates the model parameters, each client trains the local time series dataset to generate the global graph and the corresponding global node embedding vector And the updated model parameters after back propagation Upload to the federated learning server for collaborative learning. After the federated learning server collects the data sent by the client, the server uses the federated average collaborative method to embed the nodes uploaded by all clients into vectors. Perform weighted fusion to get global node embedding , is the total number of clients, For Clients The total number of nodes in the uploaded graph, is the total number of nodes of all clients, through To measure the client The importance of node embedding. The more nodes a client has, the more data it has, and the deeper dependencies it can capture. The embedding vector obtained also contains more information and should have a higher influence. Then use global node embedding The global graph is constructed in the same process as the global graph structure generation process of the client's graph structure generator. The global feature vectors of all nodes Make a matrix , the dependency matrix between nodes .in and As the activation function, the one-way dependency between nodes can be captured by the antisymmetric matrix and used The calculation results are converted into probability form. Then the dependency relationship between nodes can be expressed as , that is, select node dependencies Middle The row with the largest value Nodes and nodes Establish a one-way relationship. Build a global picture It is sent to the client as a new client global graph, which combines the global graph knowledge of multiple clients, making up for the problem of missing nodes or missing edges of dependency relationships between nodes caused by missing data sets when generating a global graph under the original single client. In addition, the model parameters are also weighted averaged using the same method as global node embedding. ,in is the total number of clients, For Clients The total number of nodes in the uploaded graph, is the total number of nodes of all clients. The more nodes a client has, the more data it has and can capture deeper dependencies. Correspondingly, its model parameters should have a larger weight. Then it is sent to the client to update the model parameters, improve the generalization ability of the model, and make the model converge faster.

[0101] S207, using the trained prediction model to process the input multivariate time series to obtain a prediction result.

[0102] A specific example is given below to illustrate Figure 2 Application scenarios of the methods shown:

[0103] In environmental monitoring scenarios, especially in tourism, agriculture and some manufacturing industries, accurate prediction of future weather changes is critical for important decision-making. Traditional weather forecasting relies on multiple meteorological observation stations, each of which records a variety of meteorological characteristics, which form a multivariate time series. The present invention predicts weather characteristics in the future by analyzing the time series data obtained from different observation stations.

[0104] In this example, each meteorological observation station is regarded as a client. The model is first trained locally on the client. The weather characteristics such as temperature, humidity, air pressure, wind speed, cloud cover, and precipitation obtained by the meteorological station constitute a multivariate time series. The local feature vector is extracted through the process described in S201. , which represents the characteristics of the current sequence, and obtains the prediction graph structure through S202 , the graph structure contains spatial dependencies between sequences. For example, in this weather station, there is a certain connection between the temperature and cloudiness sequences, which is manifested as spatial dependencies in the graph structure. After that, through S203 graph neural network learning, new features that integrate the spatial dependencies between sequences and the temporal dependencies of the sequences themselves are obtained. Through S204, the weather characteristics of the future Q time are predicted, and the model loss function is calculated by comparing the predicted values ​​with the actual observed weather characteristic values.

[0105] After each weather station client goes through S201-S205, it uploads the parameters to the federated server for training through S205, and the federated server aggregates the model parameters through S206 and sends them to each weather station. After the training is completed, weather forecasting is performed through S207, and the forecast result obtained is the multivariate time series data of the future period with the same feature number as the weather station input.

[0106] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, the computer program including a computer program for executing Figure 2 In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 109, and / or installed from the removable medium 111. When the computer program is executed by the central processing unit 101, various functions defined in the apparatus of the present application are executed.

[0107] in, Figure 2 The computer program instructions corresponding to the method shown may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0109] Figure 3 It is a flowchart of a multivariate time series prediction method for a multi-device federated dynamic graph in another embodiment of the present invention. In this embodiment, first, the data set of each client is uniformly divided into a training set, a validation set, and a test set, and the ratio of the three is 6:2:2. The model of each client is trained and back-propagated on the training set, the loss function is evaluated on the validation set, and the hyperparameters of the model are adjusted. The final model is tested on the test set to obtain the final performance of the model. In addition, the time series data set input to each client is normalized and scaled to the range of 0 to 1. ,in, The input of the federated time series prediction framework for dynamic graphs, is the original data that has not been normalized. and are the minimum and maximum values ​​of the original data respectively. Normalizing the input data can speed up the convergence of model training. The federated service randomly initializes the model parameters and the embedding vector of the node The model parameters and embedding vectors are continuously optimized during the model training process.

[0110] This embodiment is mainly described from the perspective of the client and the federated server. Figure 3 The multivariate time series prediction method for a multi-device federated dynamic graph in this embodiment may include:

[0111] S301, the client captures the potential time series representation from the local multivariate time series, extracts the time dependency within the multivariate time series, and obtains a local feature vector.

[0112] S302, the client extracts the global features of the local multivariate time series, forms a global feature vector and converts it into a global graph; converts the local feature vector into a local graph and combines it with the global graph to obtain a prediction graph.

[0113] S303, the client processes the prediction graph and its corresponding local feature vector, extracts the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtains the time series features;

[0114] S304, the client uses the prediction model to process the time series features, obtains the future multi-step time series values, calculates the loss function, back-propagates and updates the prediction model parameters;

[0115] S305, the client uploads the global graph, global features and prediction model parameters to the federated server;

[0116] S306, the federated server updates the global graph according to the global graphs and global features received from all clients, and weighted averages all prediction model parameters and sends them to each client for subsequent training of the prediction model;

[0117] S307, the client uses the trained prediction model to process the input multivariate time series to obtain a prediction result.

[0118] The specific implementation process of the client in S301 to S307 of this embodiment is the same as that in the above Figure 2 The specific implementation process is the same as that in the above, and will not be described in detail here.

[0119] Figure 4is a schematic diagram of a multivariate time series prediction system for a multi-device federated dynamic graph in an embodiment of the present invention. Figure 1 This corresponds to the multivariate time series prediction method for multi-device federated dynamic graphs, such as Figure 4 As shown, the multivariate time series prediction system for multi-device federated dynamic graphs in this embodiment may include:

[0120] A local feature extraction module 401 is used to capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain a local feature vector;

[0121] In the local feature extraction module 401, a time encoder is used to capture the potential time series representation from the local multivariate time series.

[0122] The temporal encoder includes a one-dimensional two-layer convolutional structure and a linear network;

[0123] The one-dimensional two-layer convolution structure is used to extract the internal time dependency of the multivariate time series and obtain the corresponding vector;

[0124] The linear network is used to map the vector obtained by the one-dimensional two-layer convolution structure to a high-dimensional space to extract high-dimensional temporal features and obtain local feature vectors.

[0125] A prediction graph calculation module 402 is used to extract global features of the local multivariate time series, form a global feature vector and convert it into a global graph; convert the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph;

[0126] In the prediction graph calculation module 402, a graph structure generator is used to convert the global feature vector into a global graph and the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph.

[0127] The time series feature extraction module 403 is used to process the prediction graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the time dependency of the multivariate time series itself, and obtain the time series features.

[0128] In the time series feature extraction module 403, a graph attention neural network is used to process the prediction graph and its corresponding local feature vector.

[0129] Specifically, the process of using the graph attention neural network to process the prediction graph and its corresponding local feature vector is as follows:

[0130] The graph attention neural network learns the potential spatial dependency from the prediction graph through the graph structure, aggregates the time series feature information of the neighboring nodes corresponding to the local feature vector, obtains the updated node feature information, and forms the time series feature;

[0131] Among them, the time series features integrate the spatial dependencies between multivariate time series and the temporal dependencies of the multivariate time series themselves.

[0132] The prediction model parameter updating module 404 is used to process the time series features using the prediction model to obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters.

[0133] The federation information uploading module 405 is used to upload the global graph, global features and prediction model parameters to the federation server.

[0134] The federation information receiving module 406 is used to receive the updated global graph and the weighted average prediction model parameters sent by the federation server for subsequent training of the prediction model.

[0135] Wherein, in the federation information receiving module 406, the updated global graph sent by the federation server is received as follows:

[0136] The global graph is completed by the federated server based on the global graphs uploaded by all clients.

[0137] In the federation information receiving module 406, the weighted average prediction model parameters received from the federation server are:

[0138] The federation server uses the federation averaging collaborative method to perform weighted averaging on the prediction model parameters uploaded by all clients.

[0139] The time series prediction module 407 is used to process the input multivariate time series using the trained prediction model to obtain a prediction result.

[0140] Figure 4 The specific implementation process of modules 401 to 407 in the multivariate time series prediction system for multi-device federated dynamic graphs is shown in FIG. Figure 2 The specific implementation process of steps S201 to 207 is the same and will not be described in detail here.

[0141] Figure 5 1 is a schematic diagram of the structure of a multivariate time series prediction system for a multi-device federated dynamic graph in an embodiment of the present invention. The multivariate time series prediction system for a multi-device federated dynamic graph in this embodiment may include a federated server 500 and a client 100 communicating with the federated server 500;

[0142] The client 100 is used for:

[0143] Capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain the local feature vector;

[0144] Extract the global features of the local multivariate time series, form a global feature vector and transform it into a global graph; transform the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph;

[0145] Process the forecast graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features;

[0146] Use the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters;

[0147] Upload the global graph, global features and prediction model parameters to the federated server;

[0148] The federal server 500 is used for:

[0149] The global graph and global features of all clients are received to update the global graph, and all prediction model parameters are weighted averaged and sent to each client for subsequent training of the prediction model;

[0150] The client 100 is also used to process the input multivariate time series using the trained prediction model to obtain a prediction result.

[0151] It should be noted here that the specific implementation process of the client of this embodiment is similar to Figure 2 The specific implementation process is the same as that in the above, and will not be described in detail here.

[0152] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multivariate time series prediction method for multi-device federated dynamic graphs, characterized in that: Perform the following steps in the client: Capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain the local feature vector; Extract the global features of the local multivariate time series, form a global feature vector and transform it into a global graph; transform the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph; Process the forecast graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features; Use the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters; upload the global graph, global features and prediction model parameters to the federal server; receive the updated global graph and weighted average prediction model parameters sent by the federal server for subsequent training of the prediction model; use the trained prediction model to process the input multivariate time series to obtain the prediction result; A time encoder is used to capture the potential time series representation from the local multivariate time series; the time encoder includes a one-dimensional two-layer convolution structure and a linear network; the one-dimensional two-layer convolution structure is used to extract the internal time dependency of the multivariate time series to obtain the corresponding vector; the linear network is used to map the vector obtained by the one-dimensional two-layer convolution structure to a high-dimensional space to extract high-dimensional time features and obtain a local feature vector; Using a graph structure generator, the global feature vector is converted into a global graph and the local feature vector is converted into a local graph and combined with the global graph to obtain a prediction graph; The global eigenvectors of all nodes form a matrix , the dependency matrix between nodes , the dependency relationship between nodes is expressed as , select node dependencies Middle The row with the largest value Nodes and nodes Establish a unidirectional relationship, which will be used as a node in the global graph structure Candidate nodes for dependency relationships, , the client receives the global graph sent by the federated server, and uses the sent global graph as the local global graph. The local generated global graph dependency Only used for subsequent federated learning; Generating a local graph requires calculation based on the local feature vector of the node, using the time encoder to extract the input of any two time series and The local eigenvector of and , the graph structure generator uses a multi-head attention mechanism to capture the dependencies of local feature variables, and the final attention score is: ,in is the number of nodes in the graph, which is also the number of time series. is the score threshold, for The adjacency matrix of the local graph structure corresponding to the time series, the generated graph structure is a local graph; After generating the local graph of the current input, the graph structure generator combines the global graph and the local graph to generate the final prediction graph. , The dynamic graph structure generated for prediction; The graph attention neural network is used to process the prediction graph and its corresponding local feature vector. The process is as follows: the graph attention neural network learns the potential spatial dependency from the prediction graph through the graph structure, aggregates the time series feature information of the neighbor nodes corresponding to the local feature vector, obtains the updated node feature information, and forms the time series feature; among them, the time series feature integrates the spatial dependency between multivariate time series and the time dependency of the multivariate time series itself.

2. The multivariate time series prediction method for multi-device federated dynamic graphs according to claim 1, characterized in that: The updated global map sent by the federated server is: The global graph is completed by the federated server based on the global graphs uploaded by all clients.

3. The multivariate time series prediction method for multi-device federated dynamic graphs according to claim 1, characterized in that: The weighted average prediction model parameters received from the federated server are: The federation server uses the federation averaging collaborative method to perform weighted averaging on the prediction model parameters uploaded by all clients.

4. A multivariate time series prediction system for multi-device federated dynamic graphs, characterized in that: Its clients include: A local feature extraction module is used to capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain a local feature vector; A prediction graph calculation module is used to extract the global features of the local multivariate time series, form a global feature vector and convert it into a global graph; convert the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph; The time series feature extraction module is used to process the prediction graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features; The prediction model parameter update module is used to process the time series features using the prediction model to obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters; A federated information upload module, which is used to upload the global graph, global features and prediction model parameters to the federated server; A federated information receiving module, which is used to receive the updated global graph and the weighted average prediction model parameters sent by the federated server for subsequent training of the prediction model; The time series prediction module is used to process the input multivariate time series using the trained prediction model to obtain the prediction results; A time encoder is used to capture the potential time series representation from the local multivariate time series; the time encoder includes a one-dimensional two-layer convolution structure and a linear network; the one-dimensional two-layer convolution structure is used to extract the internal time dependency of the multivariate time series to obtain the corresponding vector; the linear network is used to map the vector obtained by the one-dimensional two-layer convolution structure to a high-dimensional space to extract high-dimensional time features and obtain a local feature vector; Using a graph structure generator, the global feature vector is converted into a global graph and the local feature vector is converted into a local graph and combined with the global graph to obtain a prediction graph; The global eigenvectors of all nodes form a matrix , the dependency matrix between nodes , the dependency relationship between nodes is expressed as , select node dependencies Middle The row with the largest value Nodes and nodes Establish a unidirectional relationship, which will be used as a node in the global graph structure Candidate nodes for dependency relationships, When the federated information receiving module receives the global graph sent by the federated server, it uses the sent global graph as the local global graph. The local generated global graph dependency Only used for subsequent federated learning; Generating a local graph requires calculation based on the local feature vector of the node, using the time encoder to extract the input of any two time series and The local eigenvector of and , the graph structure generator uses a multi-head attention mechanism to capture the dependencies of local feature variables, and the final attention score is: ,in is the number of nodes in the graph, which is also the number of time series. is the score threshold, for The adjacency matrix of the local graph structure corresponding to the time series, the generated graph structure is a local graph; After generating the local graph of the current input, the graph structure generator combines the global graph and the local graph to generate the final prediction graph. , The dynamic graph structure generated for prediction; The graph attention neural network is used to process the prediction graph and its corresponding local feature vector. The process is as follows: the graph attention neural network learns the potential spatial dependency from the prediction graph through the graph structure, aggregates the time series feature information of the neighbor nodes corresponding to the local feature vector, obtains the updated node feature information, and forms the time series feature; among them, the time series feature integrates the spatial dependency between multivariate time series and the time dependency of the multivariate time series itself.

5. The multivariate time series prediction system for multi-device federated dynamic graphs as claimed in claim 4, characterized in that: The updated global map sent by the federated server is: The global graph is completed by the federated server based on the global graphs uploaded by all clients.

6. The multivariate time series prediction system for multi-device federated dynamic graphs as claimed in claim 4, characterized in that: The weighted average prediction model parameters received from the federated server are: The federation server uses the federation averaging collaborative method to perform weighted averaging on the prediction model parameters uploaded by all clients.

7. A multivariate time series prediction method for multi-device federated dynamic graphs, characterized in that: include: The client captures the potential time series representation from the local multivariate time series, extracts the time dependency within the multivariate time series, and obtains the local feature vector; The client extracts the global features of the local multivariate time series, forms a global feature vector and converts it into a global graph; converts the local feature vector into a local graph and combines it with the global graph to obtain a prediction graph; The client processes the prediction graph and its corresponding local feature vector, extracts the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtains the time series features; The client uses the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters; The client uploads the global graph, global features, and prediction model parameters to the federated server; The federated server updates the global graph based on the global graphs and global features received from all clients, and performs weighted average of all prediction model parameters and sends them to each client for subsequent training of the prediction model. The client uses the trained prediction model to process the input multivariate time series and obtain the prediction results; A time encoder is used to capture the potential time series representation from the local multivariate time series; the time encoder includes a one-dimensional two-layer convolution structure and a linear network; the one-dimensional two-layer convolution structure is used to extract the internal time dependency of the multivariate time series to obtain the corresponding vector; the linear network is used to map the vector obtained by the one-dimensional two-layer convolution structure to a high-dimensional space to extract high-dimensional time features and obtain a local feature vector; Using a graph structure generator, the global feature vector is converted into a global graph and the local feature vector is converted into a local graph and combined with the global graph to obtain a prediction graph; The global eigenvectors of all nodes form a matrix , the dependency matrix between nodes , the dependency relationship between nodes is expressed as , select node dependencies Middle The row with the largest value Nodes and nodes Establish a unidirectional relationship, which will be used as a node in the global graph structure Candidate nodes for dependency relationships, , the client receives the global graph sent by the federated server, and uses the sent global graph as the local global graph. The local generated global graph dependency Only used for subsequent federated learning; Generating a local graph requires calculation based on the local feature vector of the node, using the time encoder to extract the input of any two time series and The local eigenvector of and , the graph structure generator uses a multi-head attention mechanism to capture the dependencies of local feature variables, and the final attention score is: ,in is the number of nodes in the graph, which is also the number of time series. is the score threshold, for The adjacency matrix of the local graph structure corresponding to the time series, the generated graph structure is a local graph; After generating the local graph of the current input, the graph structure generator combines the global graph and the local graph to generate the final prediction graph. , The dynamic graph structure generated for prediction; The graph attention neural network is used to process the prediction graph and its corresponding local feature vector. The process is as follows: the graph attention neural network learns the potential spatial dependency from the prediction graph through the graph structure, aggregates the time series feature information of the neighbor nodes corresponding to the local feature vector, obtains the updated node feature information, and forms the time series feature; among them, the time series feature integrates the spatial dependency between multivariate time series and the time dependency of the multivariate time series itself.

8. A multivariate time series prediction system for multi-device federated dynamic graphs, characterized in that: include: The federated server and the clients that communicate with it; The client is used to: Capture the potential time series representation from the local multivariate time series, extract the time dependency within the multivariate time series, and obtain the local feature vector; Extract the global features of the local multivariate time series, form a global feature vector and transform it into a global graph; transform the local feature vector into a local graph and combine it with the global graph to obtain a prediction graph; Process the forecast graph and its corresponding local feature vector, extract the spatial dependency between local multivariate time series and the temporal dependency of the multivariate time series itself, and obtain the time series features; Use the prediction model to process the time series features, obtain the future multi-step time series values, calculate the loss function, back-propagate and update the prediction model parameters; The global map, global features and prediction model parameters are uploaded to the federal server; the federal server is used to update the global map according to the global maps and global features received from all clients, and to weighted average all prediction model parameters and send them to each client for subsequent training of the prediction model; The client is also used to process the input multivariate time series using the trained prediction model to obtain a prediction result; A time encoder is used to capture the potential time series representation from the local multivariate time series; the time encoder includes a one-dimensional two-layer convolution structure and a linear network; the one-dimensional two-layer convolution structure is used to extract the internal time dependency of the multivariate time series to obtain the corresponding vector; the linear network is used to map the vector obtained by the one-dimensional two-layer convolution structure to a high-dimensional space to extract high-dimensional time features and obtain a local feature vector; Using a graph structure generator, the global feature vector is converted into a global graph and the local feature vector is converted into a local graph and combined with the global graph to obtain a prediction graph; The global eigenvectors of all nodes form a matrix , the dependency matrix between nodes , the dependency relationship between nodes is expressed as , select node dependencies Middle The row with the largest value Nodes and nodes Establish a unidirectional relationship, which will be used as a node in the global graph structure Candidate nodes for dependency relationships, , the client receives the global graph sent by the federated server, and uses the sent global graph as the local global graph. The local generated global graph dependency Only used for subsequent federated learning; Generating a local graph requires calculation based on the local feature vector of the node, using the time encoder to extract the input of any two time series and The local eigenvector of and , the graph structure generator uses a multi-head attention mechanism to capture the dependencies of local feature variables, and the final attention score is: ,in is the number of nodes in the graph, which is also the number of time series. is the score threshold, for The adjacency matrix of the local graph structure corresponding to the time series, the generated graph structure is a local graph; After generating the local graph of the current input, the graph structure generator combines the global graph and the local graph to generate the final prediction graph. , The dynamic graph structure generated for prediction; The graph attention neural network is used to process the prediction graph and its corresponding local feature vector. The process is as follows: the graph attention neural network learns the potential spatial dependency from the prediction graph through the graph structure, aggregates the time series feature information of the neighbor nodes corresponding to the local feature vector, obtains the updated node feature information, and forms the time series feature; among them, the time series feature integrates the spatial dependency between multivariate time series and the time dependency of the multivariate time series itself.

9. The multivariate time series prediction system for multi-device federated dynamic graphs as claimed in claim 8, characterized in that: The updated global map sent by the federated server is: The global graph is completed by the federated server based on the global graphs uploaded by all clients.

10. The multivariate time series prediction system for multi-device federated dynamic graphs according to claim 8, characterized in that: The weighted average prediction model parameters received from the federated server are: The federation server uses the federation averaging collaborative method to perform weighted averaging on the prediction model parameters uploaded by all clients.