An extrapolation model for multivariate time series analysis and a training method thereof

Abstract

The application provides an extrapolation model for multivariate time series analysis and a training method thereof. The model comprises: a data acquisition module for acquiring to-be-extrapolated time series data and metadata of all sensors, the to-be-extrapolated time series data comprising time series data of each sensor within a predetermined time period, the all sensors comprising a target sensor missing all time series data and the time series data of the target sensor being replaced by a preset missing value; a metadata graph modeling module for extracting a dynamic space-time relationship between each sensor based on the to-be-extrapolated time series data and the metadata, and constructing a metadata graph according to the dynamic space-time relationship; an encoder for encoding the to-be-extrapolated time series data based on a predefined graph and the metadata graph to obtain an encoding result; a decoder for decoding the encoding result based on the predefined graph and the metadata graph to obtain a decoding result; and an aggregation layer for aggregating the encoding result and the decoding result to obtain an extrapolation result; wherein the trained extrapolation model is obtained by using an adversarial training method.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, specifically, to the field of multivariate time series extrapolation technology, and more specifically, to an extrapolation model for multivariate time series analysis and a training method thereof. Background Art

[0002] Multivariate time series extrapolation is the process of filling missing values ​​in time series data. The goal is to infer the time series of sensors with missing time series data by leveraging known sensor time series data and spatial relationships within the scene. This task is very common in real-world applications, such as transportation, energy, and meteorology, where cost constraints or the sudden demand for new locations may necessitate using known data and spatial information to infer missing data elements.

[0003] There are currently two main types of solutions for multivariate time series extrapolation technology, including traditional methods based on trajectory similarity research and extrapolation based on deep learning models.

[0004] Traditional methods, such as Kriging and matrix decomposition, analyze and fit data based on mathematical methods, often requiring the data to meet stringent conditions such as unbiasedness or stationarity. However, in real-world extrapolation scenarios, there are often complex spatial correlations or randomness between sensors, making it difficult to achieve satisfactory extrapolation results.

[0005] Among deep learning models, deep learning-based multivariate time series extrapolation methods have recently garnered significant attention. For example, the IGNNK model, the first deep learning model specifically targeting this field, stacks multiple graph convolutional networks (GCNs) to facilitate inter-sensor message passing. The SATCN model builds on the IGNNK model by adding a dilated causal convolutional network (TCN) to analyze time series patterns and proposes multiple predefined graph variants to compensate for the lack of prior spatial information. The GRIN model is generally considered a superior interpolation model. Its core concept is to analyze forward and backward temporal patterns with the help of a gated recurrent neural network (GRU) and apply a spatial network based on a graph convolutional network (GCN) to perform in-depth inference. The INCREASE model requires manual filtering of information from multiple spatially closest nodes to aggregate information from unknown sensors. It also proposes leveraging metadata, such as points of interest (POIs), to construct predefined graph relationships, achieving promising results.

[0006] Although the above-mentioned multi-variate time series extrapolation method achieves good extrapolation effect, there is still room for improvement in many aspects, and it is difficult to apply in real scenarios. Existing works usually fail to effectively capture the timestamp drift between sensors, which is particularly critical when transferring time information between sensors, as the physical distance may cause a delay in information recording. Considering that extrapolation mainly relies on the spatial relationship between sensors, some models optimize the static spatial dependency, i.e., optimize the pre-defined graph to improve the extrapolation performance. However, if there is a problem in the pre-defined graph that the spatial dependency between two sensors is inaccurately represented or ignored due to the lack of interaction or error accumulation in graph learning, the extrapolation performance will also decrease. For example, the SATCN model and the INCREASE model in the existing method, although they propose to construct a pre-defined graph based on prior knowledge, they cannot perceive the spatial relationship in the scene, so they cannot adaptively learn the spatial relationship in the scene, which will lead to their inability to obtain more accurate spatial relationship and their inability to aggregate valuable information according to the spatial relationship.

[0007] Therefore, the existing extrapolation model cannot adaptively learn and perceive the spatial relationship in the scene, which leads to the problem that it cannot obtain effective spatial relationship, so it cannot obtain more accurate extrapolation results.

[0008] It should be noted that the background art is only used to introduce the relevant information of the present application, so as to help understand the technical solutions of the present application, but does not mean that the relevant information must be the prior art. In the absence of evidence that the relevant information has been disclosed before the filing date of the present application, the relevant information should not be regarded as prior art. SUMMARY

[0009] Therefore, the purpose of the present application is to overcome the defects of the above-mentioned prior art, and to provide an extrapolation model for multi-variate time series analysis and a training method thereof.

[0010] The purpose of the present application is achieved by the following technical solutions:

[0011] According to a first aspect of the present application, an extrapolation model for multivariate time series analysis is provided, the model comprising: a data acquisition module configured to acquire time series data to be extrapolated of all sensors in a scene within a same predetermined time period, a predefined graph and metadata, the time series data to be extrapolated comprising time series data of each sensor within the predetermined time period, wherein the target sensor with missing all time series data is included in all sensors, the time series data of the target sensor is replaced by a preset missing value, the metadata comprises the geographical position of each sensor and the timestamp corresponding to each data in the time series data, and the predefined graph comprises the initial spatial relationship between each sensor in the scene; a metadata graph modeling module configured to extract the dynamic spatio-temporal relationship between each sensor based on the time series data to be extrapolated and the metadata, and construct a metadata graph according to the dynamic spatio-temporal relationship; an encoder configured to encode the time series data to be extrapolated based on the predefined graph and the metadata graph by using a neural network to obtain an encoding result; a decoder configured to decode the encoding result based on the predefined graph and the metadata graph to obtain a decoding result; and an aggregation layer configured to aggregate the encoding result and the decoding result to obtain an extrapolation result, including the time series data of the target sensor within the predetermined time period.

[0012] In some embodiments of the present application, the encoder comprises: a first graph neural network configured to process the time series data to be extrapolated based on the predefined graph by using a graph isomorphism network after masking parameters to obtain time series information to be extrapolated; a decoupling phase module configured to decouple the time series information to be extrapolated, convert the result after decoupling into the amplitude spectrum and the phase spectrum corresponding to each sensor, and encode the metadata graph, the amplitude spectrum and the phase spectrum corresponding to each sensor to obtain the encoding result.

[0013] In some embodiments of the present application, the metadata graph modeling module comprises:

[0014] an embedding unit configured to perform embedding processing on the geographical position and the timestamp of each sensor in the metadata to obtain the spatial embedding information and the time embedding information corresponding to each sensor;

[0015] a Fourier transform unit configured to perform Fourier transform operation on the time series information to be extrapolated to obtain the amplitude spectrum information corresponding to each sensor;

[0016] a feature extraction unit configured to extract the dynamic spatio-temporal relationship between each sensor according to the spatial embedding information, the time embedding information and the amplitude spectrum information corresponding to each sensor;

[0017] a metadata graph construction unit configured to construct the metadata graph based on the dynamic spatio-temporal relationship, the metadata graph comprising a plurality of nodes representing sensors and a plurality of edges representing the dynamic spatio-temporal relationship between each sensor.

[0018] In some embodiments of the application, the feature extraction unit comprises:

[0019] a concatenation layer configured to concatenate the time embedding information and the amplitude spectrum information corresponding to each sensor to obtain concatenation information corresponding to each sensor;

[0020] a perception machine layer configured to perform linear processing on the concatenation information corresponding to each sensor to obtain a linear processing result corresponding to each sensor;

[0021] a multi-head attention layer configured to perform attention calculation on the spatial embedding information, the time embedding information and the amplitude spectrum information corresponding to each sensor by using a multi-head attention mechanism to obtain an attention score corresponding to each sensor;

[0022] a fully connected network layer configured to obtain an initial adjacency matrix from the linear processing result and the attention score corresponding to each sensor, wherein the elements of the initial adjacency matrix are initial spatio-temporal relationship features between sensors;

[0023] a feature extraction layer configured to retain a preset number of values arranged in descending order in each row of the initial adjacency matrix, and set the remaining values to 0, to obtain dynamic spatio-temporal relationships between sensors.

[0024] In some embodiments of the application, the decoupling phase module comprises:

[0025] a decoupling layer configured to perform decoupling processing on the to-be-interpolated time series information to obtain a time series decoupling result, including a trend value and a residual value corresponding to each sensor;

[0026] a Fourier transform layer configured to convert the trend value corresponding to each sensor by Fourier transform to obtain a first amplitude spectrum and a first phase spectrum corresponding to each sensor, and convert the residual value corresponding to each sensor by Fourier transform to obtain a second amplitude spectrum and a second phase spectrum corresponding to each sensor;

[0027] an embedding layer configured to perform embedding processing on the first phase spectrum and the second phase spectrum corresponding to each sensor respectively to obtain first phase embedding information and second phase embedding information corresponding to each sensor;

[0028] a graph isomorphism network layer configured to process the first phase embedding information and the second phase embedding information corresponding to each sensor by using a graph isomorphism network based on a metadata graph to obtain first phase features and second phase features corresponding to each sensor;

[0029] a reconstruction layer configured to perform inverse Fourier transform on the first amplitude spectrum and the first phase features corresponding to each sensor to obtain reconstructed trend values corresponding to each sensor, and perform inverse Fourier transform on the second amplitude spectrum and the second phase features corresponding to each sensor to obtain reconstructed residual values corresponding to each sensor;

[0030] The splicing layer is configured to splice the reconstructed trend value and the reconstructed residual value corresponding to each sensor to obtain an encoding result.

[0031] In some embodiments of the present application, the decoder comprises a plurality of decoding units, each decoding unit being a double-flow structure comprising:

[0032] The multi-layer second graph neural network is configured to process the encoding result based on the predefined graph and the metadata graph by using a multi-layer graph isomorphic network to obtain a first processing result.

[0033] The multi-layer third graph neural network is configured to process the encoding result based on the predefined graph and the metadata graph by using a multi-layer graph isomorphic network with the masked parameters to obtain a second processing result.

[0034] The decoding result is obtained by aggregating the first processing result and the second processing result corresponding to each decoding unit.

[0035] According to a second aspect of the present application, a training method of an extrapolation model for multivariate time series analysis is provided, comprising:

[0036] obtaining a training set comprising a plurality of samples, each sample comprising input data and a label sequence, the input data being the to-be-extrapolated time series data of all sensors in a same predetermined time period, a predefined graph and metadata, and the label sequence comprising a first label and a second label, the first label being used to indicate whether all time series data of a sensor is missing in the predetermined time period, and the second label being the real time series data of the sensor missing all time series data in the predetermined time period;

[0037] In a preset round, a pre-trained discriminator is introduced, and the extrapolation model of the first aspect of the present application is trained by using the discriminator and the training set, comprising obtaining an extrapolation result by the extrapolation model according to the input data, and obtaining a discrimination result by the discriminator according to the encoding result of the encoder, the discrimination result comprising whether all time series data of a sensor is missing in the predetermined time period, and updating the parameters of the extrapolation model and the discriminator according to a preset total loss function, wherein the total loss function comprises a sub-loss function for calculating the difference between the discrimination result and the first label, and a sub-loss function for calculating the difference between the extrapolation result and the second label.

[0038] In each round after the preset round, only the training set is used to train the extrapolation model, comprising obtaining an extrapolation result by the extrapolation model according to the input data, and updating the parameters of the extrapolation model to minimize the difference between the extrapolation result and the second label as an optimization target to obtain a trained extrapolation model.

[0039] According to a third aspect of the present application, an extrapolation method for multi-variate time series analysis is provided for extrapolating a missing full time series data of a sensor in a traffic scenario, a meteorological monitoring scenario or an energy monitoring scenario to obtain an extrapolation result, the method comprising: obtaining the time series data to be extrapolated of all sensors in the scenario in a same predetermined time period, a predefined graph and metadata; and obtaining the extrapolation result according to the input time series data to be extrapolated, the predefined graph and the metadata by using a trained extrapolation model obtained based on the method according to the first aspect of the present application.

[0040] According to a fourth aspect of the present application, an electronic device is provided, comprising: one or more processors; and a memory, wherein the memory is configured to store executable instructions; and the one or more processors are configured to implement the steps of the method according to the first aspect and / or the second aspect of the present application by executing the executable instructions.

[0041] Compared with the prior art, the present application has the following advantages:

[0042] In the model of the present application, a metadata graph modeling module is provided, which is configured to extract the dynamic spatio-temporal relationship between sensors by using a neural network based on the time series data to be extrapolated and the metadata, and to construct a metadata graph according to the dynamic spatio-temporal relationship. The model encodes and decodes the time series data to be extrapolated based on the metadata graph and a predefined graph to obtain an extrapolation result. The metadata graph modeling module proposed in the present application can adaptively learn the spatio-temporal relationship in the scenario based on the deep learning method, without the need for manual graph construction by professionals, and can better and more accurately capture the spatio-temporal relationship and be applicable to any scenario. On the other hand, the metadata graph modeling module can perceive the spatio-temporal relationship in the scenario through the neural network, thereby helping the model to more accurately aggregate valuable information according to the spatio-temporal relationship to improve the accuracy of the extrapolation result. BRIEF DESCRIPTION OF DRAWINGS

[0043] The embodiments of the present application are further described below with reference to the accompanying drawings, in which:

[0044] Figure 1 A schematic diagram of the structure principle of the extrapolation model for multi-variate time series analysis according to the embodiments of the present application;

[0045] Figure 2 A schematic diagram of the processing principle of the metadata graph modeling module according to the embodiments of the present application;

[0046] Figure 3 A schematic diagram of the processing principle of the decoupling phase module according to the embodiments of the present application;

[0047] Figure 4 A schematic diagram of the structure of the model of the present application and its data processing principle according to the embodiments of the present application;

[0048] Figure 5 Fig. 1 is a schematic diagram of a principle of training an extrapolation model using a discriminator and a training set according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely intended to explain the present application and should not be used to limit the present application.

[0050] As mentioned in the background section, the existing extrapolation model cannot adaptively learn and perceive the spatial relationship in the scene, resulting in the problem that it cannot obtain effective spatial relationship, and thus cannot obtain more accurate extrapolation results.

[0051] Based on the above problems, the inventors propose an extrapolation model for multivariate time series analysis. The model extrapolates the target sensor with missing time series data in the scene according to the time series data to be extrapolated of all sensors in the scene within a predetermined time period, a predefined graph and metadata, and obtains an extrapolation result, including the time series data of the target sensor extrapolated within the predetermined time period. Wherein, the inventors found that the existing problem of being unable to obtain effective spatial relationship is due to the fact that the existing research fails to simulate dynamic spatial relationship, and it pays insufficient attention to the generalization of spatiotemporal patterns extrapolated to unknown nodes, while dynamic spatiotemporal relationship is proved to be particularly important in spatiotemporal tasks. Therefore, the model of the present application sets up a separate metadata graph modeling module, which is used to extract the dynamic spatiotemporal relationship between sensors based on the time series data to be extrapolated and the metadata using a neural network, and construct a metadata graph according to the dynamic spatiotemporal relationship. The model encodes and decodes the time series data to be extrapolated based on the metadata graph and a predefined graph constructed according to the spatial distance between sensors to obtain an extrapolation result. The metadata graph modeling module proposed by the present application can adaptively learn the spatiotemporal relationship in the scene based on the deep learning method, which is different from the method of constructing a predefined graph by interest points and spatial coordinates in the existing method, and the method of changing the original graph structure through other graph theory data. The present application not only does not need professional personnel to manually construct a graph, but also realizes more accurate and convenient capture of spatiotemporal relationship, which is suitable for any scene. On the other hand, the metadata graph modeling module can perceive the spatiotemporal relationship in the scene through a neural network, thereby helping the model to more accurately aggregate valuable information according to the spatiotemporal relationship and improve the accuracy of the extrapolation result.

[0052] According to an embodiment of the present application, referring to Figure 1, which is an extrapolation model structure principle diagram for multivariate time series analysis. The model comprises a data acquisition module, a metadata graph modeling module, an encoder, a decoder and an aggregation layer. Taking a scenario comprising N sensors and the time series data dimension of the sensors in a predetermined time period as L as an example, the processing principles of each part of the model are described. The data acquisition module is configured to acquire all the sensors in the scenario in the same predetermined time period, the pre-defined graph and the metadata of the to-be-extrapolated time series data, the to-be-extrapolated time series data is N*L dimension, which comprises the time series data of each sensor in the predetermined time period, wherein all the sensors comprise a target sensor with missing time series data, the time series data of the target sensor is replaced by a preset missing value, the metadata comprises the geographical position of each sensor and the timestamp corresponding to each data in the time series data, and the pre-defined graph is N*N dimension, which comprises the initial spatial relationship between the sensors in the scenario; the metadata graph modeling module is configured to extract the dynamic space-time relationship between the sensors based on the to-be-extrapolated time series data and the metadata by using a neural network, and construct a metadata graph based on the dynamic space-time relationship, which is N*N dimension; the encoder is configured to encode the to-be-extrapolated time series data based on the pre-defined graph and the metadata graph by using a neural network to obtain an encoding result, which is N*L dimension; the decoder is configured to decode the encoding result based on the pre-defined graph and the metadata graph to obtain a decoding result, which is N*L dimension; and the aggregation layer is configured to aggregate the encoding result and the decoding result to obtain an extrapolation result, which is N*L dimension, comprising the time series data of the target sensor in the predetermined time period. The technical scheme of the embodiment can at least achieve the following beneficial technical effects: the model learns the timestamp and geographical position of the recorded time series data in the sensor scenario, and then learns the space-time information related to the sensors and establishes the dynamic space-time relationship between the sensors, thereby effectively sensing the space-time relationship between the sensors in the scenario, improving the accuracy of the extrapolation result, and allowing any number of sensors to be expanded in the extrapolation task.

[0053] In order to better understand the present application, the model structure, model training and application scenarios are described in detail below in combination with specific embodiments.

[0054] I. Model structure

[0055] (1) Data acquisition module

[0056] According to one embodiment of the present application, the module is used to obtain all the sensors in the scene within the same predetermined time period, the to-be-interpolated timing data, the predefined graph and the metadata, the to-be-interpolated timing data including the timing data of each sensor within the predetermined time period, wherein the target sensor missing all timing data is included in all sensors, the timing data of the target sensor is replaced by a preset missing value, and the metadata includes the geographical position of each sensor and the timestamp corresponding to each data in the timing data. Illustratively, the dimension of the timing data of the sensor within the predetermined time period is denoted as L, if the preset missing value is 0, the timing data of the target sensor is composed of L 0 elements, the total number of sensors is denoted as N, wherein the number of target sensors missing all timing data is denoted as N M .

[0057] According to one embodiment of the present application, the predefined graph refers to a non-Euclidean graph structure of a sensor in a sensor scene, which is usually calculated by the Euclidean distance between sensors and a threshold. The construction method of the predefined graph includes: calculating the distance between any two sensors according to the geographical positions of the two sensors, and connecting the two sensors to construct an edge according to a preset threshold when the distance between the two sensors is less than the preset threshold, that is, the two sensors are close nodes, so as to obtain the predefined graph. The predefined graph can also be directly given by the downstream task. The present application does not limit this.

[0058] (2) Metadata graph modeling module

[0059] According to one embodiment of the present application, the metadata graph modeling module includes: an embedding unit, configured to perform embedding processing on the geographical positions and timestamps of each sensor in the metadata, to obtain spatial embedding information and time embedding information corresponding to each sensor; a Fourier transform unit, configured to perform Fourier transform operation on the to-be-interpolated timing information, to obtain amplitude spectrum information corresponding to each sensor; a feature extraction unit, configured to extract dynamic space-time relationship between each sensor according to the spatial embedding information, the time embedding information and the amplitude spectrum information corresponding to each sensor; and a metadata graph construction unit, configured to construct a metadata graph based on the dynamic space-time relationship, which includes a plurality of nodes representing sensors and a plurality of edges representing dynamic space-time relationship between each sensor. The technical scheme of the embodiment can at least achieve the following beneficial technical effects: in the metadata graph modeling process, in addition to relying on the to-be-interpolated timing information, some metadata such as timestamps and geographical positions are also embedded, so that the graph modeling can obtain rich and reliable space-time information.

[0060] According to an embodiment of the present application, the geographical position of each sensor is represented by latitude and longitude coordinates, denoted as Labelcoor. In the embedding unit, the embedding processing mode of the geographical position of each sensor includes directly embedding the numbers in the latitude and longitude coordinates. For example, embedding the numbers 0 to 9, each number in the coordinates shares an index, and is concatenated to form a high-dimensional coordinate embedding. Illustratively, the latitude and longitude coordinates are 01N and 01W. Since the two value ranges are -180-180 and -90-90 degrees, and the representation space of this coordinate uses positive numbers, the latitude and longitude coordinates are initially represented as (181, 91). At this time, the numbers in the latitude and longitude coordinates are directly embedded, because the first number in the latitude and longitude coordinates must be a 3-digit number (0-360), and the second number must also be a 3-digit number (0-180). The present application retains both digits in the latitude and longitude coordinates to the last three decimal places, so the latitude and longitude coordinates are finally represented as: a 12-digit information, all the numbers in the latitude and longitude coordinates are uniformly embedded, a total of 10 numbers from 0 to 9, embedding (Embnumber) of each number is obtained by embedding the 10-dimensional space, so it is initially an E x 12-dimensional vector, which is mapped to E dimensions to become a vector Emb coor . coor as spatial embedding information.

[0061] According to an embodiment of the present application, the time stamp is denoted as Labeltime, and the embedding processing mode of the time stamp in the embedding unit includes mapping the time stamp to a uniform dimension to obtain time embedding information Emb time . During embedding processing, it can be divided into time of day, day of the week, etc., depending on the sampling rate. For example: hours, months, weeks, if it is Monday, it can be mapped to a 7-dimensional vector with 7 days in a week, and the value 1 is assigned to the 1st position to represent Monday, and the time embedding information of the time stamp is: [1, 0, 0, 0, 0, 0, 0], where it can also be mapped to a uniform dimension with hours and months, 24-dimensional vectors are constructed for hours, and 12-dimensional vectors are constructed for months.

[0062] According to an embodiment of the present application, in the Fourier transform unit, the obtaining mode of the to-be-extrapolated time sequence information input into the Fourier transform unit includes: processing the to-be-extrapolated time sequence data based on a pre-defined graph by using a first graph neural network of an encoder to obtain the to-be-extrapolated time sequence information, the first graph neural network is a graph isomorphism network after masking parameters, denoted as Masked GNN network, which includes a multi-layer perceptron network, and the processing mode of each layer of the perceptron network includes:

[0063]

[0064] wherein, represents the i-th sequence information output by the k-th layer of the perception network, k represents the layer number of the perception network, i represents the number of the sensor, MLP (k) represents the k-th layer of the perception network, j∈V represents that the sensor j belongs to all sensors, V represents the total number of sensors, e i,j represents the edge between the i-th sensor and the j-th sensor (also referred to as the initial spatial relationship between the i-th sensor and the j-th sensor), represents the j-th sequence information of the k-1-th layer of the perception network, A represents a predefined graph, X (k-1) represents all sequence information of the k-1-th layer of the perception network. Among them, all sequence information obtained by the last layer of the perception network of the first graph neural network is taken as the to-be-outside-inserted time sequence information.

[0065] According to an embodiment of the present application, the feature extraction unit comprises: a concatenation layer, configured to concatenate the time embedding information and the amplitude spectrum information corresponding to each sensor to obtain the concatenation information corresponding to each sensor; a perception layer, configured to perform linear processing on the concatenation information corresponding to each sensor to obtain the linear processing result corresponding to each sensor; a multi-head attention layer, configured to perform attention calculation on the spatial embedding information, the time embedding information and the amplitude spectrum information corresponding to each sensor according to the multi-head attention mechanism to obtain the attention score corresponding to each sensor; a fully connected network layer, configured to obtain an initial adjacency matrix according to the linear processing result and the attention score corresponding to each sensor, wherein the elements of the initial adjacency matrix are initial spatio-temporal relationship features between sensors; and a feature extraction layer, configured to retain the preset number of values arranged in descending order in each row of the initial adjacency matrix, and set the remaining values to 0, so as to obtain the dynamic spatio-temporal relationship between the sensors.

[0066] According to an embodiment of the present application, the processing mode of the perception layer in the feature extraction unit is as follows:

[0067] DE1=Linear(Concat(Emb time ,FFT(X))), (2)

[0068] wherein, DE1 represents the linear processing result, Linear represents the perception layer, Concat represents the concatenation operation, Emb time represents the time embedding information, FFT(X) represents the amplitude spectrum information obtained by the Fourier transform unit, and X represents the to-be-outside-inserted time sequence information.

[0069] According to an embodiment of the present application, the processing mode of the multi-head attention layer in the feature extraction unit is as follows:

[0070] DE2=MHA(Embtime ,Emb coor ,FFT(X)),(3)

[0071] wherein DE2 represents the attention score, MHA represents the multi-head attention layer (i.e., the multi-head attention network), Emb coor represents the spatial embedding information.

[0072] According to an embodiment of the present application, the processing mode of the feature extraction layer in the feature extraction unit is as follows:

[0073] A gm =TopK(W e (DE1·DE2 T )),(4)

[0074] wherein A gm represents the dynamic spatio-temporal relationship between sensors and is an adjacency matrix, TopK represents selecting the top K values of each row of the initial adjacency matrix and retaining them, and other values become 0, and W e is a learnable parameter of the fully connected network layer. In the metadata graph modeling module of the present application, an edge information (i.e., the initial spatio-temporal relationship feature between sensors) in the initial adjacency matrix is learned by assigning a learnable parameter W e . This enables the model of the present application to adaptively learn the spatio-temporal relationship in the scene, thereby capturing more effective spatio-temporal relationship and improving the accuracy of the extrapolation result.

[0075] According to an embodiment of the present application, refer to Figure 2which is a schematic diagram of the processing principle of the metadata graph modeling module. Taking N sensors as an example, the metadata of all sensors and the to-be-interpolated time series data are input into the module for processing. Among them, the geographic position in the metadata is represented by latitude and longitude coordinates, therefore, the geographic positions of all sensors form a dimension of N x 2, and the to-be-interpolated time series information has a dimension of N x L. First, the module embeds the numbers in the geographic position in the manner of the above embodiment to obtain normalized spatial embedding information, which has a dimension of N x E. Similarly, the timestamp is embedded in the manner of the above embodiment to obtain normalized time embedding information, which has a dimension of N x E, and the to-be-interpolated time series information is subjected to Fourier transform processing to obtain normalized amplitude spectrum information, which has a dimension of N x F. Second, the normalized time embedding information and the amplitude spectrum information are spliced to obtain spliced information, and a linear processing result with a dimension of N x E is obtained by using a perception machine layer to linearly process the spliced information. Then, the attention score with a dimension of N x E is obtained by using a multi-head attention mechanism to perform attention calculation according to the spatial embedding information, the time embedding information and the amplitude spectrum information corresponding to each sensor. A tensor product with a dimension of N x N x E is obtained according to the linear processing result and the attention score with a dimension of N x E. An initial adjacency matrix with a dimension of N x N is obtained by a fully connected network layer according to the tensor product with a dimension of N x N x E, wherein the elements are initial spatio-temporal relationship features between sensors. The initial adjacency matrix is processed by a feature extraction layer to retain a preset number of values arranged in descending order in each row of the initial adjacency matrix, and the remaining values are set to 0, so as to obtain the dynamic spatio-temporal relationship between sensors. Illustratively, if the Top K preset number is retained, it means that the top K values in each row of the initial adjacency matrix are retained. Finally, a metadata graph with a dimension of N x N is constructed according to the dynamic spatio-temporal relationship between sensors.

[0076] According to another embodiment of the application, the metadata graph modeling module can also only include a feature extraction unit and a metadata graph construction unit. At this time, the metadata graph modeling module does not need to process and normalize the geographic positions, timestamps and to-be-interpolated time series information respectively. The feature extraction unit of the module directly extracts the dynamic spatio-temporal relationship between sensors according to the geographic positions, timestamps and to-be-interpolated time series information corresponding to each sensor in the metadata. The metadata graph construction unit is used to construct a metadata graph based on the dynamic spatio-temporal relationship, which includes a plurality of nodes representing sensors and a plurality of edges representing the dynamic spatio-temporal relationship between sensors.

[0077] According to another embodiment of the present application, the feature extraction unit comprises an attention mechanism layer, a perception machine layer, a fully connected network layer and a feature extraction layer. The attention mechanism layer is configured to obtain an attention score corresponding to each sensor according to the geographical position, the timestamp and the to-be-interpolated time series information corresponding to each sensor. The perception machine layer is configured to perform linear processing on the to-be-interpolated time series information to obtain a linear processing result corresponding to each sensor. The fully connected network layer is configured to obtain an initial adjacency matrix according to the linear processing result corresponding to each sensor and the attention score, wherein the elements of the initial adjacency matrix are initial spatio-temporal relationship features between the sensors. The feature extraction layer is configured to retain a preset number of values arranged in descending order in each row of the initial adjacency matrix, and set the remaining values to 0, so as to obtain a dynamic spatio-temporal relationship between the sensors.

[0078] (3) Encoder

[0079] The inventors have found that when all sensors are interpolated at the same time window, there may be a delay between the sensors when recording the same information due to the physical distance between the sensors, and the existing model cannot capture the timestamp drift between the sensors, thereby ignoring the problem of the timestamp drift between the sensors, so that in a real application scenario, the interpolation result obtained by evaluation is not aligned with the true value in terms of timestamp, resulting in that the interpolation result does not correspond to the true value, the evaluation index calculated is not accurate enough, and the evaluation of the interpolation result is affected. Therefore, according to one embodiment of the present application, a decoupling phase module is arranged in the encoder, which converts the to-be-interpolated time series information into frequency domain information through the model to obtain an amplitude spectrum and a phase spectrum, learns the correlation between the timestamp drift and the phase shift through learning the phase spectrum, and learns the timestamp drift between the sensors, so as to correct the timestamp drift between the multiple time series data in reality and improve the accuracy of the interpolation result.

[0080] According to one embodiment of the present application, the encoder comprises: a first graph neural network configured to process the to-be-interpolated time series data based on a predefined graph using a graph isomorphism network after masking, to obtain to-be-interpolated time series information; and a decoupling phase module configured to perform decoupling processing on the to-be-interpolated time series information, convert the result after the decoupling processing into an amplitude spectrum and a phase spectrum corresponding to each sensor, and encode the metadata graph, the amplitude spectrum and the phase spectrum corresponding to each sensor to obtain an encoding result.

[0081] According to one embodiment of the present application, the first graph neural network adopts a graph isomorphism network (GIN) as a basic spatial network. The target sensor initially has no time series data. Therefore, the inventors modify the graph isomorphism network GIN, delete the parameter that controls its own historical information, denoted as ∈, which is a weight parameter for controlling the retention of historical information of each node. Since the target sensor has no historical information, the parameter ∈ in this network is always set to 0. First, the graph isomorphism network GIN is described as follows:

[0082]

[0083] where ∈ (k-1) represents the parameters of the k-1 layer perceptron network in the GIN network, when ∈ (k-1) is set to 0, the processing process described in formula (1) of the above embodiment is obtained, that is, the graph isomorphism network after masking the parameters uses the processing method of formula (1) of the above embodiment to process the external interpolation time series data.

[0084] According to one embodiment of the present application, referring to Figure 3 which is a schematic diagram of the processing principle of the decoupling phase module. The decoupling phase module includes a decoupling layer, a Fourier transform layer, an embedding layer, a graph isomorphism network layer, a reconstruction layer and a splicing layer. Again taking N sensors as an example to illustrate, the processing process of the decoupling phase module includes:

[0085] 1) Through the decoupling layer, the to-be-interpolated time series information is decoupled to obtain time series decoupling results, including trend values and residual values corresponding to each sensor, and the dimensions of the trend values corresponding to all sensors and the residual values corresponding to all sensors are both N x L;

[0086] 2) Through the Fourier transform layer, the trend values corresponding to each sensor are converted using Fourier transform to obtain first amplitude spectrum and first phase spectrum corresponding to each sensor, and the residual values corresponding to each sensor are converted using Fourier transform to obtain second amplitude spectrum and second phase spectrum corresponding to each sensor, wherein the dimensions of the first amplitude spectrum, the first phase spectrum, the second amplitude spectrum and the second phase spectrum corresponding to all sensors are all N x F;

[0087] 3) Through the embedding layer, the first phase spectrum and the second phase spectrum corresponding to each sensor are embedded respectively to obtain first phase embedding information and second phase embedding information corresponding to each sensor, wherein the dimensions of the first phase embedding information and the second phase embedding information corresponding to all sensors are both N x F x E;

[0088] 4) Through the graph isomorphism network layer, the first phase embedding information and the second phase embedding information corresponding to each sensor are processed based on the metadata graph by using GIN, to obtain the first phase feature and the second phase feature corresponding to each sensor, wherein the dimensions of the first phase feature and the second phase feature corresponding to all sensors are NxF;

[0089] 5) Through the reconstruction layer, inverse Fourier transform is performed on the first amplitude spectrum and the first phase feature corresponding to each sensor to obtain the reconstructed trend value corresponding to each sensor, and inverse Fourier transform is performed on the second amplitude spectrum and the second phase feature corresponding to each sensor to obtain the reconstructed residual value corresponding to each sensor, wherein the dimensions of the reconstructed trend value and the reconstructed residual value corresponding to all sensors are NxL;

[0090] 6) Through the splicing layer, the reconstructed trend value and the reconstructed residual value corresponding to each sensor are spliced to obtain the encoding result.

[0091] According to an embodiment of the present application, in the decoupling layer, the decoupling processing can be performed on the to-be-interpolated time sequence information by using the decoupling method commonly used in the field of time series. Wherein, the time series is decomposed into trend value by using average pooling convolution, and the decomposition method of trend value is as follows:

[0092] X t =Average(Pooling(X)),(6)

[0093] Wherein, X t represents the trend value corresponding to each sensor, Average represents average, Pooling represents pooling operation, and X represents the to-be-interpolated time sequence information obtained by the first graph neural network. By calculating the difference between the to-be-interpolated time sequence information and the decomposed trend value, the residual value X r corresponding to each sensor is obtained, that is, the residual value X r =X-X t .

[0094] According to an embodiment of the present application, in view of the correlation between the timestamp offset and the phase offset, the phase learning is performed by the embedding layer and the graph isomorphism network layer in the decoupling phase module. Wherein, the process principles of the embedding layer and the graph isomorphism network layer in the decoupling phase module for processing the first phase spectrum and the second phase spectrum are the same. Therefore, only the processing of the first phase spectrum is used to illustrate the phase learning process:

[0095] According to an embodiment of the present application, first, when the embedding layer is used to perform embedding processing on the first phase spectrum corresponding to each sensor, since the range of phase is only a circle, the range from-π to π can be divided into M parts, and any phase can obtain its phase embedding information. Illustratively, denotes the first phase spectrum of each sensor corresponding to the embedding layer i The embedding process is performed to obtain the first phase embedding information corresponding to each sensor. Next, according to the metadata graph given by the predefined graph and metadata graph modeling module, the phase offset is learned through GIN in the graph isomorphic network layer. GIN allows the phase embedding of each frequency domain component corresponding to each sensor to influence each other. Finally, there will be a fully connected layer to convert the high-dimensional phase embedding into a one-dimensional phase feature of the final K-layer output, and the one-dimensional phase feature is taken as the first phase feature corresponding to each sensor. The output one-dimensional phase feature is represented as follows:

[0096]

[0097] wherein, denotes the i-th sequence of the last layer (K-th layer) network of GIN (the sequence is the first phase feature corresponding to the i-th sensor), tanh denotes the tanh activation function, and FC denotes the fully connected layer, denotes the m-th phase of the i-th node of the k-th layer network of GIN, denotes the m-th phase of the i-th node of the k-1-th layer network of GIN, A phase denotes the adjacency matrix in the phase learning process (including the metadata graph and the predefined graph), denotes all phases of the k-th layer network of GIN.

[0098] The technical scheme of the above embodiment can at least achieve the following beneficial technical effects: by decoupling the trend item and the residual item in the time series information and disassembling the frequency domain information into the amplitude spectrum and the phase spectrum, the representation of the phase spectrum is learned, by the phase offset characteristics of the Fourier transform, the phase spectrum information in the frequency domain in the sequence can be modified, so that the time stamp of the sequence in the time domain can be changed, and then the time stamp offset between the target sensor and other sensors can be actively learned, the time stamp drift existing between multiple time series data in reality is corrected, and the accuracy of the extrapolation result is improved. Therefore, the decoupling phase module can be used to sense and supplement the time stamp drift between sensors after the phase learning.

[0099] (4) Decoder

[0100] According to one embodiment of the present invention, the decoder includes multiple decoding units, each of which is a dual-stream structure, wherein each stream structure includes a normalization layer and an inverse normalization layer, and the first stream structure includes a multi-layer second graph neural network, and the second stream structure includes a multi-layer third graph neural network. The data processing process of the first stream structure of the decoding unit includes: the normalization layer uses the Revin method to normalize the input encoding result to obtain a normalized encoding result, the multi-layer second graph neural network uses a multi-layer graph isomorphism network to process the normalized encoding result based on a predefined graph and a metadata graph to obtain a first processing result, and finally, the first processing result is inversely normalized by the inverse normalization layer to obtain an inversely normalized first processing result. The data processing process of the second stream structure of the decoding unit includes: the normalization layer uses the Revin method to normalize the input encoding result to obtain a normalized encoding result, and the multi-layer third graph neural network uses a graph isomorphism network after multi-layer masking parameters (i.e., a multi-layer Masked GNN network) to process the encoding result based on the predefined graph and the metadata graph to obtain a second processing result; wherein, the decoding result is obtained by aggregating the first processing results and the second processing results corresponding to multiple decoding units.

[0101] According to one embodiment of the present invention, the multi-layer second graph neural network includes a 2-layer graph isomorphism network GIN, and the multi-layer third graph neural network includes a 2-layer graph isomorphism network after masking parameters (i.e., a 2-layer Masked GNN network). It should be understood that this is only for illustration and the present invention is not limited to this. Schematically, the multi-layer second graph neural network may also include a 3-layer graph isomorphism network GIN, and the multi-layer third graph neural network includes a 3-layer graph isomorphism network after masking parameters (i.e., a 3-layer Masked GNN network).

[0102] The technical solution of the above embodiment can at least achieve the following beneficial technical effects: normalization removes the mean and variance of the time series information, thereby ignoring the intrinsic feature information of the time series information, performing denormalization processing, and adding the previously calculated variance and mean back to the learned time series to avoid the model learning such information that will drift in distribution, so as to reduce the impact of the intrinsic characteristics of the time series information changing over time.

[0103] (5) Polymerization layer

[0104] According to one embodiment of the present invention, the aggregation layer is a convolutional layer in the channel dimension that aggregates encoding and decoding results to obtain an aggregated result, which is then used as the extrapolated result of the model output. This aggregation method includes performing weighted aggregation of the encoding and decoding results through the aggregation layer to obtain the aggregated result, which also serves as information filtering.

[0105] According to an embodiment of the present application, the model structure and its data processing flow of the present application will be described in whole in combination with the above embodiment. Referring to Figure 4 which is a schematic diagram of the model structure and its data processing principle of the present application. The diagram includes a metadata graph modeling module, an encoder, a decoder and an aggregation layer. The data processing flow among them is as follows:

[0106] First, the to-be-interpolated time series data is encoded by using the metadata graph modeling module and the encoder, including: inputting a pre-defined graph with a dimension of N x N and to-be-interpolated time series data with a dimension of N x L into the encoder, the first graph neural network of the encoder processes the to-be-interpolated time series data based on the pre-defined graph to obtain to-be-interpolated time series information with a dimension of N x L, the to-be-interpolated time series information and the metadata are input into the metadata graph modeling module to obtain a metadata graph with a dimension of N x N, the metadata graph is input into the decoupling phase module, the decoupling phase module processes the to-be-interpolated time series information based on the metadata graph to obtain an encoding result;

[0107] Second, the encoding result is decoded by using the decoder, wherein the multi-layer second graph neural network in each decoding unit of the decoder includes 2 layers of graph isomorphism network GIN, represented as GIN x 2, the multi-layer third graph neural network includes 2 layers of masked parameter graph isomorphism network (i.e. 2 layers of Masked GNN network), represented as Masked GNN x 2, each decoding unit is a double-flow structure, each flow structure in it includes a normalization layer and an inverse normalization layer, the processing process of each decoder is the same as the above embodiment, and will not be repeated here; for illustration, there are M decoding units in the diagram, therefore, the decoding result is obtained by aggregating the first processing result and the second processing result corresponding to each of the M decoding units;

[0108] Finally, the encoding result and the decoding result are aggregated by using the aggregation layer to obtain an interpolation result with a dimension of N x L, which includes the time series data of the target sensor and the time series data of other sensors which do not miss all the time series data.

[0109] II. Model training

[0110] The inventors find that in the existing training phase, the sensors missing time series data can only be simulated by randomly masking the sensors without missing time series data. However, when facing the sensors with real missing time series data and new spatial relationships, the generalization ability of the model is difficult to guarantee. The performance of the model obtained by the existing training method can only be guaranteed by the semi-supervised learning theory of the graph, and no deliberate generalization optimization is performed, which leads to the fact that the existing model cannot evaluate whether the extrapolation result is suitable for being applied to the real scene, so that the model greatly lacks generalization. Therefore, according to one embodiment of the present application, a training method of an extrapolation model for multivariate time series analysis is provided. In the training process, discriminators are introduced in the first few rounds and the model is trained in an adversarial transfer training manner using the training set, and the model is trained only using the training set in the subsequent rounds. The sensors without missing all time series data are used as known sensors, and the sensors missing all time series data are used as target sensors. In the adversarial transfer training process, the purpose is to hope that the discriminator can distinguish which sensor is without missing all time series data and which sensor is missing all time series data, and according to the discrimination result of the discriminator, the update direction of the encoder is changed from distinguishing whether each sensor is a known sensor to confusing the label of whether each sensor is a known sensor, that is, the discriminator will be in confrontation with the encoder, and through this way, the information of the known sensor is generalized to the target sensor in this confrontation process, thereby improving the generalization of the model.

[0111] According to one embodiment of the present application, the training method comprises the following steps S1, S2 and S3:

[0112] Step S1: obtaining a training set comprising a plurality of samples, each sample comprising input data and a label sequence, the input data being time series data to be extrapolated of all sensors in a same predetermined time period, a predefined graph and metadata, the label sequence comprising a first label and a second label, the first label being used to indicate whether a sensor is missing all time series data in the predetermined time period, and the second label being real time series data of the sensor missing all time series data in the predetermined time period.

[0113] According to one embodiment of the present application, the training set can be constructed according to the actual application scene. For the traffic scene of monitoring traffic speed, the PEMS-BAY data set or the METR-LA data set can be used to construct the training set, and for the traffic scene of monitoring traffic flow, the PEMS03 data set or the PEMS04 data set can be used to construct the data set. Illustratively, taking the PEMS-BAY data set as an example, it contains data of 325 sensors in the Bay Area from January 2017 to June 2017, with a total of 52116 time steps. Traffic information is recorded at a frequency of every 5 minutes. PEMS-BAY also includes a sensor graph for indicating the dependency relationship between sensors and contains latitude and longitude position information. The pair-wise road network distance between sensors is calculated using the data-driven method for traffic prediction (Diffusion Convolutional Recurrent Neural Network, DCRNN) and the adjacency matrix is constructed using the threshold Gaussian kernel. When constructing the training set based on the PEMS-BAY data set, the input data generation method of each sample includes:

[0114] obtaining data collected by all sensors in a predetermined time period, setting the data collected by part of the sensors in the predetermined time period to 0, obtaining to-be-interpolated time series data, wherein when the data collected by the sensors are all 0, it indicates that all time series data of the corresponding sensor is missing, the adjacency matrix is used as a predefined graph, and the timestamp and latitude and longitude position information of the data collected by each sensor are used as metadata, and in this way, the input data is obtained;

[0115] The setting method of the label sequence of each sample includes:

[0116] The first label is set to indicate whether the sensor is missing all time series data in the predetermined time period, for example, 1 indicates that the sensor is missing all time series data in the predetermined time period, and 0 indicates that the sensor is not missing all time series data in the predetermined time period. Illustratively, if there are 6 sensors numbered 1-6, the sensors numbered 2 and 4 are missing all time series data, and the first labels of the 6 sensors numbered 1-6 are 0, 1, 0, 1, 0, and 0 respectively.

[0117] The second label is set to be the real time series data of the sensor missing all time series data in the predetermined time period.

[0118] Step S2: in a preset round, a pre-trained discriminator is introduced, the extrapolation model of the above embodiment is trained by using the discriminator and the training set, including that the extrapolation model obtains an extrapolation result according to input data, and the discriminator obtains a discrimination result according to the encoding result of the encoder, including whether all time series data of the sensor is missing in a predetermined time period, and the parameters of the extrapolation model and the discriminator are updated according to a preset total loss function, wherein the total loss function includes a sub-loss function for calculating the difference between the discrimination result and a first label, and a sub-loss function for calculating the difference between the extrapolation result and a second label.

[0119] According to an embodiment of the present application, the pre-trained discriminator includes a window sampling unit and a perception machine network, wherein the perception machine network is an MLP (Multilayer Perceptron). The way the discriminator obtains the discrimination result includes: sampling the encoding result output by the encoder through the window sampling unit with a preset fixed-length sliding window to obtain a sampling result, the preset fixed length being less than the length of the time series data corresponding to each sensor; and processing the sampling result by using the perception machine network to obtain the discrimination result. The way to generate the discrimination result is as follows:

[0120] Label D =MLP(Window(X encoder )),(8)

[0121] Wherein, Label D represents the discrimination result, MLP represents the perception machine network, Window(·) represents the window sampling unit, and X encoder represents the encoding result output by the encoder.

[0122] According to an embodiment of the present application, the preset total loss function adopts the following form:

[0123] Loss=υLoss D +ωLoss main ,(9)

[0124] Wherein, Loss D represents a sub-loss function for calculating the difference between the discrimination result and the first label, and υ represents a weight parameter. Loss main represents a sub-loss function for calculating the difference between the extrapolation result and the second label, and ω represents a weight parameter. Wherein, Loss D =BinaryCrossEntropy(Label D ,Label real ), BinaryCrossEntropy(·) represents a binary cross-entropy function, Label D represents the discrimination result, and Labelreal Represents the first label, and calculates the sub-loss function Loss between the extrapolated result and the second label main Regression loss function can be used. D The result of the sub-loss function calculation is used as the discriminant loss, and Loss main The result of the sub-loss function calculation is used as the extrapolation loss. In addition, when calculating the discrimination loss, the first label Label corresponding to each sensor real A small amount of error will be added to , or the weight parameter ω will be set to a value larger than υ to ensure that the extrapolation loss always dominates during training.

[0125] According to one embodiment of the present invention, using Loss D and Loss main The process of jointly training the model is a process of adversarial learning. Figure 5 , which is a schematic diagram of the principle of using the discriminator and the training set to train the extrapolation model. When the input data of the sample is input into the model, the processing process of the model is the same as in the above embodiment. Figure 4 The processing described is the same and will not be repeated here. The discriminator samples the encoder's output using a sliding window of a preset fixed length, producing a sampled result of dimension N×1×c. This sampled result is processed using a perceptron network and then passed through a gradient reversal layer (GRL) to produce a discrimination result of dimension N×2×c. A discriminant loss is calculated using the discrimination result and the first label corresponding to each sensor. A regression loss function is used to calculate the regression loss based on the extrapolated result and the second label. The model is then updated in reverse based on the discriminant and regression losses. The goal is for the discriminator to identify which sensors are not missing all time series data. Conversely, the discriminator aims to confuse the encoder to determine whether each sensor is a known sensor (i.e., one that is not missing all time series data). The gradient reversal layer (GRL) forces the encoder to optimize in the opposite direction, thus forming an adversarial learning process. In this process, the encoder attempts to confuse their features, while the discriminator attempts to distinguish them. The adversarial relationship between the discriminator and encoder helps capture common features across sensors, thereby improving the model's generalization.

[0126] According to one embodiment of the present invention, Loss D and Loss main The model will be trained together in the first few rounds of the training process, and the discriminator will be frozen in the subsequent rounds. For example, if the preset round is set to 5, the Loss will be used in the 1st to 5th rounds. D and Loss main Train the model together, using only Loss in the 6th and subsequent rounds mainTrain the model. It should be understood that this is only illustrative, and the preset number of rounds can also be 6 or 8, etc., and the present application is not limited thereto.

[0127] Step S3: In each round after the preset number of rounds, only the extrapolation model is trained using the training set, including obtaining an extrapolation result by the extrapolation model according to the input data, updating the parameters of the extrapolation model to minimize the difference between the extrapolation result and the second label, and obtaining a trained extrapolation model.

[0128] According to an embodiment of the present application, the difference between the extrapolation result and the second label is calculated using the same regression loss function as the above embodiment, the model parameters are updated to minimize the difference as the optimization objective, until the preset number of rounds or the model converges, the training is stopped, and a trained model is obtained.

[0129] III. Application scenarios

[0130] According to an embodiment of the present application, an extrapolation method for multivariate time series analysis is provided, which is used for extrapolating a missing complete time series data sensor in a traffic scene, a meteorological monitoring scene or an energy monitoring scene to obtain an extrapolation result. The method comprises: obtaining the to-be-extrapolated time series data of all sensors in the scene within a same predetermined time period, a predefined graph and metadata; using the trained extrapolation model obtained by the training method according to the above embodiment, obtaining an extrapolation result according to the input to-be-extrapolated time series data, predefined graph and metadata.

[0131] In order to verify the beneficial effects of the present application, the inventors conducted the following experiments:

[0132] ① The following four data sets are obtained:

[0133] METR-LA dataset: a traffic speed dataset collected from loop detectors on the road network in Los Angeles County. It contains data collected by 207 sensors from March 2012 to June 2012, with a total of 34272 time steps. Traffic information is recorded at a frequency of every 5 minutes. METR-LA includes a sensor graph indicating the dependency between sensors and contains latitude and longitude information. The pair-wise road network distance between sensors is calculated by DCRNN, and the adjacency matrix is constructed using a thresholded Gaussian kernel.

[0134] PEMS-BAY dataset: is a traffic speed dataset collected from the California Performance Measurement System (PeMS) of the California Department of Transportation (CalTrans). It contains data collected by 325 sensors in the Bay Area from January 2017 to June 2017, with a total of 52116 time steps. Traffic information is recorded at a frequency of every 5 minutes. PEMS-BAY also includes a sensor graph indicating the dependency between sensors and contains latitude and longitude information. The pair-wise road network distance between sensors is calculated by DCRNN, and the adjacency matrix is constructed using a thresholded Gaussian kernel.

[0135] PEMS03 dataset: is a traffic flow dataset belonging to the PEMS0X series, where X represents the region code of data collection. It contains data collected by 358 sensors from September 2018 to November 2018, with a total of 26208 time steps. Traffic information is recorded at a frequency of every 5 minutes. PEMS03 includes a sensor graph indicating the dependency between sensors, but does not provide latitude and longitude information.

[0136] PEMS04 dataset: is a traffic flow dataset belonging to the PEMS0X series. It contains data collected by 307 sensors from January 2018 to February 2018, with a total of 16992 time steps. Traffic information is recorded at a frequency of every 5 minutes. PEMS04 includes a sensor graph indicating the dependency between sensors, but does not provide latitude and longitude information.

[0137] ②Under the above 4 datasets, the performance comparison experiment of the present application and the existing model for extrapolation is carried out. Among them, the existing model includes GRIN model, INCREASE model, IGNNK model, SATCN model, GCN model and GIN model. The performance indicators for evaluating the extrapolation of the model include Mean Absolute Error (MAE), Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE). The performance comparison experiment results of the present application method and the existing method for extrapolation under the above 4 datasets are shown in Table 1 as follows:

[0138] Table 1

[0139]

[0140] As can be clearly seen from the above table, the present application is superior to other existing methods in terms of MAE and RMSE indicators, and achieves an average of more than 5% improvement in extrapolation performance. In addition, when comparing the present application with IGNNK, INCREASE and other methods, the present application is not only significantly superior to them, but also achieves nearly 10% improvement in extrapolation performance on the PEMS0X and other data sets. In general, the present application is superior to other existing methods in terms of extrapolation performance on the above four data sets. It shows that the multivariate time series extrapolation model proposed in the present application can effectively perceive the spatio-temporal relationship between sensors in the scene, thereby improving the effectiveness and generalization of the extrapolation model, so as to improve the accuracy of the extrapolation result, making it closer to the true value, and better applied in real scenarios. In addition, the training method designed for the model in the present application enables the present application to be better promoted and applied.

[0141] The existing methods are as follows:

[0142] GRIN model: [6] Cini, A., Marisca, I., & Alippi, C. (2022). Filling the Gaps: Multivariate Time Series Imputation by Graph Neural Networks. International Conference on Learning Representations (ICLR). Retrieved from https: / / arxiv.org / abs / 2108.00298.

[0143] INCREASE model: [5] Gao, H., Jiang, R., Dong, Z., Deng, J., Ma, Y., & Song, X. (2023). Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecasting. Retrieved from https: / / arxiv.org / abs / 2312.00516.

[0144] IGNNK model: [3] Wu, Y., Zhuang, D., Labbe, A., & Sun, L. (2020). Inductive Graph Neural Networks for Spatiotemporal Kriging. AAAI Conference on Artificial Intelligence. Retrieved from https: / / arxiv.org / abs / 2006.07527.

[0145] SATCN model: [4] Wu, Y., Zhuang, D., Lei, M., Labbe, A., & Sun, L. (2021). Spatial Aggregation and Temporal Convolution Networks for Real-time Kriging. Retrieved from https: / / arxiv.org / abs / 2109.12144.

[0146] GCN model: [1] Kipf, T. N., & Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. International Conference on Learning Representations (ICLR). Retrieved from https: / / arxiv.org / abs / 1609.02907.

[0147] GIN model: [2] Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2019). How Powerful are Graph Neural Networks? International Conference on Learning Representations (ICLR). Retrieved from https: / / arxiv.org / abs / 1810.00826.

[0148] It should be noted that although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order. In fact, some of these steps can be performed concurrently or even in a different order, as long as the desired function can be achieved.

[0149] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0150] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se.

[0151] Having described above several embodiments, it is to be appreciated various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure. Accordingly, the foregoing description is by way of example only and is not intended to be limiting. The use of the term "act" and "determining" along with the conjunctive terms "adapted to," "operational to," "designed to," "configured to," "made to," or the like, is intended to connote that the indicated element is so adapted, operational, designed, configured, made, or the like, at least in some way that the technical effect in question is achieved. The use of the conjunctive term "comprising" is intended to mean a open transition such that any and all subsequent elements, whether of a like or different kind, are permissible. The use of the conjunctive term "comprising" is intended to be non-limiting of the embodiments of the application.

Claims

1. An extrapolation model for multivariate time series analysis, characterized in that: Models include: A data acquisition module is configured to acquire the time series data to be extrapolated, a predefined graph, and metadata for all sensors in the scene within the same predetermined time period. The time series data to be extrapolated includes the time series data of each sensor within the predetermined time period, wherein all sensors include target sensors that are missing all time series data, and the time series data of the target sensors are replaced with preset missing values. The metadata includes the geographic location of each sensor and the timestamp corresponding to each data in the time series data. The predefined graph includes the initial spatial relationship between the sensors in the scene. A metadata graph modeling module is used to extract the dynamic spatiotemporal relationships between sensors based on the time series data to be extrapolated and the metadata using a neural network, and to construct a metadata graph based on the dynamic spatiotemporal relationships. The metadata graph modeling module includes: An embedding unit is used to embed the geographical location and time stamp of each sensor in the metadata to obtain spatial embedding information and temporal embedding information corresponding to each sensor; A Fourier transform unit is used to perform a Fourier transform operation on the extrapolated time series information to obtain amplitude spectrum information corresponding to each sensor; A feature extraction unit is used to extract the dynamic spatiotemporal relationship between the sensors based on the spatial embedding information, temporal embedding information and amplitude spectrum information corresponding to each sensor; A metadata graph construction unit, configured to construct a metadata graph based on the dynamic spatiotemporal relationship, the metadata graph comprising a plurality of nodes representing sensors and a plurality of edges representing the dynamic spatiotemporal relationship between the sensors; Wherein, the feature extraction unit includes: The splicing layer is used to splice the time embedding information and amplitude spectrum information corresponding to each sensor to obtain the splicing information corresponding to each sensor; The perception machine layer is used to perform linear processing on the splicing information corresponding to each sensor to obtain the linear processing results corresponding to each sensor; The multi-head attention layer is used to calculate the attention score of each sensor based on the spatial embedding information, temporal embedding information, and amplitude spectrum information of each sensor using the multi-head attention mechanism. The fully connected network layer is used to obtain the initial adjacency matrix based on the linear processing results and attention scores of each sensor. The elements of the matrix are the initial spatiotemporal relationship features between the sensors. The feature extraction layer is used to retain a preset number of values ​​in each row of the initial adjacency matrix in descending order and set the remaining values ​​to 0 to obtain the dynamic spatiotemporal relationship between each sensor; An encoder is used to encode the extrapolated time series data using a neural network based on a predefined graph and a metadata graph to obtain an encoding result; A decoder, configured to decode the encoding result based on the predefined graph and the metadata graph to obtain a decoding result; The aggregation layer is used to aggregate the encoding results and the decoding results to obtain an extrapolation result, including the time series data extrapolated for the target sensor within the predetermined time period.

2. The model according to claim 1, characterized in that The encoder comprises: The first graph neural network is used to process the time series data to be extrapolated based on a predefined graph using a graph isomorphism network after masking parameters to obtain the time series information to be extrapolated; The decoupling phase module is used to decouple the extrapolated timing information, convert the decoupling results into the amplitude spectrum and phase spectrum corresponding to each sensor, and encode them according to the metadata graph, the amplitude spectrum and phase spectrum corresponding to each sensor to obtain the encoded results.

3. The model according to claim 2, characterized in that The decoupling phase module includes: The decoupling layer is used to decouple the extrapolated time series information to obtain the time series decoupling results, including the trend value and residual value corresponding to each sensor; A Fourier transform layer is used to transform the trend value corresponding to each sensor through Fourier transform to obtain a first amplitude spectrum and a first phase spectrum corresponding to each sensor, and to transform the residual value corresponding to each sensor through Fourier transform to obtain a second amplitude spectrum and a second phase spectrum corresponding to each sensor; An embedding layer is used to embed the first phase spectrum and the second phase spectrum corresponding to each sensor respectively to obtain the first phase embedding information and the second phase embedding information corresponding to each sensor; A graph isomorphism network layer is used to process the first phase embedding information and the second phase embedding information corresponding to each sensor respectively using a graph isomorphism network based on the metadata graph to obtain the first phase feature and the second phase feature corresponding to each sensor; The reconstruction layer is used to perform an inverse Fourier transform based on the first amplitude spectrum and the first phase characteristics corresponding to each sensor to obtain a reconstructed trend value corresponding to each sensor, and to perform an inverse Fourier transform based on the second amplitude spectrum and the second phase characteristics corresponding to each sensor to obtain a reconstructed residual value corresponding to each sensor; The splicing layer is used to splice the reconstructed trend value and the reconstructed residual value corresponding to each sensor to obtain the encoding result.

4. The model according to claim 1, characterized in that The decoder includes multiple decoding units, each of which has a dual-stream structure and includes: a multi-layer second graph neural network, configured to process the encoding result based on the predefined graph and the metadata graph using a multi-layer graph isomorphism network to obtain a first processing result; a multi-layer third graph neural network, configured to process the encoding result based on the predefined graph and the metadata graph using the graph isomorphism network after the multi-layer masking parameters to obtain a second processing result; The decoding result is obtained by aggregating the first processing results and the second processing results corresponding to the multiple decoding units.

5. A method for training an extrapolation model for multivariate time series analysis, characterized in that: include: Obtain a training set, which includes multiple samples, each sample includes input data and a label sequence, the input data is the time series data to be extrapolated, the predefined graph and metadata of all sensors in the same predetermined time period, and the label sequence includes a first label and a second label, the first label is used to indicate whether the sensor is missing all time series data in the predetermined time period, and the second label is the real time series data of the sensor missing all time series data in the predetermined time period; In a preset round, a pre-trained discriminator is introduced, and the extrapolation model according to any one of claims 1 to 4 is trained using the discriminator and the training set, including the extrapolation model obtaining an extrapolation result based on the input data, and using the discriminator to obtain a discrimination result based on the encoding result of the encoder, including whether the sensor lacks all time series data within a predetermined time period, and updating the parameters of the extrapolation model and the discriminator according to a preset total loss function, wherein the total loss function includes a sub-loss function for calculating the difference between the discrimination result and the first label and a sub-loss function for calculating the difference between the extrapolation result and the second label; In each round after the preset round, the extrapolation model is trained only using the training set, including the extrapolation model obtaining an extrapolation result based on the input data, and updating the parameters of the extrapolation model with the optimization goal of minimizing the difference between the extrapolation result and the second label to obtain a trained extrapolation model.

6. An extrapolation method for multivariate time series analysis, for extrapolating sensors that are missing all time series data in a traffic scenario, a weather monitoring scenario, or an energy monitoring scenario to obtain an extrapolation result, characterized in that: Methods include: Obtain the time series data to be extrapolated, predefined graphs, and metadata for all sensors in the scene within the same predetermined time period; The extrapolation result is obtained by using the trained extrapolation model obtained based on the method of claim 5 according to the input time series data to be extrapolated, the predefined graph and the metadata.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to claim 5 or 6.

8. An electronic device, characterized in that: include: one or more processors; as well as a memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of claim 5 or 6 by executing the executable instructions.

Images