Natural gas flow prediction method based on segment graph convolution and time attention mechanism

Abstract

The application discloses a natural gas flow prediction method based on a segmented graph convolution and a time attention mechanism. The method designs a multivariate time series prediction model based on a segmented graph convolution and a time attention mechanism, a segmented graph convolution layer of the model is based on a cosine similarity to construct a segmented graph structure, a sequence correlation between changes of paired variables is extracted through the segmented graph convolution, a time expansion convolution layer of the model extracts time features in the sequence and reduces the dimension of the sequence, and an output part of the model is designed with a time sequence pattern enhancement module based on a two-stage attention mechanism and used for aggregating time sequence patterns in the sequence. Through the trained multivariate time series prediction model, the future time step of the gas time sequence can be predicted, and the prediction precision is high.

Description

A Natural Gas Flow Forecasting Method Based on Piecewise Graph Convolution and Temporal Attention Mechanism Technical Field

[0001] This invention relates to the field of time series data prediction, and in particular to a natural gas flow prediction method based on segmented graph convolution and time attention mechanism. Background Technology

[0002] Natural gas is an indispensable energy source in modern society, widely used in homes, industry, commerce, and transportation. Forecasting natural gas consumption is crucial for balancing energy market supply and demand. Accurate consumption forecasts provide valuable information for natural gas suppliers, pipeline operators, and market decision-makers, enabling them to balance the natural gas supply and demand market. Furthermore, accurate consumption forecasts allow natural gas suppliers to better plan their dispatch and procurement schedules, improving allocation efficiency. Therefore, overall, the need for accurate gas volume forecasting is increasingly strong for the effective management and allocation of natural gas resources.

[0003] In the current natural gas industry, gas companies deliver natural gas to various consumption points through city gas pipelines. Industrial and commercial gas flow meters and other instruments deployed at these points automatically collect relevant data on natural gas usage and upload it to the gas company's database periodically via IoT technology. Therefore, this time-series data recorded by different sensors lays the data foundation for predicting natural gas consumption.

[0004] Natural gas consumption time-series data is a typical example of multivariate time series data synchronized by multiple sensors. Current technologies for multivariate time series forecasting require simultaneous modeling of inter-series correlations and intra-series features. Traditional statistical methods, such as VAR and GP, can only uncover linear relationships between variables in a multivariate time series. Current methods based on recurrent neural networks and convolutional neural networks, such as LSTNet and TPA-LSTM, can handle the temporal patterns of single series and make predictions for all variables in a multivariate time series simultaneously, but their ability to extract pairwise variable correlations is limited. Inspired by using graph neural networks to model the correlations in road traffic, some existing technologies treat the variables in a multivariate time series as nodes in a graph, using graph structure learning modules to learn the graph structure between time series, and then using graph convolutional networks to extract correlation information between variables. Although extending graph neural networks to multivariate time series forecasting has proven effective, some challenges remain in multivariate time series forecasting.

[0005] One challenge is modeling the implicit and time-evolving correlations between time series. In real-world scenarios, natural gas metering devices are not physically interconnected like roads, meaning there are no predefined topological relationships. Instead, they follow an implicit, device- or user-dependent pattern, significantly increasing the difficulty of extracting correlation information between series. While introducing graph structures can characterize the correlations between series, these correlations are not static but evolve over time.

[0006] Secondly, there is the challenge of extracting the diverse and potentially discontinuous natural gas time-series patterns from multivariate time series data. Natural gas multivariate time-series data originates from different devices, meaning that different types of patterns generated by different users coexist. For example, some industrial plants concentrate their natural gas usage during working hours, while restaurants' peak gas consumption is distributed across mealtimes. The diverse and discontinuous nature of these complex natural gas consumption time-series patterns significantly increases the difficulty of model prediction.

[0007] Therefore, how to solve the problems existing in the natural gas consumption forecasting process and improve the accuracy of natural gas consumption forecasting is an urgent technical problem to be solved. Summary of the Invention

[0008] To overcome the shortcomings of the existing technology and improve the accuracy of natural gas consumption forecasting, this invention provides a natural gas flow forecasting method based on segmented graph convolution and time attention mechanism, which can effectively solve the above problems.

[0009] The specific technical solution adopted in this invention is as follows:

[0010] A natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism includes the following steps:

[0011] S1. For each target flow sensor that the user needs to monitor, obtain the historical time series of natural gas flow recorded by each target flow sensor. After normalizing each series, use a fixed-length time window to slide and extract on the series to generate labeled training samples and build a training sample set. The input of each training sample is a multivariate time series segment composed of natural gas flow data of each target flow sensor within a time window. The label is the recorded value of each target flow sensor at a specified time step after the time window. All labeled training samples constitute the training sample set.

[0012] S2. Using the training sample set obtained in S1, train the multivariate time series prediction model based on segmented graph convolution and time attention mechanism until the model converges, so that it can predict the recorded values ​​of each target flow sensor at a specified time step after the time window where the input data is located.

[0013] The multivariate time series prediction model based on segmented graph convolution and temporal attention mechanisms includes a segmented graph convolution module, a time dilation convolution module, and a temporal information enhancement module. The input to the model is a multivariate time series segment. The segmented graph convolution module segments the input multivariate time series segment according to the time dimension, constructs a segmented adjacency matrix based on the cosine similarity between sequences within a segment, and then performs multi-layer segmented graph convolution to extract the correlation between sequences, obtaining a first output signal. The time dilation convolution module inputs the first output signal from the segmented graph convolution module into a multi-layer stacked threshold control unit with dilation convolution to further extract the features within each sequence. The second output signal is obtained by dimensionality reduction and genotyping. The time information enhancement module inputs the multivariate time series segments into the LSTM network, groups the signals of each hidden step of the LSTM network according to time, and applies an attention mechanism to extract segmented context vectors for each group. Then, the attention mechanism is applied to the segmented context vectors to generate time context vectors. Finally, the time context vectors are concatenated with the last signal of the hidden layer of the LSTM network, and the concatenation result is passed through a fully connected layer to obtain the third output signal. Finally, the second output signal and the third output signal are added together, and after inverse normalization, the final prediction output of the multivariate time series prediction model is obtained.

[0014] S3. In actual prediction, the natural gas flow data within the most recent time window is obtained from the data recorded by each target flow sensor and a real-time multivariate time series segment is formed. After normalization, the real-time multivariate time series segment is input into the trained multivariate time series prediction model to predict the natural gas flow value corresponding to each target flow sensor at a specified time step in the future.

[0015] Preferably, step S1 specifically includes the following sub-steps:

[0016] S11. Obtain the historical time series of natural gas flow recorded by the N target flow sensors that the user needs to monitor. Each historical time series of natural gas flow Normalization is performed by using the maximum value of each element as the scale value, resulting in a normalized time series. From all normalized time series Composition of multivariate time series

[0017] S12, using a time window of fixed length W in a multivariate time series The slide proceeds according to a preset step size s, with each time window transitioning from a multivariate time series during the slide. Extracting multivariate time series segments X∈R within a window {N×W} and in multivariate time series The value Y∈R at the h-th time step after the time window. {N} As regression labels for the model, the training dataset for the model is generated, where each sample is represented as {X∈R}. {N×W} ,Y∈R {N}}

[0018] Preferably, the specific processing steps in the multivariate time series prediction model based on segmented graph convolution and temporal attention mechanism in S2 are as follows:

[0019] S21. Input the multivariate time series segment X of the input model into the segmented graph convolution module. First, according to the preset segment number hyperparameter P, the input multivariate time series segment X is evenly divided into P segments in the time dimension. Each segment is represented as X. k ∈R {N×S} S = W / P;

[0020] Then use segment X k The sequences corresponding to each target flow sensor within the data are nodes, and cosine similarity is used to calculate the segmentation X. k The adjacency degree of the internal nodes is used to obtain the segment X. k The corresponding graph adjacency matrix A′ k ∈R {N×N} A′ k Any node i and node j are in the same segment X k The strength of the connecting edge is expressed as:

[0021]

[0022] In the formula: X k,i and X k,j Representing segment X respectively k The sequence corresponding to the i-th and j-th target flow sensors within;

[0023] Then use the Softmax function to apply the graph adjacency matrix A′ k Normalize by row to form the normalized graph adjacency matrix A. k A k In and A′ k,i,j The corresponding adjacency degree A k,i,j The calculation formula is:

[0024]

[0025] Finally, for each segment X... k Based on the obtained segmented adjacency matrix A k The graph convolution network performs a total of G layers of graph convolution computation, and outputs the final graph convolution result H. (G)Furthermore, the graph convolution process of the l-th layer in the graph convolutional network is represented as follows:

[0026]

[0027] Where D k Let A be the adjacency matrix. k The degree matrix, For a trainable parameter matrix, It is the k-th segment after the final output of the previous graph convolutional layer has been decomposed; l = 1, 2, ..., G.

[0028] The graph convolutional network obtains the segmented convolution result from the graph convolution of the l-th layer. Recombining and stitching the data along the time dimension yields the output of the l-th layer graph convolution:

[0029]

[0030] The graph convolutional network converts the graph convolution result of the l-th layer into a graph convolutional network. The l-th layer graph convolution is obtained by performing a residual connection with the original input multivariate time series segment X.

[0031] S22. Convolve the final graph result H obtained in S21. (G) In the input temporal dilation convolution module, a total of C layers of temporal dilation convolution are performed, where the process of the c-th layer of temporal dilation convolution is represented as follows:

[0032]

[0033] Z (c-1) It is the output of the previous dilated convolutional layer. σ(·) represents matrix dot product, σ(·) represents the sigmoid activation function, * represents dilated convolution operation, and W represents matrix multiplication. f (c) and Let C and C represent the learnable parameters of the two convolutional kernels, respectively. The number of kernels in both parts is M, and the size of the kernel is C1; c = 1, 2, ..., C.

[0034] The signal Z output from each time-dilated convolution layer (c) After weighted summation, a regular convolution layer is used to process the signal into the predicted target size, resulting in the final output of the time-dilated convolution module:

[0035]

[0036] Among them W s(c) Z represents the signal Z generated by the convolution of the c-th layer. (c) The trainable weight matrix for weighted calculation; W e (·) represents a regular convolution with M kernels. e The size of the convolution kernel is C e ;

[0037] S23. Input the multivariate time series segment X into the time information enhancement module. First, an LSTM network with N hidden layer units performs feature extraction. Then, according to the segmentation number hyperparameter P, the hidden layer output H′ of the LSTM network with a length of W is segmented to obtain P segmented hidden layer signals H′. k :

[0038] H′=LSTM(X)=[h1,h2,…,h W ]

[0039] H′ k =[h k*S-s+1 ,h k*s-S+2 ,…,h k*S ], k=1,2,…P

[0040] Then the obtained segmented hidden layer signal H′ k After trainable parameters W k After weighting, we get The corresponding segment weights a′ are obtained by calculating using the Softmax function. k The hidden layer signals H′ of each segment k With the weights a′ of each segment k The weighted summation yields the segmented context vector C′:

[0041]

[0042]

[0043]

[0044] Then, the segmented context vector C′ is weighted by the trainable parameters W″ to obtain... The hidden layer signal contains S time steps. The time step weights a″ are obtained after calculation using the Softmax function. i The time context vector C″ is composed of the hidden layer signals at each time step. With time step weight a″ i Weighted summation yields:

[0045]

[0046]

[0047]

[0048] Finally, the obtained time context vector C″ is compared with the last time step signal h of the LSTM network hidden layer. W The signal is spliced ​​together and then passed through a fully connected layer to enhance the time information output.

[0049] H A =W A ·concat(C″,h W )+b

[0050] In the formula: W A b and b represent the learnable weights and biases of the fully connected layer, respectively;

[0051] S24. Convert the final output H of the time-dilated convolution module. C The time information enhancement signal H output by the time information enhancement module A The sums are then inversely normalized to obtain the final prediction output of the network model:

[0052]

[0053] In the formula: scale represents the inverse normalization coefficient.

[0054] Preferably, the length W of the time window is 168, and the sliding step s of the time window is 1.

[0055] Preferably, the hyperparameter P for the number of segments is 7.

[0056] Preferably, the number of graph convolutional layers G in the graph convolutional network is 2.

[0057] Preferably, the number of time-dilated convolutional layers C in the time-dilated convolutional module is 5, the number of convolutional kernels M in each layer is 24, the size of the convolutional kernel is 1×3, and the dilation rate D of the 5 convolutional kernels is 1, 3, 6, 12, and 24, respectively.

[0058] Preferably, in the time-dilated convolution module, the number of convolution kernels M in the last ordinary convolution layer is... e Take 1, the size of the convolution kernel C e Take 1×76.

[0059] Preferably, the loss function used when training the multivariate time series prediction model is the mean absolute error.

[0060] Preferably, when training the multivariate time series prediction model, the AdamW optimizer is used to update all trainable parameters of the neural network model, and the learning rate α is 0.0001 and the decay weight β is 0.00001.

[0061] The multi-energy time series prediction method proposed in this invention, based on segmented graph convolution and temporal attention mechanism, has the following beneficial effects:

[0062] This invention, through learning from historical multivariate time series data composed of multiple sensors, designs a neural network model that can accurately predict the flow value of each gas flow sensor at a future time step based on the latest historical gas flow data. This model utilizes a segmented graph convolutional neural network to learn the correlation between sequences, and uses time dilation convolution and time information enhancement modules to learn the temporal characteristics within the sequences. Attached Figure Description

[0063] Figure 1 is a flowchart of a natural gas flow prediction method based on segmented graph convolution and temporal attention mechanism.

[0064] Figure 2 is a schematic diagram of the multivariate time series prediction model proposed in this invention. Detailed Implementation

[0065] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0066] As shown in Figure 1, in a preferred embodiment of the present invention, the natural gas flow prediction method based on segmented graph convolution and temporal attention mechanism specifically includes steps S1 to S3. The specific implementation of each step is described in detail below.

[0067] S1. For each target flow sensor that the user needs to monitor, obtain the historical time series of natural gas flow recorded by each target flow sensor. After normalizing each series, use a fixed-length time window to slide and extract on the series to generate labeled training samples and build a training sample set. The input of each training sample is a multivariate time series segment composed of natural gas flow data of each target flow sensor within a time window. The label is the recorded value of each target flow sensor at a specified time step after the time window (denoted as the h-th step after the time window). All labeled training samples constitute the training sample set.

[0068] In embodiments of the present invention, step S1 specifically includes the following sub-steps:

[0069] S11. Obtain the historical time series of natural gas flow recorded by the N target flow sensors that the user needs to monitor. Each historical time series of natural gas flow Using their respective maximum values ​​as scale values, maximum value normalization is performed to form a normalized time series. From all normalized time series Composition of multivariate time series

[0070] The above-mentioned maximum value normalization is based on existing technology, assuming that each natural gas flow historical time series... If the maximum value in the sequence is MAX(S[i]), then it can be used as the scaling value for each sequence. Perform maximum value normalization:

[0071]

[0072] in This represents the historical time series of natural gas flow for the i-th entry. Let i represent the i-th historical time series after extreme value normalization.

[0073] S12, using a time window of fixed length W in a multivariate time series The slide proceeds according to a preset step size s, with each time window transitioning from a multivariate time series during the slide. Extracting multivariate time series segments X∈R within a window {N×W} and in multivariate time series The value Y∈R at the h-th time step after the time window. {N} As regression labels for the model, the training dataset for the model is generated, where each sample is represented as {X∈R}. {N×W} ,Y∈R {N}}

[0074] S2. Using the training sample set obtained in S1, train the multivariate time series prediction model based on segmented graph convolution and time attention mechanism until the model converges, so that it can predict the recorded values ​​of each target flow sensor at a specified time step after the time window where the input data is located.

[0075] Figure 2 illustrates the structure of the multivariate time series prediction model based on piecewise graph convolution and temporal attention mechanisms provided by this invention. The model includes a piecewise graph convolution module, a temporal dilation convolution module, and a temporal information enhancement module. The input to the model is the aforementioned multivariate time series segment X. The general processing flow of the model is as follows:

[0076] The segmented graph convolution module described above segments the input multivariate time series according to the time dimension, constructs a segmented adjacency matrix based on the cosine similarity between sequences within a segment, and then performs multi-level segmented graph convolution to extract the correlation between sequences and obtain the first output signal.

[0077] In some time series data, the temporal pattern of a single sequence is not constant but evolves over time. Therefore, this invention designs the aforementioned segmented graph convolution structure to adapt to changes in graph connectivity between different time periods within the same time window. Taking the natural gas flow rate of this invention as an example, assuming that a user's gas consumption pattern is stable in the short term (e.g., within a day), then short-term natural gas sequences of length W have the same graph adjacency matrix.

[0078] The aforementioned time-dilated convolution module inputs the first output signal from the segmented graph convolution module into a multi-layered stacked threshold control unit with dilated convolution, further extracting features within each sequence and reducing dimensionality to obtain the second output signal.

[0079] Temporally dilated causal convolutions allow for adjustable dilation rates, and the receptive field of a single convolutional kernel grows exponentially with the number of convolutional layers. Compared to undilated kernels, dilated kernels can capture a larger historical range of the time series. The task of the aforementioned temporally dilated convolution module is to extract the intra-sequence temporal patterns of each node, while simultaneously reducing the dimensionality of the output signal from the segmented graph convolution module, and making preliminary predictions for each variable in the multivariate time series.

[0080] The aforementioned time information enhancement module inputs the multivariate time series segments into the LSTM network, groups the signals of each hidden step of the LSTM network according to time, and applies an attention mechanism to extract segmented context vectors for each group. Then, it applies the attention mechanism again to the segmented context vectors to generate time context vectors. Finally, it concatenates the time context vectors with the last signal of the hidden layer of the LSTM network, and passes the concatenation result through a fully connected layer to obtain the third output signal. Finally, it adds the second and third output signals, performs inverse normalization, and obtains the final prediction output of the multivariate time series prediction model.

[0081] While time-dilated convolutional layers effectively extract intra-sequence temporal patterns for each variable, the trend and periodic information of the original sequence decreases as the number of convolutional layers increases. When the model predicts the values ​​of variables within future periods, in addition to focusing on recent changes in each variable, it also needs to extract trends and patterns from the same time period in previous periods, while irrelevant historical parts, such as those with consecutive zero values, should be ignored. Traditional global attention mechanisms calculate the weights for each time step in the entire sequence and aggregate them into a context vector to correct the network's output. As the sequence length increases, the context vector contains too much invalid historical information, offering limited improvement to the model's prediction performance. Therefore, the temporal information enhancement module employs a two-stage attention mechanism to extract features from long sequences. The main idea of ​​the two-stage attention mechanism is to integrate key segments into segmented context vectors, filter out irrelevant segments, and then further extract temporal context vectors from the segmented context.

[0082] In an embodiment of the present invention, the specific processing procedure in the multivariate time series prediction model based on segmented graph convolution and temporal attention mechanism in step S2 above is as follows:

[0083] S21. Input the multivariate time series segment X of the input model into the piecewise graph convolution module to obtain the final graph convolution result H. (G) The execution flow in the segmented graph convolution module is shown in S211 to S214 below:

[0084] S211. First, based on the preset segmentation number hyperparameter P, the input multivariate time series segment X is divided into P segments in the time dimension on an average basis. Each segment is represented as X. k ∈R {N×S} S = W / P.

[0085] S212, then segmented X k The sequences corresponding to each target flow sensor within the data are nodes, and cosine similarity is used to calculate the segmentation X. k The adjacency degree of the internal nodes is used to obtain the segment X. k The corresponding graph adjacency matrix A′ k ∈R{N×N} A′ k Any node i and node j are in the same segment X k The strength of the connecting edge is expressed as:

[0086]

[0087] In the formula: X k,i and X k,j Representing segment X respectively k The sequence corresponding to the i-th and j-th target flow sensors, where || represents the modulo operation.

[0088] S213. Then use the Softmax function to modify the graph adjacency matrix A′. k Normalize by row to form the normalized graph adjacency matrix A. k A k In and A′ k,i,j The corresponding adjacency degree A k,i,j The calculation formula is:

[0089]

[0090] Since the range of cosine similarity is [-1, 1], a value closer to 1 indicates that the two vectors are more similar in direction, while a value less than 0 indicates that they are in opposite directions. Therefore, the above model sets 0 as the lower limit to filter irrelevant nodes when calculating the strength of connecting edges, and uses the Softmax function to apply the similarity matrix A of each multivariate time series segment. k Normalize line by line.

[0091] S214, Finally, for each segment X k Based on the obtained segmented adjacency matrix A k The graph convolution network performs a total of G layers of graph convolution computation, and outputs the final graph convolution result H. (G) Furthermore, the graph convolution process of the l-th layer in the graph convolutional network is represented as follows:

[0092]

[0093] Where D k Let A be the adjacency matrix. k The degree matrix, For a trainable parameter matrix, It is the k-th segment after the final output of the previous graph convolutional layer has been decomposed; l = 1, 2, ..., G.

[0094] The graph convolutional network obtains the segmented convolution result from the graph convolution of the l-th layer. Recombining and stitching the data along the time dimension yields the output of the l-th layer graph convolution:

[0095]

[0096] The graph convolutional network converts the graph convolution result of the l-th layer into a graph convolutional network. The l-th layer graph convolution is obtained by performing a residual connection with the original input multivariate time series segment X. The introduction of residual connections is to prevent gradient vanishing and make the model easier to train.

[0097] Through a multi-layer graph convolution process, H G Each node in the process collects association information from its neighboring nodes and then passes it to the time-dilated convolutional layer.

[0098] S22. Convolve the final graph result H obtained in S21. (G) The input to the temporally dilated convolution module performs a total of C layers of temporally dilated convolutions, yielding the final output H of the temporally dilated convolution module. C The system stacks C layers of temporally dilated convolutional layers. Each layer is a gate linear unit (GLU) with dilated convolution, containing a feature learning part and a semaphore control part. It learns the temporal variation of each variable, and each convolutional layer is divided into two signal paths: a filter and a signal threshold. The feature learning part learns the hidden features of the signal through linear weighting. The semaphore control part uses the sigmoid activation function to control the amount of signal entering the next layer from the current convolutional layer's output. The specific process in the temporally dilated convolution module is shown in S221-S222 below:

[0099] S221. Perform C layers of temporally dilated convolution sequentially, where the process of any l-th layer of temporally dilated convolution is represented as follows:

[0100]

[0101] Where: c = 1, 2, ..., C; Z (c-1) It is the output of the previous dilated convolutional layer. σ(·) represents matrix dot product (i.e., Hadamard product), σ(·) represents the sigmoid activation function, * represents dilated convolution operation, and W f (c) and Let M and C represent the learnable parameters of the two convolutional kernels, respectively. Both kernels have the number M, a kernel size of C1, and a dilation rate of D. Since most time series have relatively fixed sampling frequencies (e.g., 1 hour or 10 minutes) and exhibit periodicity across different time scales, a 3x2 dilated convolution can be used here.n As the dilation rate, it allows the receptive field of the convolution kernel to better cover time scales from small to large (i.e., when the dilation rate is 24 (3*2) n (When the sampling frequency is 1 hour, the receptive field of the convolution kernel will cover the same hour over several consecutive days). The effectiveness of this setting will be demonstrated in subsequent embodiments.

[0102] S222, Obtain the output signals Z of each of the time-dilated convolution layers C. (c) Then, the signal Z output from each time-dilated convolution layer is... (c) After weighted stacking, a regular convolution layer is used to process the signal into the size of the predicted target (i.e., the predicted output size of the multivariate time series prediction model), resulting in the final output of the time-dilated convolution module:

[0103]

[0104] Among them W s (c) Z represents the signal Z generated by the convolution of the c-th layer. (c) The trainable weight matrix for weighted calculation; W e (·) represents a regular convolution with M kernels. e The size of the convolution kernel is C e H C By stacking convolutional outputs with different dilation rates, sequence information from different receptive fields can be captured.

[0105] S23. Input the multivariate time series segment X of the input model into the time information enhancement module to obtain the time information enhancement signal H. A The execution flow in the time information enhancement module is shown in S231 to S234 below:

[0106] S231. First, feature extraction is performed using an LSTM network with N hidden layer units. Then, the hidden layer output H′ of the LSTM network with length W is segmented according to the segmentation number hyperparameter P, resulting in P segmented hidden layer signals H′. k :

[0107] H′=LSTM(X)=[h1,h2,…,h W ]

[0108] H′ k =[h k*S-S+1 ,h k*S-S+2 ,…,h k*S ], k=1,2,…P

[0109] S232, and then the obtained segmented hidden layer signal H′ k After trainable parameters Wk After weighting, we get The corresponding segment weights a′ are obtained by calculating using the Softmax function. k The hidden layer signals H′ of each segment k With the weights a′ of each segment k The weighted summation yields the segmented context vector C′:

[0110]

[0111]

[0112]

[0113] S233, then the segmented context vector C′ is weighted by the trainable parameter W″ to obtain... The hidden layer signal contains S time steps. The time step weights a″ are obtained after calculation using the Softmax function. i The time context vector C″ is composed of the hidden layer signals at each time step. With time step weight a″ i Weighted summation yields:

[0114]

[0115]

[0116]

[0117] S234. Finally, the obtained time context vector C″ is compared with the last time step signal h of the LSTM network hidden layer. W The signal is spliced ​​together and then passed through a fully connected layer to enhance the time information output.

[0118] H A =W A ·concat(C″,h W )+b

[0119] In the formula: W A b and b represent the learnable weights and biases of the fully connected layer, respectively;

[0120] S24. Convert the final output H of the time-dilated convolution module. c The time information enhancement signal H output by the time information enhancement moduleA The sums are then inversely normalized to obtain the final prediction output of the network model:

[0121]

[0122] In the formula: scale represents the inverse normalization coefficient.

[0123] It should be noted that the inverse normalization here corresponds to the normalization in S11. Therefore, the inverse normalization coefficient scale corresponding to the predicted value of each target flow sensor is the maximum value of the sequence corresponding to that sensor, i.e., the aforementioned scale value.

[0124] It should be noted that the specific training method of the above-mentioned multivariate time series prediction model belongs to existing technology, and the loss function used in training can be the mean absolute error (MAE). Specifically, during training, the neural network model prediction output obtained in S24 can be used... The mean absolute error is calculated between the actual label value Y of the predicted target of the input sample and the error, thus obtaining the loss function value L:

[0125]

[0126] Since Y has been normalized beforehand, it needs to be denormalized before MAE calculation can be performed.

[0127] The stochastic gradient of the model can be calculated based on the obtained neural network prediction loss L. The AdamW optimizer is then used to update all trainable parameters of the neural network model with a learning rate α and decaying weights β. This training process is repeated until the model converges.

[0128] S3. In actual prediction, the natural gas flow data within the most recent time window is obtained from the data recorded by each target flow sensor and a real-time multivariate time series segment is formed. After normalization, the real-time multivariate time series segment is input into the trained multivariate time series prediction model to predict the natural gas flow value corresponding to each target flow sensor at a specified time step in the future.

[0129] To demonstrate the advantages of the methods described in S1 to S3 of the present invention, they are applied to a specific example below to illustrate their technical effects.

[0130] Example

[0131] The steps in this embodiment are the same as those described in S1 to S3 above, and will not be repeated here. The following section demonstrates some of the implementation process and results:

[0132] The original data used in this embodiment are natural gas datasets (Nature-gas-1 and Nature-gas-2): both datasets come from a database of industrial and commercial natural gas flow meters provided by a certain company. The original data requires pre-sampling, interpolation, and differencing of the standard cumulative flow of the flow meters to generate natural gas standard flow sequences. Since different natural gas sensors have different upload frequencies, two natural gas datasets were constructed accordingly to demonstrate the model's performance at different sampling frequencies. Natural-gas-1 contains 52 high-frequency natural gas flow sequences from July 1, 2020 to September 5, 2021, with a sampling frequency of 10 minutes; Natural-gas-2 contains 118 low-frequency natural gas flow sequences from March 9, 2020 to September 5, 2021, with a sampling frequency of 1 hour. Due to the special nature of the natural gas industry, some industrial and commercial users suspended gas use around the Spring Festival; therefore, the data for February 2021 in both datasets was deleted in this embodiment. The processed data is divided into a training set (60%), a validation set (20%), and a test set (20%) in chronological order. The model is trained according to the methods described in S1-S2 of this invention, with the validation set as a reference, on the training set. Finally, the model is tested using the test set according to step S3.

[0133] For ease of description, the training method described in S1 to S3 above is referred to as SGTANN. In this embodiment, the parameters of SGTANN are selected as follows: the length of the time window W is 168, the sliding step size s of the time window is 1, the number of segments hyperparameter P is 7, the number of graph convolutional layers G in the graph convolutional network of the segmented graph convolutional module is 2, the number of layers C of the time-dilated convolutional module is 5, the number of convolutional kernels M per layer is 24, the kernel size C1 is 1×3, the dilation rate D of the 5 convolutional kernels is 1, 3, 6, 12, and 24 respectively, and the number of convolutional kernels M in the ordinary convolutional output layer of the time-dilated convolutional module is... e Take 1, the size of the convolution kernel C e The value is 1×76. During the training of the multivariate time series prediction model, the AdamW optimizer updates all trainable parameters of the neural network model, with a learning rate α of 0.0001 and a decay weight β of 0.00001.

[0134] The experiment in this embodiment compares SGTANN with several prediction methods. The prediction methods used as comparisons are: (1) VAR: an autoregressive model; (2) GP: a regression model that uses Gaussian processes for time series modeling; (3) GRU: a recurrent neural network that uses fully connected GRU units; (4) MTGNN: a neural network model based on GCN that uses a self-learned adjacency matrix to describe the correlation between paired variables.

[0135] To verify the technical effectiveness of the present invention, the relative root square error (RSE) and empirical correlation coefficient (CORR) were selected as evaluation indicators for the method, and the calculation formulas are as follows:

[0136]

[0137]

[0138] Where y t,i and These are the true and estimated values ​​of the multivariate time series variable i at time step t, respectively. A smaller RSE value indicates a smaller prediction error by the model for that multivariate time series. CORR is the mean of the empirical correlation coefficients between each time series and its fitted series; a higher value indicates a better fit of the prediction model to the trend of the multivariate time series.

[0139] All methods were tested at different prediction steps, i.e., h = 3, 6, 12, and 24, where different prediction steps represent using the value at the h-th time step after the input time window as the prediction target. The experimental results of each method on the test set are shown in Tables 1 and 2.

[0140] Table 1 shows the performance of all methods on the Nature-gas-1 dataset.

[0141]

[0142] Table 2 shows the performance of all methods on the Nature-gas-2 dataset.

[0143]

[0144] Overall, the RSE of all prediction methods on the datasets increases slowly with the increase of prediction steps, which means that the prediction ability of all prediction methods, including those of this invention, also decreases with the increase of prediction steps. However, a horizontal comparison of different prediction methods shows that traditional statistical method models (i.e., VAR and GP) perform the worst, with their RSE increasing significantly with the increase of prediction steps. Among deep learning models (i.e., GRU, TCN, and LSTNet-skip), TCN outperforms the other two RNN-based models on most datasets, with a lower RSE and higher CORR due to the introduction of dilated causal convolution. Graph neural network-based models (i.e., MTGNN, StemGNN, and SGTANN) outperform traditional statistical learning models and deep learning models in most experiments, demonstrating the effectiveness of introducing graph neural networks. SGTANN achieved leading results on both natural gas datasets, especially in terms of RSE. Except for slightly lower CORR in a few prediction steps, SGTANN generally outperforms other models at different prediction steps. Furthermore, SGTANN performs better on the Nature-gas-1 dataset with a higher sampling frequency than on the Nature-gas-2 dataset with a lower sampling frequency. This demonstrates the effectiveness of the natural gas flow prediction method based on segmented graph convolution and temporal attention mechanisms proposed in this invention.

[0145] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism, characterized in that, Includes the following steps: S1. For each target flow sensor that the user needs to monitor, obtain the historical time series of natural gas flow recorded by each target flow sensor. Normalize each series and then use a fixed-length time window to slide extract data on the series, generating labeled training samples and constructing a training sample set. The input of each training sample is a multivariate time series segment composed of natural gas flow data from each target flow sensor within a time window. The label is the recorded value of each target flow sensor at a specified time step after that time window. All labeled training samples constitute the training sample set. S2. Using the training sample set obtained in S1... A multivariate time series prediction model based on piecewise graph convolution and temporal attention mechanisms is trained until convergence, enabling it to predict the recorded values ​​of each target flow sensor at a specified time step after the input data's time window. The multivariate time series prediction model includes a piecewise graph convolution module, a time-dilated convolution module, and a time information enhancement module. The input to the model is a multivariate time series segment. The piecewise graph convolution module segments the input multivariate time series segment according to the time dimension, constructs a segmented adjacency matrix based on the cosine similarity between sequences within a segment, and then performs multi-level segmentation. The graph convolution module extracts the correlation between sequences to obtain the first output signal. The time-dilated convolution module inputs the first output signal from the segmented graph convolution module into a multi-layered stacked threshold control unit with dilated convolution to further extract features within each sequence and reduce dimensionality to obtain the second output signal. The time information enhancement module inputs the input multivariate time series segments into the LSTM network, groups the signals of each hidden step of the LSTM network according to time, and applies an attention mechanism to each group to extract segmented context vectors. Then, it continues to apply an attention mechanism to the segmented context vectors to generate a time context vector. Finally, it combines the time context vectors with the time context vectors. The signal is concatenated with the last signal of the hidden layer of the LSTM network, and the concatenation result is passed through a fully connected layer to obtain the third output signal. Finally, the second and third output signals are added together and inversely normalized to obtain the final prediction output of the multivariate time series prediction model. S3. In actual prediction, the natural gas flow data within the most recent time window is obtained from the data recorded by each target flow sensor and a real-time multivariate time series segment is formed. The real-time multivariate time series segment is normalized and input into the trained multivariate time series prediction model to predict the natural gas flow value corresponding to each target flow sensor at a specified time step in the future.

2. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism as described in claim 1, characterized in that, S1 specifically includes the following sub-steps: S11, obtaining the user's monitoring requirements. Historical time series of natural gas flow recorded by a target flow sensor Historical time series of natural gas flow Normalization is achieved by using the maximum value of each element as the scale value, resulting in a normalized time series. From all normalized time series Composition of multivariate time series S12, using a fixed-length W time window in a multivariate time series The slide proceeds according to a preset step size s, with each time window transitioning from a multivariate time series during the slide. Extracting multivariate time series segments within a window and in multivariate time series The above is located after the time window. The value at each time step As regression labels for the model, the training dataset for the model is generated, where each sample is represented as { }。 3. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism as described in claim 1, characterized in that, In step S2, the specific processing steps in the multivariate time series prediction model based on segmented graph convolution and temporal attention mechanism are as follows: S21, input the multivariate time series segments into the model... In the input segmented graph convolution module, the input multivariate time series segments are first processed according to the preset segmentation number hyperparameter P. Divide the time dimension into P equal segments, each segment being represented as... ， Where W represents a fixed length; then segmented The sequences corresponding to each target flow sensor within the system are nodes, and cosine similarity is used to calculate segmentation. The adjacency of internal nodes is used to obtain the segmentation. Corresponding graph adjacency matrix ， any node and nodes In the same segment The strength of the connecting edge is expressed as: In the formula: and Each represents a segment. The sequence corresponding to the i-th and j-th target flow sensors within the graph; then, the Softmax function is used to optimize the graph adjacency matrix. Normalize by row to form a normalized graph adjacency matrix. ,in Zhongyu Corresponding adjacency The calculation formula is: Finally, for each segment Based on the obtained segmented adjacency matrix Total output via graph convolutional network Layered graph convolution calculation, outputting the final graph convolution result. ; and the first in the graph convolutional network The graph convolution process of a layer is represented as follows: in Adjacency matrix The degree matrix, For a trainable parameter matrix, It is the final output of the previous graph convolutional layer after decomposition. Each segment; ， ;Graph convolutional networks will... Piecewise convolution results obtained from layer graph convolution By recombining and piecing together the data along the time dimension, we obtain the [number]. Output of layer graph convolution: Graph convolutional networks will... Layered graph convolution results Multivariate time series segments of the original input Perform residual join to obtain the first... The final output of layer graph convolution S22. Convolve the final graph obtained in S21. In the input time-dilated convolution module, a total of The time-dilated convolution of the layer, where the first layer... The process of layer-time dilated convolution is represented as follows: in It is the output of the previous dilated convolutional layer, and ⊗ represents matrix dot product. The symbol represents the sigmoid activation function, and * represents the dilated convolution operation. Let M and M represent the learnable parameters of the two convolutional kernels, respectively. The number of kernels in both kernels is M, and the size of the kernels is M. The expansion rate is ； The signal output of each time-dilated convolution layer After weighted summation, a regular convolution layer is used to process the signal into the predicted target size, resulting in the final output of the time-dilated convolution module: in Indicates the first signals from layer convolution The trainable weight matrix for weighted calculation; This represents a regular convolution, with the number of kernels being... The size of the convolution kernel is S23. Input the multivariate time series segments into the model. In the input time information enhancement module, the first step is to have... Feature extraction is performed on an LSTM network with 10 hidden layer units, and the hidden layer output of the LSTM network of length W is adjusted according to the hyperparameter P of the number of segments. The signal is segmented into P segments to obtain the segmented hidden layer signal. ： Then the obtained segmented hidden layer signals Trainable parameters After weighting, we get ， The corresponding segment weights are obtained by calculating using the Softmax function. The hidden layer signals of each segment with the weights of each segment Weighted summation yields the segmented context vector ： Then segment the context vector Trainable parameters After weighting, we get ， The hidden layer signal contains S time steps. ， The time step weights are calculated using the Softmax function. Time context vector Hidden layer signals at each time step With time step weights Weighted summation yields: Finally, the obtained time context vector The last time step signal of the hidden layer of the LSTM network The signal is spliced ​​together and then passed through a fully connected layer to enhance the time information output. In the formula: and S1 and S24 represent the learnable weights and biases of the fully connected layer, respectively; S24 represents the final output of the time-dilated convolution module. And the time information enhancement signal output by the time information enhancement module The sums are then inversely normalized to obtain the final prediction output of the network model: In the formula: This represents the inverse normalization coefficient.

4. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism according to claim 1, characterized in that, The length of the time window Take 168, the sliding step of the time window Take 1.

5. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism according to claim 3, characterized in that, The hyperparameter for the number of segments It is 7.

6. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism according to claim 3, characterized in that, The number of graph convolutional layers in the graph convolutional network The value is 2.

7. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism according to claim 1, characterized in that, The number of time-dilated convolutional layers in the time-dilated convolutional module The number of kernels per layer is 5. The kernel size is 24. for The dilation rate of 5 convolutional kernels Take the values ​​1, 3, 6, 12, and 24 respectively.

8. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism according to claim 1, characterized in that, In the time-dilated convolution module, the number of convolution kernels in the last ordinary convolution layer... Take 1, the size of the convolution kernel Pick 。 9. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism as described in claim 1, characterized in that, The loss function used during model training of the multivariate time series prediction model is the mean absolute error.

10. The natural gas flow prediction method based on piecewise graph convolution and temporal attention mechanism according to claim 1, characterized in that, During model training, the multivariate time series prediction model uses the AdamW optimizer to update all trainable parameters of the neural network model, and the learning rate... Pick Decay weight Take 0.00001.

Images