A factory power consumption prediction method based on TSPG architecture

Abstract

The application belongs to the field of time series data processing and prediction, and discloses a factory power consumption prediction method based on TSPG architecture. The method comprises the following steps: step 1, data acquisition: collecting multivariate time series data of the factory, including production line power consumption and environmental temperature; step 2, data preprocessing: standardizing the multivariate time series data to obtain standardized data; step 3, constructing a TSPG model; step 4, model training; step 5, model prediction and evaluation: inputting test set data into the trained TSPG model to generate factory power consumption prediction at future time. The method has better performance than traditional benchmark models such as Transformer and PatchTST in factory power consumption prediction, and has higher prediction accuracy and lower computational complexity. The application is suitable for power consumption prediction in the fields of industry and energy management, and provides effective decision support for production scheduling and energy optimization, without the need for additional hardware upgrades and can be quickly deployed and implemented.

Description

Technical Field

[0001] This invention belongs to the field of time series data processing and prediction, specifically relating to a factory power consumption prediction method based on the TSPG architecture. Background Technology

[0002] In the field of factory electricity consumption forecasting, accurate forecasting is crucial for improving industrial energy efficiency and achieving sustainable production. Factory electricity consumption is influenced by various factors, such as production scheduling, equipment status, and environmental conditions, resulting in noisy, nonlinear, and non-stationary characteristics in its time-series data, making forecasting extremely challenging. Although the rapid development of information technology and machine learning has provided new solutions for electricity consumption forecasting, existing methods still face many limitations.

[0003] Traditional prediction methods, such as recurrent neural networks (RNNs), are prone to gradient vanishing or exploding problems when processing long-sequence data, limiting their predictive performance. While the Transformer model has achieved success in fields such as natural language processing due to its self-attention mechanism, it still suffers from high computational complexity and sensitivity to outlier data in time series prediction, leading to inefficiency in long-sequence applications. Furthermore, traditional self-attention mechanisms assign the same level of attention to all time points in the input sequence, which can lead to overfitting to noisy and unimportant data points when processing factory electricity consumption data, affecting prediction accuracy. Summary of the Invention

[0004] To address the aforementioned technical challenges, we propose a factory power consumption forecasting method based on the TSPG architecture.

[0005] The proposed TSPG-based factory power consumption forecasting method demonstrates excellent performance in industrial power consumption forecasting. By introducing a patch embedding mechanism and gated attention units (GAU), we effectively address the problems of high computational complexity and sensitivity to outlier data points inherent in traditional Transformer models. Furthermore, the TSPG model exhibits high prediction accuracy and generalization ability, making it adaptable to various industrial power consumption scenarios. Therefore, this invention is of great significance for improving the accuracy and efficiency of factory power consumption forecasting.

[0006] Specifically, the TSPG model includes the following innovations:

[0007] Patch Mechanism: In the TSPG model, we first introduce the patch mechanism to handle time series data. By dividing the time series data into multiple data patches, the model can better capture the local and global features of the data and reduce the impact of outlier data points on attention calculation. The patch mechanism effectively reduces the input dimensionality of the model, thereby reducing computational complexity.

[0008] Gated Attention Unit (GAU): In traditional Transformer models, self-attention is a core component. However, self-attention suffers from high computational complexity when dealing with long-term series. To address this issue, we introduce GAU into the TSPG model. GAU simplifies attention computation by introducing a gating mechanism, reducing computational overhead, and while maintaining the core functionality of the attention mechanism, it reduces global dependence on input data, thereby improving prediction efficiency and robustness.

[0009] Time-Sensitive L1 Loss Function: To further improve the prediction accuracy of the model, we introduce a time-sensitive L1 loss function with decaying weights. This loss function reduces the influence of older data on the prediction by assigning higher weights to more recent time points, thereby improving the stability and accuracy of the prediction in non-stationary data environments.

[0010] Experimental Validation of the TSPG Model: To verify the effectiveness of the TSPG model in factory electricity consumption forecasting, we conducted extensive experiments. The results show that, compared with the traditional Transformer model and other advanced time series forecasting models, the TSPG model outperforms in prediction accuracy, computational complexity, and execution efficiency, validating its potential value and reliability in industrial applications.

[0011] The technical solution of the present invention is as follows:

[0012] A factory power consumption forecasting method based on TSPG architecture includes the following steps:

[0013] Step 1: Data Acquisition: Collect multivariate time series data of the factory, including electricity consumption of the production line and ambient temperature;

[0014] Step 2, Data Preprocessing: Standardize the multivariate time series data to obtain standardized data, then sort the data in chronological order and divide it into training and test sets, and normalize each time period.

[0015] Step 3: Construct the TSPG model, which consists of a Patch embedding module, an encoder-decoder structure, a gated attention unit (GAU), and a multilayer perceptron (MLP).

[0016] Step 4, Model Training: Train the TSPG model using the training set data, and evaluate the model's generalization ability and prediction accuracy using the validation set data.

[0017] Step 5, Model Prediction and Evaluation: Input the test set data into the trained TSPG model to generate future power consumption predictions for the factory. Evaluate the model's predictive performance by comparing and analyzing the predictions with actual power consumption data.

[0018] Furthermore, step 2 includes:

[0019] For multivariate time series data Standardization is performed to eliminate differences between different units of measurement and to adapt the data to the model's input requirements, where L is the number of time steps and F is the number of features. Standardization uses instance normalization, and its formula is as follows:

[0020]

[0021] in: It is standardized data; X ij μ represents the data value of the j-th feature at the i-th time step; i Let σ be the mean of all features at the i-th time step; i Let be the standard deviation of all features at the i-th time step.

[0022] Furthermore, step 3 includes:

[0023] Step 3.1: Input the standardized data into the Patch embedding module for block processing to generate Patch data, thereby capturing local and global features in the time series;

[0024] The standardized time series data is Where L is the number of time steps and F is the number of features, the time series data is divided into several blocks of fixed length using a sliding window. The specific block division process is as follows:

[0025] The time series data is divided into blocks using a sliding window. The length of the sliding window is P, which is the number of time steps contained in each patch, and the sliding step size is S. The number of patches after block division is:

[0026]

[0027] Each patch data point is linearly mapped to a high-dimensional feature space using weights and bias parameters to generate a patch embedding:

[0028] X 2D =Unfold(ReplicationPad(X) 1D ))

[0029] Standardized time series data Converted into a collection of multiple patch data:

[0030] {PatchEmbedding(X (0) ),PatchEmbedding(X (1) ),…,PatchEmbedding(X (N-1) )}

[0031] Step 3.2: The standardized and embedded Patch dataset is used as input and passed to the encoder-decoder structure to extract features. Attention is calculated on the Patch data through gated attention units (GAU) to generate encoded-decoded features.

[0032] Step 3.3: Employ a time-sensitive L1 loss function with decay weights. The decay weights prioritize nearby data points. The time-sensitive L1 loss function is as follows:

[0033]

[0034] Where α is the decay factor controlling the decay rate, t is the time distance from the predicted value, and the weight vector ω is defined as decreasing with time. i =e -αt .

[0035] Step 3.4: Input the decoded features into a Multilayer Perceptron (MLP) layer, and process the decoded features through multiple fully connected layers.

[0036] The features are further processed to extract high-level features, and the output layer generates a prediction of the factory's future electricity consumption; furthermore, the specific process of step 3.2 is as follows:

[0037] The encoder structure, where the embedding of each patch is represented as follows: All patch data are integrated into a matrix format:

[0038]

[0039] Gated attention units (GAUs) are used for efficient attention calculation, and their calculation formula is as follows:

[0040] Output = (U⊙AV)W O

[0041] Where U and V are tensors obtained through linear transformation of the input tensor H0, and W... O As weight;

[0042] U = H O W U V = H O W V

[0043] WU , This is the weight matrix;

[0044] A is the attention weight matrix, calculated using the following formula:

[0045]

[0046] Here, Q(Z) and K(Z) are obtained through a simple mapping transformation, Z is obtained through a linear transformation of the input sequence, s is the number of attention heads, B is the bias, and the ReLU function is used as the activation function. The formula for the ReLU function is as follows:

[0047] f(x) = max(0,x)

[0048] Let x be the input independent variable;

[0049] The encoder processes the input data through multiple stacked GAU modules to extract high-dimensional encoded feature representations.

[0050] The decoder receives the encoder output H enc and historical input of the current time Decode:

[0051]

[0052] The decoder uses a GAU module similar to the encoder for feature extraction, focusing on combining encoded features H. enc and the input at the current moment Decoding features are generated step by step.

[0053] Furthermore, step 3.4 specifically includes:

[0054] An MLP layer consists of one or more fully connected layers. Each layer processes the input features through linear transformations and nonlinear activation functions. The output features of the l-th layer are represented as follows:

[0055] H (l) =ReLU(H (l-1) W (l) +b (l) )

[0056] in: The output features of the (l-1)th layer, These are the weight matrix and bias vector of the l-th layer, respectively, D l-1 and D l These are the feature dimensions of the (l-1)th and lth layers, respectively, with the ReLU function used as the activation function;

[0057] MLPs typically consist of several layers, with each layer progressively extracting features at a higher level. The feature dimension output by the last layer is the size required for the target prediction. Through the linear transformation of the last layer, the MLP generates a prediction of the factory's future electricity consumption.

[0058] Furthermore, step 5 includes: evaluating the model using mean squared error (MSE) and mean absolute error (MAE);

[0059] The formula for mean squared error (MSE) is:

[0060]

[0061] The formula for Mean Absolute Error (MAE) is:

[0062]

[0063] Among them, the predicted value is Actual value y i With a sample size of n, the smaller the mean squared error (MSE) and mean absolute error (MAE), the closer the predicted value is to the true value.

[0064] The beneficial effects of this invention are:

[0065] (1) The Patch mechanism reduces the model's reliance on anomalous data when capturing local and global features by dividing time series data into block inputs, thereby enhancing its adaptability to complex time series; GAU simplifies the computation process and reduces computational complexity by introducing a gating mechanism to replace the traditional multi-head self-attention, thus improving the model's efficiency in long-sequence prediction; the time-sensitive L1 loss function with decaying weights aims to prioritize recent data and reduce the influence of old data to improve the stability and accuracy of prediction, further enhancing prediction performance.

[0066] (2) The TSPG model based on the Patch mechanism and GAU has significant application value in factory electricity consumption forecasting. By reducing computational complexity and optimizing attention calculation, the model solves the shortcomings of traditional Transformer in time series forecasting, providing a more efficient and accurate electricity consumption forecasting solution for the industrial field, and helping intelligent production and energy management. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a flowchart of the present invention;

[0069] Figure 2 This is a structural diagram of the TSPG architecture model;

[0070] Figure 3 This is a comparison chart of the time costs of the TSPG model. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] like Figure 1 As shown, this application provides a factory power consumption forecasting method based on the TSPG architecture, including the following steps:

[0073] Step 1: Data Acquisition: Collect multivariate time series data of the factory, including electricity consumption of the production line and ambient temperature;

[0074] Step 2, Data Preprocessing: Standardize the multivariate time series data to obtain standardized data, then sort the data in chronological order and divide it into training and test sets, and normalize each time period.

[0075] For multivariate time series data Standardization is performed to eliminate differences between different units of measurement and to adapt the data to the model's input requirements, where L is the number of time steps and F is the number of features. Standardization uses instance normalization, and its formula is as follows:

[0076]

[0077] in: It is standardized data; X ij μ represents the data value of the j-th feature at the i-th time step; i Let σ be the mean of all features at the i-th time step; i Let be the standard deviation of all features at the i-th time step.

[0078] Step 3: Construct the TSPG model, such as Figure 2 As shown, the model consists of a Patch embedding module, an encoder-decoder structure, a gated attention unit (GAU), and a multilayer perceptron (MLP).

[0079] Step 3.1: Input the standardized data into the Patch embedding module for block processing to generate Patch data, thereby capturing local and global features in the time series;

[0080] The standardized time series data is Where L is the number of time steps and F is the number of features, the time series data is divided into several blocks of fixed length using a sliding window. The specific block division process is as follows:

[0081] The time series data is divided into blocks using a sliding window. The length of the sliding window is P, which is the number of time steps contained in each patch, and the sliding step size is S. The number of patches after block division is:

[0082]

[0083] Each patch data point is linearly mapped to a high-dimensional feature space using weights and bias parameters to generate a patch embedding:

[0084] X 2D =Unfold(ReplicationPad(X) 1D ))

[0085] Standardized time series data Converted into a collection of multiple patch data:

[0086] {PatchEmbedding(X (0) ),PatchEmbedding(X (1) ),…,PatchEmbedding(X (N-1) )}

[0087] Step 3.2: The standardized and embedded Patch dataset is used as input and passed to the encoder-decoder structure to extract features. Attention is calculated on the Patch data through gated attention units (GAU) to generate encoded-decoded features.

[0088] The encoder structure, where the embedding of each patch is represented as follows: All patch data are integrated into a matrix format:

[0089]

[0090] Gated attention units (GAUs) are used for efficient attention calculation, and their calculation formula is as follows:

[0091] Output = (U⊙AV)W O

[0092] Where U and V are tensors obtained through linear transformation of the input tensor H0, and W... O As weight;

[0093] U = H O W U V = H O W V

[0094] W U , This is the weight matrix;

[0095] A is the attention weight matrix, calculated using the following formula:

[0096]

[0097] Here, Q(Z) and K(Z) are obtained through a simple mapping transformation, Z is obtained through a linear transformation of the input sequence, s is the number of attention heads, B is the bias, and the ReLU function is used as the activation function. The formula for the ReLU function is as follows:

[0098] f(x) = max(0,x)

[0099] Let x be the input independent variable;

[0100] The encoder processes the input data through multiple stacked GAU modules to extract high-dimensional encoded feature representations.

[0101] The decoder receives the encoder output H enc and historical input of the current time Decode:

[0102]

[0103] The decoder uses a GAU module similar to the encoder for feature extraction, focusing on combining encoded features H. enc and the input at the current moment Decoding features are generated step by step.

[0104] Step 3.3: Employ a time-sensitive L1 loss function with decay weights. The decay weights prioritize nearby data points. The time-sensitive L1 loss function is as follows:

[0105]

[0106] Where α is the decay factor controlling the decay rate, t is the time distance from the predicted value, and the weight vector ω is defined as decreasing with time. i =e -αt .

[0107] Step 3.4: Input the decoded features into the Multilayer Perceptron (MLP) layer, and further process the decoded features through multiple fully connected layers to extract high-level features. Then, generate the prediction results of the future electricity consumption of the factory through the output layer.

[0108] An MLP layer consists of one or more fully connected layers. Each layer processes the input features through linear transformations and nonlinear activation functions. The output features of the l-th layer are represented as follows:

[0109] H (l) =ReLU(H (l-1) W (l) +b (l) )

[0110] in: The output features of the (l-1)th layer, These are the weight matrix and bias vector of the l-th layer, respectively, D l-1 and D l These are the feature dimensions of the (l-1)th and lth layers, respectively, with the ReLU function used as the activation function;

[0111] MLPs typically consist of several layers, with each layer progressively extracting features at a higher level. The feature dimension output by the last layer is the size required for the target prediction. Through the linear transformation of the last layer, the MLP generates a prediction of the factory's future electricity consumption.

[0112] Step 4, Model Training: Train the TSPG model using the training set data, and evaluate the model's generalization ability and prediction accuracy using the validation set data.

[0113] During the model training phase, we trained the TSPG model using the training set data. We employed a time-sensitive L1 loss function combined with decaying weights to prioritize recent data and reduce the influence of older data, thereby improving the model's predictive accuracy. Through multiple iterations, we updated the model parameters to ensure that the model gradually learned the changing patterns of factory electricity consumption. We then used the validation set data to evaluate the model's generalization ability and predictive accuracy.

[0114] Step 5, Model Prediction and Evaluation: Input the test set data into the trained TSPG model to generate future power consumption predictions for the factory. Evaluate the model's predictive performance by comparing and analyzing the predictions with actual power consumption data.

[0115] like Figure 3The TSPG architecture model demonstrates superior performance in terms of both time overhead and accuracy in predicting factory electricity consumption. Compared to other mainstream architectures, the TSPG architecture has lower time overhead. As shown in Table 1, the TSPG architecture model performs exceptionally well in predicting factory electricity consumption. Evaluation metrics using mean squared error (MSE) and mean absolute error (MAE) indicate that the TSPG model significantly outperforms traditional models such as Transformer and PatchTST in both prediction accuracy and computational efficiency.

[0116] Mean squared error (MSE) is a commonly used metric for evaluating regression models, measuring the squared average of the differences between predicted and actual values. It is more sensitive to samples with larger prediction errors, making it suitable for capturing the degree to which a model deviates from outliers. and actual value y i If the sample size is n, then the formula for the mean squared error (MSE) is:

[0117]

[0118] Mean Absolute Error (MAO) is a metric used to assess the average magnitude of prediction errors; it directly measures the average absolute value of the difference between predicted and actual values. It is insensitive to outliers and is suitable for evaluating the overall prediction error of a model. and actual value y i If the sample size is n, then the formula for the mean absolute error (MAE) is:

[0119]

[0120] The smaller the mean squared error (MSE) and mean absolute error (MAE), the closer the predicted value is to the true value.

[0121] Table 1. Comparison of the TSPG model's performance in predicting factory electricity consumption.

[0122]

[0123] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A factory power consumption forecasting method based on TSPG architecture, characterized in that, The method includes the following steps: Step 1: Data Acquisition: Collect multivariate time series data of the factory, including electricity consumption of the production line and ambient temperature; Step 2, Data Preprocessing: Standardize the multivariate time series data to obtain standardized data, then sort the data in chronological order and divide it into training and test sets, and normalize each time period. Step 3: Construct the TSPG model, which consists of a Patch embedding module, an encoder-decoder structure, a gated attention unit (GAU), and a multilayer perceptron (MLP). Step 4, Model Training: Train the TSPG model using the training set data, and evaluate the model's generalization ability and prediction accuracy using the validation set data. Step 5, Model Prediction and Evaluation: Input the test set data into the trained TSPG model to generate future power consumption predictions for the factory. Evaluate the model's predictive performance by comparing and analyzing the predictions with actual power consumption data.

2. The factory power consumption forecasting method based on TSPG architecture as described in claim 1, characterized in that, Step 2 includes: For multivariate time series data Standardization is performed to eliminate differences between different units of measurement and to adapt the data to the model's input requirements, where L is the number of time steps and F is the number of features. Standardization uses instance normalization, and the formula is as follows: in: It is standardized data; X ij μ represents the data value of the j-th feature at the i-th time step; i Let σ be the mean of all features at the i-th time step; i Let be the standard deviation of all features at the i-th time step.

3. The factory power consumption forecasting method based on TSPG architecture as described in claim 1, characterized in that, Step 3 includes: Step 3.1: Input the standardized data into the Patch embedding module for block processing to generate Patch data, thereby capturing local and global features in the time series; The standardized time series data are Where L is the number of time steps and F is the number of features, the time series data is divided into several blocks of fixed length using a sliding window. The specific block division process is as follows: The time series data is divided into blocks using a sliding window. The length of the sliding window is P, which is the number of time steps contained in each patch, and the sliding step size is S. The number of patches after block division is: Each patch data point is linearly mapped to a high-dimensional feature space using weights and bias parameters to generate a patch embedding: X 2D =Unfold(ReplicationPad(X 1D )) Standardized time series data Converted into a collection of multiple patch data: {PatchEmbedding(X (0) ),PatchEmbedding(X (1) ),…,PatchEmbedding(X (N-1) )} Step 3.2: The standardized and embedded Patch dataset is used as input and passed to the encoder-decoder structure to extract features. Attention is calculated on the Patch data through gated attention units (GAU) to generate encoded-decoded features. Step 3.3: Employ a time-sensitive L1 loss function with decay weights. The decay weights prioritize nearby data points. The time-sensitive L1 loss function is as follows: Where α is the decay factor controlling the decay rate, t is the time distance from the predicted value, and the weight vector ω is defined as decreasing with time. i =e -αt . Step 3.4: Input the decoded features into a multilayer perceptron (MLP) layer, and further process the decoded features through multiple fully connected layers to extract high-level features. Then, generate the prediction results of the future electricity consumption of the factory through the output layer.

4. The factory power consumption forecasting method based on TSPG architecture as described in claim 3, characterized in that, The specific process of step 3.2 is as follows: The encoder structure, where the embedding of each patch is represented as follows: All patch data are integrated into a matrix format: Gated attention units (GAUs) are used for efficient attention calculation, and their calculation formula is as follows: Output=(U⊙AV)W O Where U and V are tensors obtained through linear transformation of the input tensor H0, and W... O As weight; U=H O W U V=H O W V This is the weight matrix; A is the attention weight matrix, calculated using the following formula: Here, Q(Z) and K(Z) are obtained through a simple mapping transformation, Z is obtained through a linear transformation of the input sequence, s is the number of attention heads, B is the bias, and the ReLU function is used as the activation function. The formula for the ReLU function is as follows: f(x) = max(0,x) Where x is the input independent variable; The encoder processes the input data through multiple stacked GAU modules to extract high-dimensional encoded feature representations. The decoder receives the encoder output H enc and historical input of the current time Decode: The decoder uses a GAU module similar to the encoder for feature extraction, focusing on combining encoded features H. enc and the input at the current moment Decoding features are generated step by step.

5. The factory power consumption forecasting method based on TSPG architecture as described in claim 1, characterized in that, Step 3.4 specifically involves: An MLP layer consists of one or more fully connected layers. Each layer processes the input features through linear transformations and nonlinear activation functions. The output features of the l-th layer are represented as follows: A (l) =ReLU(H (l-1) W (l) +b (l) ) in: The output features of the (l-1)th layer, These are the weight matrix and bias vector of the l-th layer, respectively, D l-1 and D l These are the feature dimensions of the (l-1)th and lth layers, respectively, with the ReLU function used as the activation function; MLPs typically consist of several layers, with each layer progressively extracting features at a higher level. The feature dimension output by the last layer is the size required for the target prediction. Through the linear transformation of the last layer, the MLP generates a prediction of the factory's future electricity consumption.

6. The factory power consumption forecasting method based on TSPG architecture as described in claim 1, characterized in that, Step 5 includes: evaluating the model using mean squared error (MSE) and mean absolute error (MAE); the formula for mean squared error (MSE) is: The formula for Mean Absolute Error (MAE) is: Among them, the predicted value is Actual value y i With a sample size of n, the smaller the mean squared error (MSE) and mean absolute error (MAE), the closer the predicted value is to the true value.

Images