Time prediction method and device based on different channel attention focusing mechanism

Abstract

The application belongs to the technical field of time prediction, and particularly relates to a time prediction method and device based on different channel attention focusing mechanisms. A scaled household electricity time sequence data vector is input into an MLP to obtain a processed output vector, and a stationary wavelet transform is performed on the output vector; based on high-frequency detail components, Granger causality analysis is performed, inverse Granger transformation is performed on the high-frequency detail components, and reconstructed household electricity time sequence data is obtained; based on low-frequency components and medium-frequency components, the correlation between different channel attentions is calculated, and multiple channels with the strongest dependency are screened; self-attention coupling is used for the channels with strong dependency, and geometric self-attention coupling is used for the channels with weak dependency; the outputs of the self-attention and the geometric self-attention are fused to obtain comprehensive features; the reconstructed frequency domain sequence data and the comprehensive features are inversely transformed again, and a final prediction result is obtained through an FFN.

Description

Technical Field

[0001] This invention belongs to the field of time prediction technology, specifically relating to a time prediction method and apparatus based on different channel attention mechanisms. Background Technology

[0002] With the rapid development of information technology and data acquisition methods, data generated by various systems increasingly exhibits temporal correlation and sequential characteristics. In the historical observation of household electricity consumption in urban neighborhoods, a large number of decision-making activities rely on the scientific prediction of future trends to cope with the uncertainties of environmental change and resource allocation. Against this backdrop, "time series forecasting," as a method of inferring future behavior by analyzing historical data sequences, will become an important research direction in data analysis and intelligent decision-making.

[0003] Traditional time series models such as autoregressive (AR), moving average (MA), and ARIMA perform well in describing linear relationships, but they have limitations in handling nonlinear, long-term dependent, and multidimensional complex structures. In recent years, with the rapid development of machine learning and deep learning, the application of novel models based on recurrent neural networks (RNN), long short-term memory networks (LSTM), and Transformers in time series forecasting has deepened, significantly improving their accuracy and generalization capabilities.

[0004] However, existing time series predictions based on historical household electricity consumption observations do not consider the impact of external noise on the internal channels; they only focus on capturing the correlations between different channels. Furthermore, the captured channels are either all captured or only a portion are randomly captured, without a deep understanding of the correlations between channels. Summary of the Invention

[0005] To overcome the problems in the prior art, this invention proposes a time prediction method and apparatus based on different channel attention mechanisms.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0007] In a first aspect, the present invention provides a time prediction method based on different channel attention mechanisms, comprising the following steps:

[0008] The input household electricity consumption time series data is processed in batches. The household electricity consumption time series data of each channel in each batch is normalized and scaled to obtain the scaled household electricity consumption time series data vector.

[0009] The scaled household electricity consumption time series data vector is input into a pre-trained MLP model to obtain the processed output vector; the output vector is then subjected to a stationary wavelet transform to obtain high-frequency detail components, low-frequency components, and mid-frequency components.

[0010] Granger causality analysis is performed based on high-frequency detail components; inverse Granger transform is then performed on the high-frequency detail components after Granger causality analysis to obtain the reconstructed frequency domain sequence data.

[0011] Based on low-frequency and mid-frequency components, the correlation between attention in different channels is calculated; based on the correlation between attention in different channels, the channels with the strongest dependence are selected; self-attention coupling is used for channels with strong dependence, and geometric self-attention coupling is used for channels with weak dependence. The outputs of self-attention and geometric self-attention are fused to obtain a comprehensive feature representation.

[0012] The reconstructed frequency domain sequence data is then subjected to an inverse stationary wavelet transform with the comprehensive feature representation to convert it from the frequency domain to the time domain, thus obtaining the power time series data features. Based on the power time series data features, a household electricity consumption time series prediction model is constructed and trained, and the trained household electricity consumption time series prediction model is used for prediction.

[0013] Furthermore, the household electricity consumption time series data for each channel in each batch is normalized and scaled to obtain a scaled household electricity consumption time series data vector, including:

[0014] RMS values ​​are obtained by performing RMS normalization on the time series data of household electricity consumption for each channel in each batch.

[0015] The RMS value is used to normalize the input household electricity consumption time series data to obtain normalized household electricity consumption time series data.

[0016] By introducing a learnable scaling parameter, the output scale of the normalized household electricity consumption time series data is adjusted to obtain a scaled household electricity consumption time series data vector.

[0017] Furthermore, the scaled household electricity consumption time series data vector is input into the pre-trained MLP model to obtain the processed output vector, including:

[0018] Construct a multilayer perceptron (MLP) model, which includes an input layer, a hidden layer, and an output layer;

[0019] The MLP model is trained using training data to obtain a trained MLP model;

[0020] The scaled household electricity consumption time series data is input into the trained MLP model. The MLP model performs feature extraction and nonlinear transformation on the input scaled household electricity consumption time series data, and finally outputs the prediction result of the regression task, that is, the processed output vector.

[0021] Furthermore, a stationary wavelet transform is performed on the output vector to obtain high-frequency detail components, low-frequency components, and mid-frequency components, including:

[0022] Based on low-pass and high-pass filters, calculate the low-frequency approximation components and high-frequency detail components of each layer;

[0023] The low-frequency approximation components of each layer are further decomposed into approximation components and detail components of the next layer. The above process is repeated until the preset number of decomposition layers is reached, and finally the high-frequency detail components of each scale and the relative low-frequency approximation components of the last layer are obtained. The relative low-frequency approximation components include low-frequency components and mid-frequency components.

[0024] Furthermore, an inverse Granger transform is performed on the high-frequency detail components after Granger causality analysis to obtain the reconstructed frequency domain sequence data, including:

[0025] From the high-frequency detail components, we screen out the high-frequency detail components that have a significant Granger causal relationship with the Granger transform target variable;

[0026] The filtered high-frequency detail components and the relatively low-frequency approximation components of the last layer are recombined using the stationary wavelet inverse transform to form the reconstructed frequency domain sequence data.

[0027] Furthermore, self-attention coupling is used for channels with strong dependencies, and geometric self-attention coupling is used for channels with weak dependencies. The outputs of self-attention and geometric self-attention are fused to obtain a comprehensive feature representation.

[0028] For channels with strong dependencies, self-attention coupling is used. Self-attention weights are calculated using the query matrix and key matrix. Based on the self-attention weights and value matrix, a feature representation that strengthens global dependencies is obtained.

[0029] For channels with weak dependencies, geometric self-attention coupling is used. Attention weights with geometric relationships are calculated through query matrix, key matrix and value matrix. Based on the attention weights with geometric relationships and value matrix, features that fuse local geometric relationships are obtained.

[0030] By combining feature representations that enhance global dependencies with feature fusion that integrates local geometric relationships, a comprehensive feature representation is obtained.

[0031] Furthermore, the reconstructed frequency domain sequence data is subjected to an inverse stationary wavelet transform with the comprehensive feature representation to convert it from the frequency domain to the time domain, obtaining the power time series data features, including:

[0032] ;

[0033] in, Indicates the characteristics of power time series data; It is a scale Approximation coefficients, It is a scale Detail factor; scale This refers to the decomposition depth of the wavelet transform, that is, the decomposition into several approximation coefficients and detail coefficients; It is a low-pass filter in the inverse transform process. It is a high-pass filter in the inverse transform process.

[0034] Furthermore, the household electricity consumption time series data prediction model adopts FFN. When training the household electricity consumption time series data prediction model, the training data includes input data and corresponding label data. The input data is the electricity time series data feature, and the label data is the true value corresponding to the input data.

[0035] Secondly, a time prediction device based on different channel attention mechanisms is provided, the device comprising:

[0036] The preprocessing module is used to process the input household electricity consumption time series data in batches, and to normalize and scale the household electricity consumption time series data of each channel in each batch to obtain the scaled household electricity consumption time series data vector.

[0037] The MLP module is used to input the scaled household electricity consumption time series data vector into the pre-trained MLP model to obtain the processed output vector;

[0038] The stationary wavelet transform module is used to perform a stationary wavelet transform on the output vector to obtain high-frequency detail components, low-frequency components, and mid-frequency components;

[0039] The Granger causality analysis module is used to perform Granger causality analysis based on high-frequency detail components; and to perform inverse Granger transform on the high-frequency detail components after Granger causality analysis to obtain the reconstructed frequency domain sequence data.

[0040] The channel attention mechanism module is used to calculate the correlation between different channel attentions based on low-frequency and mid-frequency components; based on the correlation between different channel attentions, it selects the multiple channels with the strongest dependence; it uses self-attention coupling for channels with strong dependence and geometric self-attention coupling for channels with weak dependence, and fuses the outputs of self-attention and geometric self-attention to obtain a comprehensive feature representation.

[0041] The FFN module is used to perform an inverse stationary wavelet transform on the reconstructed frequency domain sequence data and the comprehensive feature representation to convert it from the frequency domain to the time domain, thereby obtaining the power time series data features. Based on the power time series data features, a household electricity consumption time series prediction model is constructed and trained, and the trained household electricity consumption time series prediction model is used for prediction.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] (1) This invention decouples the household electricity consumption time series into high-frequency noise components and low-frequency trend components through stationary wavelet transform. By separating the high-frequency noise components for specialized processing, the interference of noise on the overall prediction process is effectively avoided, and data features can be captured more accurately, thereby significantly improving prediction accuracy. At the same time, this processing method fully considers the impact of noise on prediction accuracy. By specifically analyzing and processing noise, it provides a cleaner and more reliable data input for the prediction model, further ensuring the accuracy of the prediction results.

[0044] (2) This invention uses Granger causality analysis, which can clearly trace the root causes of noise and its influence path, and accurately quantify the contribution of noise to the prediction results. Furthermore, the results of this analysis are cleverly integrated into the subsequent predictions, so that the prediction model can fully consider the noise factor, effectively make up for the shortcomings of previous prediction methods in dealing with the impact of noise, and greatly improve the comprehensiveness and accuracy of the prediction.

[0045] (3) This invention employs a channel attention mechanism to filter different channels based on the strength of their dependency correlation. Based on the correlation between different channel attention, multiple channels with the strongest dependency are selected. Self-attention coupling is used for channels with strong dependency, and geometric self-attention coupling is used for channels with weak dependency. The synergistic effect of the dual attention mechanism enables the model to understand the intrinsic relationship between data features more comprehensively and deeply, thereby capturing the essential laws of the data more accurately and providing strong support for improving prediction performance. Attached Figure Description

[0046] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the process of the present invention;

[0048] Figure 2This is the fitting effect of the present invention on the ETTh1 dataset;

[0049] Figure 3 The fitting performance of the DLinear device on the ETTh1 dataset;

[0050] Figure 4 The fitting performance of the Autoformer device on the ETTh dataset;

[0051] Figure 5 This is the fitting effect of the present invention on the ETTh2 dataset;

[0052] Figure 6 The fitting performance of the DLinear device on the ETTh2 dataset;

[0053] Figure 7 This shows the fitting performance of the Autoformer device on the ETTh dataset. Detailed Implementation

[0054] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0055] In one embodiment of the present invention, reference is made to... Figure 1 This paper presents a time prediction method based on different channel attention mechanisms, including the following steps:

[0056] The input household electricity consumption time series data is processed in batches. The household electricity consumption time series data of each channel in each batch is normalized and scaled to obtain the scaled household electricity consumption time series data vector.

[0057] The scaled household electricity consumption time series data vector is input into a pre-trained MLP model to obtain the processed output vector; the output vector is then subjected to a stationary wavelet transform to obtain high-frequency detail components, low-frequency components, and mid-frequency components.

[0058] Granger causality analysis is performed based on high-frequency detail components; inverse Granger transform is then performed on the high-frequency detail components after Granger causality analysis to obtain the reconstructed frequency domain sequence data.

[0059] Based on low-frequency and mid-frequency components, the correlation between attention in different channels is calculated; based on the correlation between attention in different channels, the channels with the strongest dependence are selected; self-attention coupling is used for channels with strong dependence, and geometric self-attention coupling is used for channels with weak dependence. The outputs of self-attention and geometric self-attention are fused to obtain a comprehensive feature representation.

[0060] The reconstructed frequency domain sequence data is then subjected to an inverse stationary wavelet transform with the comprehensive feature representation to convert it from the frequency domain to the time domain, thus obtaining the power time series data features. The power time series data features are then passed through an FFN (Feed-Forward Neural Network) to obtain the final prediction result.

[0061] The following is a detailed explanation of each of the above steps:

[0062] Step 100: Process the input household electricity consumption time series data in batches. Normalize and scale the household electricity consumption time series data of each channel in each batch to obtain the scaled household electricity consumption time series data vector, thus eliminating the differences between scales within different channels.

[0063] As an example, this step includes the following sub-steps:

[0064] Step 110: Process the input household electricity consumption time series data in batches, and perform RMS (Root Mean Square Layer Normalization) normalization on the household electricity consumption time series data of each channel in each batch to obtain the RMS value.

[0065] The input household electricity consumption time-series data is first divided into multiple batches, each containing several samples. This batch processing helps optimize memory usage and computational efficiency, especially when dealing with large-scale data.

[0066] For each channel's household electricity consumption time series data vector in each batch, RMSNorm normalization is performed to calculate the RMS value. The core idea of ​​RMSNorm normalization is to first calculate the root mean square of each input household electricity consumption time series data vector. That is, to square each element in the household electricity consumption time series data vector, calculate the average, and then take the square root. The formula is:

[0067] ;

[0068] in, It is the RMS value; It is the input household electricity consumption time series data vector xThe i One component; This refers to the dimension of the input household electricity consumption time series data vector, i.e., the total number of elements. In this way, RMSNorm can unify the scale of different channels or features while maintaining computational efficiency.

[0069] Step 120: Use the RMS value to normalize the household electricity consumption time series data vector to obtain the normalized household electricity consumption time series data vector.

[0070] That is, divide each element by its root mean square value, the formula is:

[0071] ;

[0072] in, x i It is the input household electricity consumption time series data vector x The i A vector, It is the input household electricity consumption time series data vector x After processing, the first i A vector. In this way, each element is divided by its root mean square value, thereby normalizing the input vector.

[0073] Step 130: Introduce a learnable scaling parameter to adjust the output scale of the normalized household electricity consumption time series data vector, and obtain the scaled household electricity consumption time series data vector.

[0074] To ensure the expressive power of the network, RMSnorm typically adds a scientific scaling parameter γ to adjust the scale of the output after normalization. The formula is as follows:

[0075] ;

[0076] in, It is the scaled-up number i A vector of time-series data on household electricity consumption; It is a learnable parameter, usually initialized to 1.

[0077] Through these steps, RMSnorm eliminates scale differences between different channels or features and maintains the network's flexibility through learnable scaling parameters.

[0078] Step 200: Input the scaled household electricity consumption time series data vector into the pre-trained MLP model. The MLP model performs a nonlinear transformation on the scaled household electricity consumption time series data to obtain the processed output vector.

[0079] As an example, this step may include the following sub-steps:

[0080] Step 210: Construct a multilayer perceptron (MLP) model, which includes an input layer, a hidden layer, and an output layer.

[0081] The input layer receives external input data, namely a scaled vector of household electricity consumption time series data, where each input node represents a feature. Assume the dimension of the input scaled household electricity consumption time series data vector is... d The input layer contains d There are 10 nodes, and each node corresponds to a feature of the input data.

[0082] Hidden layers typically consist of one or more intermediate layers used to extract features through nonlinear transformations. Neurons in the hidden layers are activated using the SwiGLU activation function, increasing the model's nonlinear representation capability; given a scaled vector of household electricity consumption time-series data... ∈R d The formula for the SwiGLU activation function is:

[0083] ;

[0084] in, This represents the SwiGLU activation function. The introduction of the SwiGLU activation function enhances the non-linear representation capability of the model and helps to capture complex patterns in the data. It is the weight matrix of the first linear transformation; It is the weight matrix of the second linear transformation;

[0085] The Swish function is defined as:

[0086] ;

[0087] in, Represents the Sigmoid function;

[0088] Right now:

[0089] ;

[0090] The output layer outputs the final result based on the task type. In this task, the task type is regression.

[0091] Step 220: Train the MLP model using the training data to obtain the trained MLP model.

[0092] Training data typically consists of two parts: input data and corresponding label data. The input data is the scaled vector of household electricity consumption time series data. The label data is the true value corresponding to the input data; in the household electricity consumption time series data, the label data is the output vector after nonlinear transformation.

[0093] Label data refers to the data used during model training. The parameters of an MLP model learn from and continuously adjust their values ​​based on the labeled data. In other words, labeled data serves as a reference to provide correct output during model training; the model learns the mapping relationship between input and output by comparing itself with the labeled data.

[0094] During training, the optimization objective of the model is to minimize the prediction error, typically using mean squared error (MSE) as the loss function. By continuously adjusting the model parameters, the error between the model's predictions on the training data and the actual values ​​is gradually reduced, ultimately resulting in a well-trained MLP model.

[0095] Step 230: Input the scaled household electricity consumption time series data into the trained MLP model. The MLP model performs feature extraction and nonlinear transformation on the input scaled household electricity consumption time series data, and finally outputs the prediction result of the regression task, that is, the processed output vector.

[0096] Step 300: Perform a stationary wavelet transform on the output vector to obtain high-frequency detail components, low-frequency components, and mid-frequency components; wherein, the high-frequency detail components correspond to the detailed parts of the signal and contain high-frequency information; the low-frequency components correspond to the approximate parts of the signal and contain trend or low-frequency information; the mid-frequency components are the part between the high and low frequencies and are extracted through intermediate layer detail coefficients.

[0097] The stationary wavelet transform (SWT) decomposes the data output by the MLP using a fixed-length filter without downsampling.

[0098] As an example, this step may include the following sub-steps:

[0099] Step 310: Initial signal:

[0100] Assume the MLP outputs a time series signal. , where n = 0, 1, ..., N-1, and N is the length of the signal. Initial approximation part .

[0101] Step 320: Generate low-pass and high-pass filters.

[0102] SWT uses fixed-length low-pass and high-pass filters to decompose the data output by the MLP.

[0103] The low-pass and high-pass filters are generated from the mother wavelet function and expanded through zero-placing. Specifically, the... j The layer filter is obtained by performing a 2-step process on the original filter. j-1 Obtained by double insertion of zero.

[0104] Step 330: Based on the low-pass filter and the high-pass filter, calculate the low-frequency approximate component and the high-frequency detail component of each layer.

[0105] No. j Low-frequency approximate components of the layer and high-frequency detail components Recursively defined by the following formula:

[0106] ;

[0107] ;

[0108] in, It is the first j The low-pass filter used in the layer It is the first j The high-pass filter used in the layer; yes j -1 layer low-frequency components.

[0109] Step 304: The low-frequency approximation component of each layer will continue to be decomposed into the approximation component and detail component of the next layer. The above process is repeated until the preset number of decomposition layers is reached, and finally the high-frequency detail components (high-frequency noise) of each scale and the relative low-frequency approximation components of the last layer are obtained. The relative low-frequency approximation components include mid-frequency components and low-frequency components.

[0110] In SWT, the approximate components of each layer are further decomposed into approximations and details of the next layer, i.e.:

[0111] ;

[0112] In the above formula, D1, D2, ..., D J Represents high-frequency detail components at various scales; A represents the initial approximation; J J represents the relatively low-frequency approximation component of the last layer; J represents the preset number of decomposition layers.

[0113] Step 400: Perform Granger causality analysis based on high-frequency detail components; perform inverse Granger transform on the high-frequency detail components after Granger causality analysis to obtain the reconstructed frequency domain sequence data.

[0114] Step 410: Perform Granger causality analysis based on high-frequency detail components.

[0115] Granger causality analysis is a method for testing causal relationships between time series data. Its basic idea is that if past values ​​of one time series can help predict future values ​​of another, then the former can be said to be Granger causally related to the latter.

[0116] Step 411: Select an appropriate lag period.

[0117] When performing Granger causality analysis on high-frequency detail components (noise) at various scales, it is important to select an appropriate lag period. The lag period refers to the influence of a time series on the current period from several past periods.

[0118] Based on information criteria such as the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), the AIC or BIC values ​​under different lag periods are calculated. The lag period that minimizes the information criterion is selected as the optimal value, and the residual white noise test is used for verification.

[0119] Let the time series vector be X t ∈R k Granger analysis is based on a vector autoregressive (VAR) model, and the formula is:

[0120] ;

[0121] in, p It is the lag order. k It is the lag coefficient matrix. This is the error term.

[0122] Step 412: In the vector autoregression model, test whether the lag period of X will significantly improve the prediction accuracy of Y; by performing an F test (or Wald test), where X represents the high-frequency detail component (high-frequency noise), the variable being tested for whether it has a Granger causal effect on Y (explanatory variable), and Y represents the target variable being predicted (explanatory variable).

[0123] Restricted model (containing only the lag period of Y):

[0124] ;

[0125] Unconstrained model (using lag times for both X and Y):

[0126] ;

[0127] in, It is the predicted value at the current moment. It is the past number j The value of the explained variable during the period. It is the past number jThe values ​​of explanatory variables in the period (lagged variables). It is the coefficient of the lagged term of Y. It is the lagged coefficient of X (whether it is significant is the key point of the test). It is the model residual term.

[0128] Based on the F-test (or Wald test), if the lag coefficient of X is significant, then X can be considered Granger causally related to Y, meaning that past values ​​of X have predictive power for the current value of Y. If the lag coefficient of X is not significant, then X is not considered Granger causally related to Y, meaning that past values ​​of X do not have significant predictive power for the current value of Y, and a nonlinear Granger causality test is required.

[0129] Step 420: Perform an inverse Granger transform on the high-frequency detail components after causal analysis to obtain the reconstructed frequency domain sequence data.

[0130] Performing an inverse Granger transform on the high-frequency detail components (high-frequency noise) after causal analysis is not only to restore the original high-frequency noise block, but also to preserve the influence of high-frequency noise and remove the weak influence of other factors.

[0131] From high-frequency detail components D1, D2, ..., D J In this process, X components that exhibit a significant Granger causal relationship with the target variable Y are selected. For example, if the first... k High-frequency details of the layer D k If a significant causal relationship exists for Y, then retain D. k If the high-frequency details in other layers are not significant, they are considered as interference items and removed.

[0132] The inverse stationary wavelet transform (ISWT) is used to recombine the filtered high-frequency detail components and the relatively low-frequency approximation components of the last layer into the reconstructed frequency domain sequence data.

[0133] Step 500: Calculate the correlation between attention in different channels based on low-frequency and mid-frequency components; based on the correlation between attention in different channels, select the channels with the strongest dependence; use self-attention coupling for channels with strong dependence and geometric self-attention coupling for channels with weak dependence, and fuse the outputs of self-attention and geometric self-attention to obtain a comprehensive feature representation.

[0134] As an example, step 500 may include the following sub-steps:

[0135] Step 510: Calculate the correlation between attention in different channels based on low-frequency and mid-frequency components.

[0136] The signal in each channel can be independently decomposed into low-frequency and mid-frequency components, and the low-frequency or mid-frequency components of different channels may be correlated. The correlation between the low-frequency or mid-frequency components of different channels is calculated to measure the channel dependency.

[0137] The Pearson correlation coefficient is a statistical indicator used to measure the strength and direction of the linear relationship between two variables. Its value ranges from -1 to 1, and it helps to understand the linear correlation between variables.

[0138] Pearson coefficient r It can be used to filter the correlation between different variables:

[0139] ;

[0140] in, It is the covariance of U and V; It is the standard deviation of U. It is the standard deviation of V;

[0141] U represents the low-frequency and mid-frequency components in one channel, and V represents the low-frequency and mid-frequency components in another channel.

[0142] Covariance measures the common variation of two variables, while standard deviation measures the volatility of each variable.

[0143] Step 511: Calculate the mean of U and V:

[0144] ;

[0145] In the above formula, This represents the mean of U; This represents the mean of V. This indicates the low-frequency and mid-frequency components in the channel; This represents the low-frequency and mid-frequency components in another channel; n represents the dimension of the household electricity consumption time series data vector, i.e., the total number of elements.

[0146] Step 512: Calculate the covariance :

[0147] ;

[0148] Step 513: Calculate the standard deviation:

[0149] ;

[0150] In the above formula, The standard deviation of U; This represents the standard deviation of V.

[0151] Step 514: Calculate the Pearson correlation coefficient by multiplying the covariance by the two standard deviations:

[0152] .

[0153] Step 520: Based on the correlation between attention in different channels, filter out the channels with the strongest dependence.

[0154] The attention mechanism works by calculating the weights between channels; for each channel, it calculates the attention weights between it and other channels.

[0155] Sort the channels according to their attention weights and select the one with the highest weight. n This channel is the most dependent channel.

[0156] Step 530: Apply self-attention coupling to channels with strong dependencies and geometric self-attention coupling to channels with weak dependencies. Fuse the outputs of self-attention and geometric self-attention to obtain a comprehensive feature representation.

[0157] As an example, step 530 may include the following sub-steps:

[0158] Step 531: Use self-attention coupling for channels with strong dependencies.

[0159] The highly dependent input sequence after step 520 , each of ∈Rd( ) is a vector.

[0160] The steps to calculate the self-attention mechanism are as follows:

[0161] Query, Key, Value Matrix:

[0162] ;

[0163] Among them, W1 Q W1 K W1 V It is the weight matrix for learning.

[0164] Calculate attention weights (Scaled Dot-Product Attention):

[0165] ;

[0166] Where, d k This is the dimension of the key vector. The attention matrix A is a matrix of shape […]. The matrix represents the weight of each position in the sequence.

[0167] Weighted summation (weighted matrix):

[0168] ;

[0169] The final output is obtained by weighted summation of the value matrix V1.

[0170] Step 532: Apply geometric self-attention coupling to channels with weak dependencies.

[0171] Given a weakly dependent input sequence after step 520 , each of ( )∈Rd is a vector.

[0172] The formula for the computational geometry self-attention mechanism is as follows:

[0173] Query, key, value: ;

[0174] Among them, W2 Q W2 K W2 V It is the weight matrix for learning.

[0175] Calculating geometric distance: Considering spatial distance or adjacency relationships:

[0176] ;

[0177] in, Represents a node i and nodes j The geometric distance between them.

[0178] Calculate attention weights with geometric relationships:

[0179] ;

[0180] in, It is a hyperparameter that adjusts the geometric influence, d k It is the geometric distance between nodes; d k (Here, the dimension of the weakly dependent channel is the same as the dimension of the strongly dependent feature) is the dimension of the key vector.

[0181] Weighted summation:

[0182] ;

[0183] The final output is obtained by weighted summation of the value matrix V2.

[0184] Step 533: Fuse the outputs of self-attention and geometric self-attention to obtain a comprehensive feature representation.

[0185] Step 600: Perform an inverse stationary wavelet transform on the reconstructed frequency domain sequence data and the comprehensive feature representation to convert it from the frequency domain to the time domain, thereby obtaining the power time series data features; based on the power time series data features, construct and train a household electricity consumption time series prediction model, and use the trained household electricity consumption time series prediction model for prediction.

[0186] As an example, this step may include the following sub-steps:

[0187] Step 610: Perform an inverse stationary wavelet transform on the reconstructed frequency domain sequence data and the comprehensive feature representation to convert it from the frequency domain to the time domain, thereby obtaining the power time series data features.

[0188] The inverse stationary wavelet transform achieves restoration by reconstructing the approximate and detail coefficients at each scale, as shown in the following formula:

[0189] ;

[0190] in, Indicates the characteristics of power time series data; It is a scale Approximation coefficients, It is a scale Detail factor; scale This refers to the decomposition depth of the wavelet transform, that is, the decomposition into several approximation coefficients and detail coefficients; It is a low-pass filter in the inverse transform process. It is a high-pass filter in the inverse transform process.

[0191] Step 620: Based on the characteristics of electricity time series data, construct and train a household electricity consumption time series prediction model, and then use the trained household electricity consumption time series prediction model.

[0192] As an example, step 620 may include the following sub-steps:

[0193] The household electricity consumption time series data prediction model adopts FFN, which includes an input layer, a hidden layer, and an output layer. The input layer receives the electricity time series data features, which are the numerical values ​​of the features. These input features are passed to each neuron in the hidden layer, and each neuron is connected to all neurons in the previous layer, with each connection having a weight. The neurons receive the outputs from the previous layer and perform a weighted sum on these outputs. The weighted sum is then transformed through an activation function to obtain the final prediction output. This FFN model does not require training; it only needs to be projected onto the original dimensions.

[0194] ;

[0195] in, Let W represent the characteristics of power time series data, W represent the weight matrix, and b represent the bias vector. y represents the activation function, and in this embodiment, the ReLU activation function is used.

[0196] The training data for the household electricity consumption time series data prediction model consists of two parts: input data and corresponding label data. The input data is the characteristics of the electricity time series data; the label data is the true value corresponding to the input data.

[0197] In household electricity consumption time series data, the label data is the output vector after nonlinear transformation.

[0198] Labeled data refers to the data used during model training. The parameters in a model learn from and continuously adjust their values ​​based on the labeled data. In other words, labeled data serves as a reference point for providing correct output during model training; the model learns the mapping relationship between input and output by comparing itself to the labeled data.

[0199] During training, the training data is divided into a training set and a validation set. The training set is used to learn the model parameters, while the validation set is used to evaluate the model's performance on unseen data. By observing the model's prediction error on the validation set, it is possible to determine whether the model is overfitting or underfitting, thus allowing for timely adjustments to the model's structure and training strategy.

[0200] Training a prediction model for household electricity consumption time series data involves iterative parameter optimization. Before training begins, the weight matrix and bias vector of the prediction model need to be initialized. Random initialization is typically used, assigning a small random value to each parameter to break the model's symmetry and ensure effective learning during training. The electricity time series data features from the training set are input into the MLP model, passing through the input layer, hidden layer, and output layer sequentially. In each layer, neurons perform linear transformations and nonlinear activation functions on the input electricity time series data. The hidden layer uses the ReLU activation function to increase the model's nonlinear representation capability, enabling it to learn complex patterns in the data. After multiple layers of processing, the model outputs predicted values. The predicted values ​​of the household electricity consumption time series data prediction model are compared with the corresponding labeled data (true values), and the prediction error is calculated. Based on the calculated loss value, the error is propagated back from the output layer to the input layer layer by layer using a backpropagation algorithm. During propagation, the gradient of each layer's parameters, i.e., the partial derivative of the loss function with respect to the parameters, is calculated. The gradient reflects the direction and magnitude of parameter adjustment. By adjusting the parameters in the opposite direction of the gradient, the loss value can be gradually reduced.

[0201] Based on the gradients calculated through backpropagation, an optimization algorithm (such as stochastic gradient descent) is used to update the weight matrix and bias vector of the household electricity consumption time series data model. The optimization algorithm controls the step size of parameter updates by adjusting parameters such as the learning rate, ensuring that the model can converge stably and quickly to the optimal solution.

[0202] Step 630: Input the household electricity consumption time series data to be predicted into the trained household electricity consumption time series prediction model to obtain the prediction result.

[0203] Based on the same inventive concept, this application also provides a time prediction transpose based on different channel attention mechanisms for implementing the time prediction method based on different channel attention mechanisms described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more device embodiments provided below can be found in the above-described limitations of the time prediction method based on different channel attention mechanisms, and will not be repeated here.

[0204] This invention provides a time prediction device based on different channel attention mechanisms, the device comprising:

[0205] The preprocessing module is used to process the input household electricity consumption time series data in batches, and to normalize and scale the household electricity consumption time series data of each channel in each batch to obtain the scaled household electricity consumption time series data vector.

[0206] The MLP (Multilayer Perceptron) module is used to input the scaled household electricity consumption time series data vector into the pre-trained MLP model to obtain the processed output vector;

[0207] The stationary wavelet transform module is used to perform a stationary wavelet transform on the output vector to obtain high-frequency detail components, low-frequency components, and mid-frequency components;

[0208] The Granger causality analysis module is used to perform Granger causality analysis based on high-frequency detail components; and to perform inverse Granger transform on the high-frequency detail components after Granger causality analysis to obtain the reconstructed frequency domain sequence data.

[0209] The channel attention mechanism module is used to calculate the correlation between different channel attentions based on low-frequency and mid-frequency components; based on the correlation between different channel attentions, it selects the multiple channels with the strongest dependence; it uses self-attention coupling for channels with strong dependence and geometric self-attention coupling for channels with weak dependence, and fuses the outputs of self-attention and geometric self-attention to obtain a comprehensive feature representation.

[0210] The FFN (Feedforward Neural Network) module is used to perform inverse stationary wavelet transform on the reconstructed frequency domain sequence data and the comprehensive feature representation to convert it from the frequency domain to the time domain, thereby obtaining the power time series data features. Based on the power time series data features, a household electricity consumption time series prediction model is constructed and trained, and the trained household electricity consumption time series prediction model is used for prediction.

[0211] ETTm2 belongs to the ETT (Electricity Transformer Temperature) dataset series. This series of data revolves around data related to the temperature of power transformers, providing researchers with real-world data for developing and testing time series forecasting algorithms. ETTm1, ETTh1, ETTh2, and Electricity are also commonly used datasets in the field of time series forecasting.

[0212] Table 1 Comparison of the method of the present invention with other model indicators

[0213]

[0214] Tables 1 and 2 compare the prediction performance of the method of this invention with mainstream models such as TimeMixer, iTransformer, and CrossGNN on power-related time series datasets using two core indicators: MSE (mean squared error) and MAE (mean absolute error). This verifies the advantages of the method of this invention in power time series prediction tasks.

[0215] Table 2 Comparison of the method of the present invention with other model indicators

[0216]

[0217] Reference Figures 2-4 , Figure 2 The fitting effect of this invention on the ETTh1 dataset is demonstrated. Compared with the fitting plots of other technical solutions, the technical solution of this invention has higher fitting accuracy at time step 96 and no obvious shift is observed. Figure 3 A simple linear structure (DLinear) is employed, which performs a linear transformation on the input time series data to learn its main trend changes. While this method has fewer parameters and lower computational cost, its performance in inflection point identification and overall trend capture is inferior to the approach described in this invention. In comparison, Figure 4 A dual decomposition module (Autoformer) is introduced to decompose the time series into a trend component and a seasonal component, which are processed by separate modules. The trend module better characterizes long-term changes, while the seasonal module helps capture short-term fluctuations. Figure 3 Significant improvements have been made. However, Figure 3 and Figure 4 The modeling methods are all limited to mining time-domain features and fail to consider periodic components in the frequency domain, which weakens their ability to model periodic patterns to some extent. Furthermore, compared to... Figure 2 and Figure 3 , Figure 4 The presence of significant parameter redundancy negatively impacts the overall efficiency of the model. This indicates that, compared to other technical solutions, the present invention has a significant advantage in capturing inflection points and overall trends.

[0218] Reference Figures 5-7 , Figure 5 The fitting effect of the technical solution of this invention on the ETTh2 dataset is demonstrated. Figure 7 Compared to using a dual-decomposition module to predict seasonal and trend changes separately, this technique achieves slightly lower fitting accuracy at time step 96, but is still superior. Figure 6 Prediction is made using a simple linear transformation. Figure 6 Overfitting exists, causing the fitting accuracy to gradually decrease at larger time steps. Although this technical solution's fitting effect at time step 96 is not as good as... Figure 7 However, in long-term forecasting, the periodicity of time series will become fully apparent (the periodicity of time series means that time series will have similar seasonal changes and trend changes at every time step), and its predictive advantages will become more and more obvious.

[0219] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A time prediction method based on different channel attention mechanisms, characterized in that, Includes the following steps: The input household electricity consumption time series data is processed in batches. The household electricity consumption time series data of each channel in each batch is normalized and scaled to obtain the scaled household electricity consumption time series data vector. The scaled household electricity consumption time series data vector is input into the pre-trained MLP model to obtain the processed output vector; The output vector is subjected to a stationary wavelet transform to obtain high-frequency detail components, low-frequency components, and mid-frequency components. This includes: calculating the low-frequency approximate components and high-frequency detail components of each layer based on low-pass and high-pass filters; the low-frequency approximate components of each layer are further decomposed into approximate components and detail components of the next layer, and the above process is repeated until a preset number of decomposition layers is reached, finally obtaining the high-frequency detail components at each scale and the relative low-frequency approximate components of the last layer, wherein the relative low-frequency approximate components include low-frequency components and mid-frequency components. Granger causality analysis is performed based on high-frequency detail components; inverse Granger transform is performed on the high-frequency detail components after Granger causality analysis to obtain reconstructed frequency domain sequence data, including: selecting high-frequency detail components that have significant Granger causal relationship with the Granger transform target variable from the high-frequency detail components; and recombining the selected high-frequency detail components and the relatively low-frequency approximation components of the last layer into reconstructed frequency domain sequence data. Based on low-frequency and mid-frequency components, Pearson correlation coefficients are calculated between different channels, i.e., the Pearson correlation coefficients between the low-frequency and mid-frequency components in one channel and the low-frequency and mid-frequency components in another channel are calculated to characterize the correlation between attention in different channels. Based on the correlation between attention in different channels, multiple channels with the strongest dependence are selected. Self-attention coupling is used for channels with strong dependence, and geometric self-attention coupling is used for channels with weak dependence. The outputs of self-attention and geometric self-attention are fused to obtain a comprehensive feature representation. The reconstructed frequency domain sequence data is then subjected to an inverse stationary wavelet transform with the comprehensive feature representation to convert it from the frequency domain to the time domain, thus obtaining the power time series data features. Based on the power time series data features, a household electricity consumption time series prediction model is constructed and trained, and the trained household electricity consumption time series prediction model is used for prediction.

2. The time prediction method based on different channel attention mechanisms according to claim 1, characterized in that, For each channel of each batch of household electricity consumption time series data, normalization and scaling are performed to obtain a scaled household electricity consumption time series data vector, including: RMS values ​​are obtained by performing RMS normalization on the time series data of household electricity consumption for each channel in each batch. The RMS value is used to normalize the input household electricity consumption time series data to obtain normalized household electricity consumption time series data. By introducing a learnable scaling parameter, the output scale of the normalized household electricity consumption time series data is adjusted to obtain a scaled household electricity consumption time series data vector.

3. The time prediction method based on different channel attention mechanisms according to claim 1, characterized in that, The scaled household electricity consumption time series data vector is input into a pre-trained MLP model to obtain the processed output vector, which includes: Construct a multilayer perceptron (MLP) model, which includes an input layer, a hidden layer, and an output layer; The MLP model is trained using training data to obtain a trained MLP model; The scaled household electricity consumption time series data is input into the trained MLP model. The MLP model performs feature extraction and nonlinear transformation on the input scaled household electricity consumption time series data, and finally outputs the prediction result of the regression task, that is, the processed output vector.

4. The time prediction method based on different channel attention mechanisms according to claim 1, characterized in that, Self-attention coupling is used for channels with strong dependencies, and geometric self-attention coupling is used for channels with weak dependencies. The outputs of self-attention and geometric self-attention are fused to obtain a comprehensive feature representation. For channels with strong dependencies, self-attention coupling is used. Self-attention weights are calculated using the query matrix and key matrix. Based on the self-attention weights and value matrix, a feature representation that strengthens global dependencies is obtained. For channels with weak dependencies, geometric self-attention coupling is used. Attention weights with geometric relationships are calculated through query matrix, key matrix and value matrix. Based on the attention weights with geometric relationships and value matrix, features that fuse local geometric relationships are obtained. By combining feature representations that enhance global dependencies with feature fusion that integrates local geometric relationships, a comprehensive feature representation is obtained.

5. The time prediction method based on different channel attention mechanisms according to claim 1, characterized in that, The reconstructed frequency domain sequence data, along with the comprehensive feature representation, is then subjected to an inverse stationary wavelet transform to convert it from the frequency domain to the time domain, yielding the power time series data features, including: ； in, Indicates the characteristics of power time series data; It is a scale Approximation coefficients, It is a scale Detail factor; scale This refers to the decomposition depth of the wavelet transform, that is, the decomposition into several approximation coefficients and detail coefficients; It is a low-pass filter in the inverse transform process. It is a high-pass filter in the inverse transform process.

6. The time prediction method based on different channel attention mechanisms according to claim 1, characterized in that, The household electricity consumption time series data prediction model adopts FFN. When training the household electricity consumption time series data prediction model, the training data includes input data and corresponding label data. The input data is the electricity time series data feature, and the label data is the true value corresponding to the input data.

7. A time prediction device based on different channel attention mechanisms, characterized in that, The device includes: The preprocessing module is used to process the input household electricity consumption time series data in batches, and to normalize and scale the household electricity consumption time series data of each channel in each batch to obtain the scaled household electricity consumption time series data vector. The MLP module is used to input the scaled household electricity consumption time series data vector into the pre-trained MLP model to obtain the processed output vector; The stationary wavelet transform module is used to perform a stationary wavelet transform on the output vector to obtain high-frequency detail components, low-frequency components, and mid-frequency components; The Granger causality analysis module is used to perform Granger causality analysis based on high-frequency detail components. It performs an inverse Granger transform on the high-frequency detail components after Granger causality analysis to obtain reconstructed frequency domain sequence data. This includes: selecting high-frequency detail components that have a significant Granger causal relationship with the Granger transform target variable from the high-frequency detail components; and recombining the selected high-frequency detail components and the relative low-frequency approximation components of the last layer into reconstructed frequency domain sequence data. This includes: calculating the low-frequency approximation components and high-frequency detail components of each layer based on low-pass and high-pass filters; further decomposing the low-frequency approximation components of each layer into approximation components and detail components of the next layer, repeating the above process until a preset number of decomposition layers is reached, ultimately obtaining the high-frequency detail components at each scale and the relative low-frequency approximation components of the last layer, where the relative low-frequency approximation components include low-frequency components and mid-frequency components. The channel attention mechanism module is used to calculate the Pearson correlation coefficient between different channels based on low-frequency and mid-frequency components. Specifically, it calculates the Pearson correlation coefficient between the low-frequency and mid-frequency components in one channel and the low-frequency and mid-frequency components in another channel to characterize the correlation between different channel attention. Based on the correlation between different channel attention, it selects the multiple channels with the strongest dependence. Self-attention coupling is used for channels with strong dependence, and geometric self-attention coupling is used for channels with weak dependence. The outputs of self-attention and geometric self-attention are fused to obtain a comprehensive feature representation. The FFN module is used to perform an inverse stationary wavelet transform on the reconstructed frequency domain sequence data and the comprehensive feature representation to convert it from the frequency domain to the time domain, thereby obtaining the power time series data features. Based on the power time series data features, a household electricity consumption time series prediction model is constructed and trained, and the trained household electricity consumption time series prediction model is used for prediction.

Images