A ground current prediction method based on spatiotemporal attention mechanism

Abstract

The present invention discloses a grounding current prediction method based on a spatiotemporal attention mechanism, which belongs to the technical fields of time series prediction, deep learning and substation fault prediction. In order to achieve high-precision prediction of grounding current at the docking point, a grounding current dataset is constructed, and training and testing are performed on the grounding current dataset; an SPTFormer deep learning model including an optimal frequency domain decomposition module, an Encoder module, and a Decoder module is constructed; training and testing are performed based on the constructed model to predict the grounding current at different grounding points; and finally, the grounding current prediction results are analyzed. Experimental results show that the spatiotemporal attention mechanism can effectively capture the spatiotemporal dependency of grounding current data and significantly improve the prediction accuracy of multivariate data of grounding current. The method proposed in the present invention has a good auxiliary effect on the precise prevention of substation equipment failures.

Description

Technical Field

[0001] The present invention belongs to the technical fields of time series prediction, deep learning and substation fault prediction, and specifically relates to a ground current prediction method based on a spatiotemporal attention mechanism. Background Art

[0002] The ground current at a substation's grounding points is a key indicator for assessing power quality and is directly related to the operating status of substation equipment. Real-time monitoring of ground current provides timely insight into the health and dynamics of equipment. Understanding the dynamics of ground current is crucial for ensuring the safe and stable operation of substations and effectively preventing equipment failures. The core of the ground current prediction task lies in the in-depth analysis and processing of historical ground current data collected by sensors installed at different grounding points, exploring the underlying patterns and trends within this data. This allows for predictions of future ground current data, providing strong data support for safety and prevention of substation equipment.

[0003] This task can be seen as an application of time series prediction in the power field. For this task, the literature (Li Bingxiao, Zhang Shiwei, Huang Feihu. Power load forecasting based on LSTM and self-attention mechanism [J]. China Testing, 2022, 48(S2): 38-43.) achieved accurate prediction of power load data by integrating LSTM and attention mechanism. The literature (Zhang Jianliang, Li Yang, Zhu Chunshan, et al. Substation equipment fault warning method based on dual-domain sparse Transformer [J]. Computer Science, 2024, 51(05): 62-69.) obtained the time series data generated by the operation of substation electrical equipment, constructed a dual-domain sparse Transformer model, learned the potential information of historical data, and predicted the future data of the equipment, which significantly improved the prediction accuracy and efficiency. However, the above traditional attention method cannot fit the potential dependency between variables well, and is not suitable for the prediction of data sets such as grounding current where there is dynamic correlation between variables (Mou Longhua, Li Guoxin, Meng Qinghai. Analysis of the influence of interconnection of high and low voltage power grid protective grounding network [J]. Journal of China Coal Society, 2004, (02): 238-242.). Summary of the Invention

[0004] In order to achieve high-precision prediction of the grounding current at the docking location, the present invention provides a grounding current prediction method based on the spatiotemporal attention mechanism based on the Autoformer model.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A ground current prediction method based on a spatiotemporal attention mechanism, the method comprising the following steps:

[0007] Step 1: Construct a ground current dataset and perform training and testing on it.

[0008] Step 2: Build an SPTFormer deep learning model that includes the optimal frequency domain decomposition module, Encoder module, and Decoder module; where:

[0009] Optimal frequency domain decomposition module: obtains high- and low-frequency component data containing different component features to enhance the features of time series data;

[0010] Encoder module: First, it extracts local features of time series data through data slicing and embedding modules. Then, it uses the spatiotemporal attention mechanism to capture the dependencies between the time dimension and variable dimension of the time series data, improving the model's feature extraction capabilities.

[0011] Decoder module: Aggregates the data features obtained by the Encoder module through the spatiotemporal cross-attention mechanism and superimposes them to obtain the predicted ground current values ​​at different grounding points.

[0012] Furthermore, the specific operations of the optimal frequency domain decomposition module in step 2 are as follows:

[0013] S2.1: The input time series X is decomposed into low-frequency component data Z by the optimal frequency domain decomposition module low and high frequency component data Z high ;

[0014] In the optimal frequency domain decomposition module, the input time series X is first subjected to Fourier transform to obtain the frequency domain value F. Then, by selecting different frequency domain demarcation points f, different time domain component data are mapped. The specific calculation process is as follows:

[0015] F=FFT(X)

[0016]

[0017] Among them, FFT stands for fast Fourier transform, F is the frequency domain sequence obtained after Fourier transform; iFFT is the inverse Fourier transform, They represent the low-frequency time domain component and the high-frequency time domain component at the dividing point f, respectively. [:f] means slicing the frequency domain sequence F, starting from the first element and ending at the f-th element of the sequence. [f+1:] means slicing the frequency domain sequence F, starting from the f+1th element and ending at the end of the sequence.

[0018] S2.2: The optimal cutoff point between high and low frequencies is obtained by minimizing the difference in the correlation coefficients between the high and low frequency components and the original data. The formula for minimizing the objective function is defined as:

[0019] O=argmin|L f -H f |

[0020] In the above formula, L f 、H f They represent the correlation coefficient between the low-frequency component and the original data, and the correlation coefficient between the high-frequency component and the original data at the dividing point f respectively; O is the objective function value. When the objective function reaches the minimum value, the correlation coefficient between the low-frequency component and the original data and the correlation coefficient between the high-frequency component and the original data reach the best balance.

[0021] Furthermore, the specific operations of the Encoder module in step 2 are as follows:

[0022] S2.3: High frequency component data Z high and low-frequency component data Z low After the feature enhancement of the optimal frequency domain decomposition module, it enters the data slicing and embedding module in the Encoder module. First, the component data is extended to ensure that the sequence length after segmentation is consistent; then the extended data is sliced ​​to obtain sliced ​​data of the same length; finally, the segmented sequence is value embedded and position embedded, and stacked to obtain the embedded vector e out , the specific calculation process is as follows:

[0023]

[0024] In the above formula, Z d represents the input sequence of the dth dimension, It represents the slicing result of the i-th sequence block on the d-th dimension, unfold represents slicing the sequence, padding represents extending the sequence, L represents the total length of the sequence, L s Represents the segment length, D represents the total number of dimensions of the variable, valueEmbedding represents value embedding, and positionEmbedding represents position embedding;

[0025] S2.4: When the embedded vector e out After multi-head mapping in the time dimension, the query matrix Q is obtained t , key matrix K t , value matrix V t , the specific calculation process is as follows:

[0026]

[0027] In the above formula, d k represents the depth of the model, and n is the number of attention heads;

[0028] S2.5: Then calculate the query matrix Q tInstead of the lagged sequence key matrix K shifted back by τ time points t The autocorrelation coefficient between Next, we obtain the k translation lengths with the largest autocorrelation coefficients and use softmax normalization to obtain the attention weights S between sequences under different translation lengths. j ; Finally, based on the k translation lengths obtained, vector V t Shift backward and multiply it by its attention weight to obtain the time dimension feature Z after time delay aggregation time , the specific calculation process is as follows:

[0029]

[0030] In the above formula, FFT * represents performing FFT on the input vector and then finding its conjugate vector; argTopk represents obtaining k maximum translation lengths; Roll performs the operation of backward translation on the input sequence; j∈[0,k] represents the jth translation length.

[0031] S2.6: When the embedded vector e out After variable dimension multi-head mapping, the query matrix Q corresponding to the input vector is obtained d , key matrix K d Sum matrix V d ; Then the query matrix Q d With the key matrix K d Perform dot product operation to obtain the attention weight score A between variables, and normalize it with the value matrix V after softmax activation function normalization. d Perform product operation to obtain the high-dimensional feature Z of the variable dimension dim , the specific calculation process is as follows:

[0032]

[0033] Where, Indicates taking the square root of the dimension value d, which is used to normalize the vector.

[0034] S2.7: After obtaining the time dimension feature Z of the input vector time and high-dimensional features Z of variable dimensions dim Then, the vector Z dim Perform dimension transformation, and vector Z time Dimensions aligned, then with Z dim Directly splice, then pass through a linear projection layer and softmax function to obtain the weight distribution h1 and h2 corresponding to the two attention modules, and based on this distribution, obtain the output result X of the spatiotemporal attention mechanism att The specific calculation process is as follows:

[0035] G=M·Concat(Z dim ,Z time )+b

[0036] h1,h2=softmax(G)

[0037] X att =Concat(h1×Z dim , h2×Z time )

[0038] Where M and b represent weight matrices, G represents the features obtained by the linear projection layer, and Concat represents the vector concatenation operation.

[0039] Furthermore, the specific operations of the Decoder module in step 2 are as follows:

[0040] S2.8: After the decoder module is initialized, it first passes through the data slicing and embedding module to obtain sequence blocks of different time regions. Then, it passes through the spatiotemporal attention module to extract the temporal dimension features of the data and learn the correlation between the variable dimensions.

[0041] S2.9: The spatiotemporal cross-attention module fuses the temporal features learned by the encoder module and the features learned by the spatiotemporal attention module in the decoder. At the same time, it uses the temporal dimension feature rules and the variable dimension correlation relationship for reasoning. After a fully connected layer, the prediction sequences of different components are obtained and added together to obtain the final prediction result.

[0042] Step 3: Model training and inference: Training and testing the constructed SPTFormer deep learning model to predict the grounding current at different grounding points;

[0043] Step 4: Analysis of ground current prediction results.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] The present invention discloses a ground current prediction method based on a spatiotemporal attention mechanism. First, in the data preprocessing stage, an optimized frequency domain decomposition algorithm is designed and implemented, aiming to effectively separate the high-frequency and low-frequency components in the time series data, thereby enhancing the feature representation of the data. Secondly, in the feature extraction stage, a data slicing and embedding module is introduced. This module is responsible for fine-tuning the component data, including data padding, slicing, and embedding operations. Through these operations, the local features of the data in the time dimension can be effectively extracted, providing richer information for the model. Finally, in order to further enhance the feature extraction capability of the model, an innovative spatiotemporal attention module is proposed. This module can capture the intrinsic features of the sequence between the time dimension and the variable dimension in parallel, thereby achieving accurate prediction of the ground current.

[0046] In contrast, the advantages of the present invention are mainly reflected in the following three aspects:

[0047] (1) High prediction accuracy. The SPTFormer deep learning model proposed in this paper can effectively learn the dependency relationship of the time dimension of time series data and the coupling relationship of the variable dimension, achieving accurate prediction of ground current.

[0048] (2) Robustness. The SPTFormer proposed in this paper designs a new time series decomposition method, which solves the problem of modal aliasing when traditional time series methods process high-frequency data. It is suitable for prediction tasks of any time series data with complex features.

[0049] (3) Intelligent early warning. The ground current prediction method of the present invention can achieve accurate prediction of ground current, thereby assisting substation operation and maintenance personnel in preventing substation equipment failures, with low cost and great benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Flowchart of a ground current prediction method based on spatiotemporal attention mechanism;

[0051] Figure 2 This is a schematic diagram of time series data after outlier processing;

[0052] Figure 3 This is a schematic diagram of the model architecture;

[0053] Figure 4 Schematic diagram of data slicing and embedding module;

[0054] Figure 5 Schematic diagram of the spatiotemporal attention module;

[0055] Figure 6 This is a schematic diagram of the prediction results of grounding point a;

[0056] Figure 7 Schematic diagram of the prediction results of grounding point b. DETAILED DESCRIPTION

[0057] To gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description thereof. However, the present invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a comprehensive understanding of the present disclosure.

[0058] A ground current prediction method based on a spatiotemporal attention mechanism, the method comprising the following steps:

[0059] Step 1: Construct a ground current dataset, and perform training and testing on the ground current dataset; perform training and testing on the ground current dataset collected from a 220kV substation in northern my country. Considering that the ground current collection process is easily affected by sensor anomalies, transmission equipment anomalies, and other issues, and data jumps and anomalies occur, the quartile method is used to detect and remove abnormal data, and the missing values ​​are filled using the regional mean filling method, resulting in 6,000 data items. It covers time series data under the influence of multiple factors, providing data support for subsequent model training. Some data from different grounding points are as follows: Figure 2 As shown;

[0060] Step 2: Build an SPTFormer deep learning model consisting of an optimal frequency domain decomposition module, an encoder module, and a decoder module. Ground current prediction is a multivariate time series prediction problem, requiring consideration of not only the dependencies within the time dimension of the time series but also the dependencies between different variables. Traditional attention mechanisms struggle to effectively capture these dependencies. This paper, based on the Autoformer model, improves its attention mechanism and backbone network, making it well-suited for multivariate prediction tasks.

[0061] Furthermore, the step 2 constructs an SPTFormer deep learning model including an optimal frequency domain decomposition module, an Encoder module, and a Decoder module; wherein:

[0062] Optimal frequency domain decomposition module: obtains high- and low-frequency component data containing different component features to enhance the features of time series data;

[0063] Encoder module: First, it extracts local features of time series data through data slicing and embedding modules. Then, it uses the spatiotemporal attention mechanism to capture the dependencies between the time dimension and variable dimension of the time series data, improving the model's feature extraction capabilities.

[0064] Decoder module: Aggregates the data features obtained by the Encoder module through the spatiotemporal cross attention mechanism and superimposes them to obtain the ground current prediction values ​​at different grounding points;

[0065] The specific operation of the optimal frequency domain decomposition module in step 2 is:

[0066] S2.1: If Figure 3 As shown, the input time series X is decomposed into the optimal frequency domain module to obtain the low-frequency component data Z low and high frequency component data Z high ;

[0067] In the optimal frequency domain decomposition module, the input time series X is first subjected to Fourier transform to obtain the frequency domain value F. Then, by selecting different frequency domain demarcation points f, different time domain component data are mapped. The specific calculation process is as follows:

[0068] F=FFT(X)

[0069]

[0070] Among them, FFT stands for fast Fourier transform, F is the frequency domain sequence obtained after Fourier transform; iFFT is the inverse Fourier transform, They represent the low-frequency time domain component and high-frequency time domain component at the dividing point f, respectively. [:f] means slicing the frequency domain sequence F, starting from the first element and ending at the f-th element of the sequence. [f+1:] means slicing the frequency domain sequence F, starting from the f+1-th element of the sequence and ending at the end of the sequence.

[0071] S2.2: The difference in the high- and low-frequency dividing points f directly affects the degree of similarity between the obtained component data and the original data. The present invention uses the Pearson correlation coefficient to measure the correlation between different component data and the original data. The optimal dividing point for high and low frequencies is obtained by minimizing the difference in the correlation coefficients between the high and low frequency components and the original data. By selecting the optimal dividing point, it can be ensured that the low-frequency component is fully utilized, and at the same time, it can also avoid the situation where the low-frequency component accounts for too small a proportion, resulting in the high-frequency component being too random and difficult to predict. The formula for minimizing the objective function is defined as:

[0072] O=argmin|L f -H f |

[0073] In the above formula, L f 、H f They represent the correlation coefficient between the low-frequency component and the original data, and the correlation coefficient between the high-frequency component and the original data at the dividing point f respectively; O is the objective function value. When the objective function reaches the minimum value, the correlation coefficient between the low-frequency component and the original data and the correlation coefficient between the high-frequency component and the original data reach the best balance.

[0074] The specific operations of the Encoder module in step 2 are as follows:

[0075] S2.3: High frequency component data Z high and low-frequency component data Z low After the best frequency domain decomposition module features are enhanced, the data enters the data slicing and embedding module in the Encoder module. Its specific structure is as follows Figure 4 First, the component data is extended to ensure that the length of the sequence after segmentation is consistent; then the extended data is sliced ​​to obtain sliced ​​data of the same length; finally, the segmented sequence is value embedded and position embedded, and stacked to obtain the embedded vector e out ; S2.3: high frequency component data Z high and low-frequency component data Z low After the feature enhancement of the optimal frequency domain decomposition module, it enters the data slicing and embedding module in the Encoder module. First, the component data is extended to ensure that the sequence length after segmentation is consistent; then the extended data is sliced ​​to obtain sliced ​​data of the same length; finally, the segmented sequence is value embedded and position embedded, and stacked to obtain the embedded vector e out , the specific calculation process is as follows:

[0076]

[0077] In the above formula, Z d represents the input sequence of the dth dimension, It represents the slicing result of the i-th sequence block on the d-th dimension, unfold represents slicing the sequence, padding represents extending the sequence, L represents the total length of the sequence, L s Represents the segment length, D represents the total number of dimensions of the variable, valueEmbedding represents value embedding, and positionEmbedding represents position embedding;

[0078] S2.4: If Figure 5 As shown on the left, when the embedded vector e out After multi-head mapping in the time dimension, the query matrix Q is obtained t , key matrix K t , value matrix V t , the specific calculation process is as follows:

[0079]

[0080] In the above formula, d k represents the depth of the model, and n is the number of attention heads;

[0081] S2.5: Then calculate the query matrix Qt Instead of the lagged sequence key matrix K shifted back by τ time points t The autocorrelation coefficient between Next, we obtain the k translation lengths with the largest autocorrelation coefficients and use softmax normalization to obtain the attention weights S between sequences under different translation lengths. j ; Finally, based on the k translation lengths obtained, vector V t Shift backward and multiply it by its attention weight to obtain the time dimension feature Z after time delay aggregation time , the specific calculation process is as follows:

[0082]

[0083] In the above formula, FFT * represents the operation of performing FFT on the input vector and then finding its conjugate vector; argTopk represents obtaining the k maximum translation lengths; Roll performs the operation of backward translation on the input sequence, and j∈[0,k] represents the jth translation length.

[0084] S2.6: If Figure 5 As shown on the right, the embedded vector e out First, through the variable dimension multi-head mapping, the query matrix Q corresponding to the input vector is obtained d , key matrix K d Sum matrix V d ; Then the query matrix Q d With the key matrix K d Perform dot product operation to obtain the attention weight score A between variables, and normalize it with the value matrix V after softmax activation function normalization. d Perform product operation to obtain the high-dimensional feature Z of the variable dimension dim , the specific calculation process is as follows:

[0085]

[0086] Where, Indicates taking the square root of the dimension value d, which is used to normalize the vector;

[0087] S2.7: After obtaining the time dimension feature Z of the input vector time and high-dimensional features Z of variable dimensions dim Then, the vector Z dim Perform dimension transformation, and vector Z time Dimensions aligned, then with Z dim Directly splice, then pass through a linear projection layer and softmax function to obtain the weight distribution h1 and h2 corresponding to the two attention modules, and based on this distribution, obtain the output result X of the spatiotemporal attention mechanism attThe specific calculation process is as follows:

[0088] G=M·Concat(Z dim ,Z time )+b

[0089] h1,h2=softmax(G)

[0090] X att =Concat(h1×Z dim , h2×Z time )

[0091] Where M and b represent weight matrices, G represents the features obtained by the linear projection layer, and Concat represents the vector concatenation operation.

[0092] The specific operations of the Decoder module in step 2 are as follows:

[0093] S2.8: The decoder input is a random number matrix with the same shape as the prediction target. After the decoder module is initialized, it first passes through the data slicing and embedding module to obtain sequence blocks of different time regions. Then, it passes through the spatiotemporal attention module to extract the temporal dimension features of the data and learn the correlation between the variable dimensions.

[0094] S2.9: The spatiotemporal features learned by the encoder module and the features learned by the spatiotemporal attention in the decoder module are integrated through the spatiotemporal cross attention module. At the same time, the temporal dimension feature patterns and the variable dimension correlation relationship are used for reasoning. After a fully connected layer, the prediction sequences of different components are obtained and added together to obtain the final prediction result.

[0095] Step 3: Model training and inference: Training and testing the constructed SPTFormer deep learning model to predict the grounding current at different grounding points;

[0096] The predicted results of grounding point a and grounding point b are as follows Figure 6 、 7 As shown in the figure, through analysis, it can be seen that the model can better predict the future trend changes of ground current.

[0097] Step 4: Analysis of ground current prediction results.

[0098] As shown in Table 1, the performance of the SPTFormer method proposed in this invention is compared with other mainstream time series prediction methods for ground current prediction tasks of different lengths. It can be seen that the prediction performance of the SPTFormer method proposed in this invention is significantly better than that of the other compared methods. Compared with the FEDformer model, the mean square error (MSE) is reduced by an average of 4.7% and the mean absolute error (MAE) is reduced by an average of 6.2%.

[0099] Table 1. Experimental results of ground current dataset model comparison

[0100]

[0101] As shown in Table 2, the actual gains in model prediction performance achieved by the optimal frequency-domain decomposition module, the data slicing and embedding module, and the spatiotemporal attention module are compared. The experiment used a controlled variable method to verify the gains achieved by each module. The experimental results show that the spatiotemporal attention mechanism has the greatest effect on improving model prediction performance compared to the optimal frequency-domain decomposition module and the data slicing and embedding module. This is because for multivariate prediction tasks, the spatiotemporal attention mechanism can better extract the implicit correlation information between the time dimension and the variable dimension of time series data.

[0102] Table 2 Ablation experiment

[0103]

[0104] Any matters not described in detail in this specification are prior art known to those skilled in the art. Although the above description of the present invention is based on specific embodiments to facilitate understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the specific embodiments. As long as various modifications are within the spirit and scope of the present invention as defined and determined by the appended claims, such modifications will be obvious to those skilled in the art, and all inventions and creations utilizing the concepts of the present invention are protected.

Claims

1. A ground current prediction method based on spatiotemporal attention mechanism, characterized in that: The method comprises the following steps: Step 1: Construct a ground current dataset and perform training and testing on it. Step 2: Build an SPTFormer deep learning model that includes the optimal frequency domain decomposition module, Encoder module, and Decoder module; Step 3: Model training and inference: Training and testing the constructed SPTFormer deep learning model to predict the grounding current at different grounding points; Step 4: Analysis of ground current prediction results; Step 2: Construct an SPTFormer deep learning model including an optimal frequency domain decomposition module, an Encoder module, and a Decoder module, wherein the specific operations of the optimal frequency domain decomposition module are as follows: S2.1: Input time series The low-frequency component data is obtained through the optimal frequency domain decomposition module and high-frequency component data ; In the optimal frequency domain decomposition module, the input time series is first Perform Fourier transform to obtain frequency domain value , and then by selecting different frequency domain demarcation points , mapping to obtain different time domain component data, the specific calculation process is as follows: , , , in, stands for Fast Fourier Transform, is the frequency domain sequence obtained after Fourier transform; is the inverse Fourier transform, 、 Represent the dividing points The low-frequency time domain component and high-frequency time domain component at Represents the frequency domain sequence Slice, starting from the first element to the The end of the element, Represents the frequency domain sequence Slice from +1 element starts and ends at the end of the sequence; S2.2: The optimal cutoff point between high and low frequencies is obtained by minimizing the difference in the correlation coefficients between the high and low frequency components and the original data. The formula for minimizing the objective function is defined as: , in the above formula, 、 Represent the dividing points The correlation coefficient between the low-frequency component and the original data, the correlation coefficient between the high-frequency component and the original data; is the objective function value. When the objective function reaches the minimum value, the correlation coefficient between the low-frequency component and the original data and the correlation coefficient between the high-frequency component and the original data reach the best balance.

2. A ground current prediction method based on spatiotemporal attention mechanism according to claim 1, characterized in that: Step 2 constructs an SPTFormer deep learning model including an optimal frequency domain decomposition module, an Encoder module, and a Decoder module; wherein: Optimal frequency domain decomposition module: obtains high- and low-frequency component data containing different component features to enhance the features of time series data; Encoder module: First, it extracts local features of time series data through data slicing and embedding modules. Then, it uses the spatiotemporal attention mechanism to capture the dependencies between the time dimension and variable dimension of the time series data, improving the model's feature extraction capabilities. Decoder module: Aggregates the data features obtained by the Encoder module through the spatiotemporal cross-attention mechanism and superimposes them to obtain the predicted ground current values ​​at different grounding points.

3. The ground current prediction method based on the spatiotemporal attention mechanism according to claim 2 is characterized in that: Step 2: Construct an SPTFormer deep learning model including an optimal frequency domain decomposition module, an Encoder module, and a Decoder module, wherein the specific operations of the Encoder module are as follows: S2.3: High-frequency component data and low-frequency component data After the feature enhancement of the optimal frequency domain decomposition module, it enters the data slicing and embedding module in the Encoder module. First, the component data is extended to ensure that the sequence length after segmentation is consistent; then the extended data is sliced ​​to obtain sliced ​​data of the same length; finally, the segmented sequence is value embedded and position embedded, and stacked to obtain the embedded vector , the specific calculation process is as follows: , , In the above formula, Indicates the An input sequence of dimensions, Indicates the Dimension The slicing result of sequence blocks, represents slicing of a sequence, Indicates the extension of the sequence. represents the total length of the sequence, Represents the segment length, Indicates the total number of dimensions of the variable, represents value embedding, Represents positional embedding; S2.4: When the embedded vector After multi-head mapping in the time dimension, the query matrix is ​​obtained , key matrix , value matrix , the specific calculation process is as follows: , , , in the above formula, Represents the model depth, is the number of attention heads; S2.5: Then calculate the query matrix Instead of moving backward The lagged sequence key matrix of time points The autocorrelation coefficient between Then, get the maximum autocorrelation coefficient The translation length is Normalize the attention weights between sequences under different translation lengths Finally, based on the acquired The translation length, the vector Shift backward and multiply it by its attention weight to obtain the time dimension feature after time delay aggregation , the specific calculation process is as follows: , , , , in the above formula, Represents the input vector first Then find its conjugate vector; Indicates acquisition Maximum translation length; The operation of shifting the input sequence backward; represents the j-th translation length; S2.6: When the embedded vector After variable dimension multi-head mapping, the query matrix corresponding to the input vector is obtained , key matrix Sum Matrix ; Then the query matrix With the key matrix Perform dot product operations to obtain the attention weight scores between variables ,go through The activation function is normalized and the value matrix Perform product operations to obtain high-dimensional features of variable dimensions , the specific calculation process is as follows: , where Represents the dimension value Take square root to normalize the vector; S2.7: After obtaining the time dimension feature of the input vector and high-dimensional features of variable dimensions After that, the vector Perform dimension transformation, and vector Dimensions aligned, then with Direct splicing, then a linear projection layer and Function, get the weight distribution corresponding to the two attention modules and , based on this distribution, we get the output of the spatiotemporal attention mechanism The specific calculation process is as follows: , , , where 、 represents the weight matrix, represents the features obtained by the linear projection layer, Represents a vector concatenation operation.

4. A ground current prediction method based on spatiotemporal attention mechanism according to claim 3, characterized in that: Step 2: Construct an SPTFormer deep learning model including an optimal frequency domain decomposition module, an Encoder module, and a Decoder module, wherein the specific operations of the Decoder module are as follows: S2.8: After the decoder module is initialized, it first passes through the data slicing and embedding module to obtain sequence blocks of different time regions. Then, it passes through the spatiotemporal attention module to extract the temporal dimension features of the data and learn the correlation between the variable dimensions. S2.9: The spatiotemporal cross-attention module fuses the temporal features learned by the encoder module and the features learned by the spatiotemporal attention module in the decoder. At the same time, it uses the temporal dimension feature rules and the variable dimension correlation relationship for reasoning. After a fully connected layer, the prediction sequences of different components are obtained and added together to obtain the final prediction result.

Images