A multi-head attention dynamic optimization method for power grid topology adaptation

Abstract

The present invention belongs to the field of deep learning technology, and in particular relates to a multi-head attention dynamic optimization method for power grid topology adaptation. The method first collects and pre-processes various data such as power grid node connection relationships and line transmission parameters; then constructs a multi-head attention structure, and performs operations such as input data processing, multi-head partitioning adaptive selection, and attention calculation. The adaptive selection uses a neural network to extract features and calculate the selection score to filter the head, and the attention calculation adjusts the score in combination with contextual information; then the multi-head outputs are spliced and the final output is obtained through a learnable weight matrix transformation. Compared with traditional methods, the present invention can process high-dimensional complex power grid data, accurately capture real-time state changes, improve the accuracy and real-time performance of power grid topology analysis, meet the real-time operation requirements of the power grid, and effectively optimize the power grid operation.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and in particular relates to a multi-head attention dynamic optimization method for power grid topology adaptation. Background Art

[0002] Against the backdrop of rapid power grid development and digital transformation, the scale of power grids continues to expand and their topologies become increasingly complex, placing higher demands on the accuracy and real-time performance of power grid topology analysis and operational optimization. Traditional power grid topology analysis methods, such as rule- and model-based methods, struggle to cope with large-scale, highly dynamic power grid data. They have limitations when processing complex topologies and are unable to quickly and accurately capture real-time changes in the power grid's state. With the development of deep learning technology, attention mechanisms have achieved remarkable results in many fields, but the direct application of traditional attention mechanisms in power grid topology analysis still faces challenges. Power grid data is characterized by high dimensionality, nonlinearity, and strong coupling. Traditional multi-head attention mechanisms have difficulty adaptively selecting effective features and lack sufficient consideration of the dynamic characteristics of power grid operation. This results in inaccurate analysis results that cannot meet the needs of real-time power grid operation. Summary of the Invention

[0003] In response to the technical problems existing in the above-mentioned background technology, the present invention proposes a multi-head attention dynamic optimization method for power grid topology adaptation.

[0004] In order to achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0005] S1. Data collection: collecting various data related to the power grid topology, including connection relationship data of power grid nodes, transmission parameter data of lines, power distribution data of each node, and real-time status data of power grid operation;

[0006] S2, data processing, pre-processing the collected data;

[0007] S3, build a multi-head attention structure, including input data processing, multi-head partitioning adaptive selection, cross-head information interaction, and attention calculation; the input data processing: linear transformation of the pre-processed data, through the learnable weight matrix 、 、 Obtain query vector Q, key vector K, and value vector V respectively; Multi-head partitioning adaptive selection: divide the query vector Q, key vector K, and value vector V into multiple heads, introduce adaptive head selection, and dynamically determine whether each head participates in the subsequent attention calculation; Attention calculation: perform scaled dot product attention calculation on each participating head;

[0008] S4. Splice the outputs of all heads involved in the calculation;

[0009] S5, through the final learnable weight matrix The final output is obtained by linear transformation.

[0010] Preferably, the query vector Q, key vector K and value vector V are divided into multiple heads in the multi-head partitioning adaptive selection in step S3, which can be expressed as: ; ; ;in Represents the feature dimension of each head, h represents the total number of heads, and the query vector Q, key vector K, and value vector V are divided into h heads.

[0011] Preferably, the specific operation of performing adaptive head selection after obtaining the h head weight matrices is as follows: using a convolutional neural network to extract the features of the power grid topology data; for each head, calculating a selection score; setting a selection threshold, if the selection score is greater than the selection threshold, the head participates in the subsequent attention calculation, otherwise the head does not participate in the calculation.

[0012] As an advantage, the selection score calculation adopts the information gain method, and for each head h, the corresponding query vector is calculated. , key vector Sum vector Information gain with input feature X;

[0013] The information gain is obtained by calculating the change in entropy. For the query vector, the information gain is , similarly we can get , , and then get the selection score : ,in is the weight coefficient.

[0014] Preferably, between calculating the selection score and setting the selection threshold, it is also necessary to determine the correlation between the heads to adjust the calculated score. The mutual information is used to measure the correlation between the heads. For each head h, the calculation of its selection score can be adjusted as follows: ,in The function is the mutual information function.

[0015] Preferably, the specific steps of attention calculation in step S3 are as follows: calculating the original attention score; introducing the context information of the power grid operation state, wherein the context information is obtained by a pre-trained model related to the power grid; calculating the adjustment factor of each attention score according to the context information and the characteristics of the power grid topology data; the calculation of the adjustment factor is: , where C is the context information, is the adjustment factor of the obtained attention score, is average pooling; the original attention score is multiplied by the adjustment factor element by element to obtain the adjusted attention score, and then the softmax function is applied to the adjusted attention score to obtain the final attention weight; finally, the final attention weight of each head is multiplied by the value vector Get the output for each head.

[0016] Compared with the existing technology, the advantages and positive effects of the present invention are that, in data processing, multi-source power grid data is collected and carefully pre-processed to ensure data quality. A multi-head attention structure is adopted to obtain multi-vectors through linear transformation, and the heads are adaptively divided and screened to participate in the calculation, thereby improving the calculation pertinence and efficiency. Scaled dot product attention calculation is used to adjust the attention score in combination with the power grid context information to accurately focus on key features. In the output stage, multi-head outputs are spliced and transformed through a learnable weight matrix to enhance the accuracy of the results. Overall, the present invention can adaptively process high-dimensional and complex power grid data, break through the limitations of traditional methods, accurately capture the real-time state changes of the power grid, effectively improve the accuracy and real-time performance of power grid topology analysis, and meet the real-time operation requirements of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 It is a flow chart of the overall structure of the present invention; Figure 2 To construct a sub-flowchart for the multi-head attention structure. DETAILED DESCRIPTION

[0019] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0021] Embodiment: With the continuous expansion of the power grid scale and the increasing complexity of the topological structure, the safety and stability of the power grid operation are facing severe challenges. Existing power grid topology analysis methods have the problem of difficulty in quickly and accurately capturing the real-time state changes of the power grid when processing large-scale, highly dynamic power grid data. At the same time, the direct application of traditional attention mechanisms cannot fully adapt to the high-dimensional, nonlinear and strongly coupled characteristics of power grid data, and it is difficult to adaptively select effective features, resulting in the analysis results being unable to meet the needs of real-time power grid operation. In order to achieve accurate analysis of power grid topology, a multi-head attention dynamic optimization method for power grid topology adaptation is proposed.

[0022] First, data collection is performed to collect various data related to the power grid topology, including connection relationship data of power grid nodes, transmission parameter data of lines, power distribution data of each node, and real-time status data of power grid operation.

[0023] The collected data is then preprocessed using data cleaning techniques to remove noise and outliers. By setting appropriate upper and lower thresholds, data points that significantly deviate from the normal range are identified and removed to prevent these abnormal data from interfering with subsequent analysis. For data with missing values, the machine learning-based KNN algorithm is used to estimate missing values based on the characteristics of similar data points, ensuring data integrity. The data is then normalized using the Z-Score method to bring data of different dimensions to the same scale.

[0024] Next, we build a multi-head attention structure, including input data processing, multi-head partitioning adaptive selection, cross-head information interaction, and attention calculation.

[0025] Input data processing is to perform linear transformation on the preprocessed data through a learnable weight matrix 、 、 The query vector Q, key vector K and value vector V are obtained respectively. Specifically, through matrix multiplication operation 、 、 , where X is the preprocessed input data. After calculation, we get the query vector Q, key vector K and value vector V.

[0026] Multi-head partitioning and adaptive selection: The query vector Q, key vector K, and value vector V are divided into multiple heads. Adaptive head selection is introduced to dynamically determine whether each head participates in the subsequent attention calculation. Specifically, the query vector Q, key vector K, and value vector V are divided into multiple heads in multi-head partitioning and adaptive selection, which can be expressed as: ; ; ;in Represents the feature dimension of each head, h represents the total number of heads, and the query vector Q, key vector K, and value vector V are divided into h heads.

[0027] Then, after obtaining the h head weight matrices as described above, the specific operations for adaptive head selection are as follows: using a neural network to extract the features of the power grid topology data; for each head, calculating a selection score; setting a selection threshold, if the selection score is greater than the selection threshold, then the head participates in the subsequent attention calculation, otherwise the head does not participate in the calculation.

[0028] Specifically, a convolutional neural network (CNN) is used to extract the features of the power grid topology data. The power grid topology data is first organized into a multi-dimensional matrix. The convolution layer of CNN slides convolution on the data through learnable convolution kernels, and convolution kernels of different sizes capture local details and macro features respectively. For example, small convolution kernels focus on the connection characteristics near the node, and large convolution kernels obtain the regional power grid connection pattern. The pooling layer reduces the dimension of the convolution layer output, retains the main features, and reduces the amount of calculation. Finally, the fully connected layer integrates the previously extracted features and outputs the power grid topology data features used for the subsequent calculation of the head selection score, helping to achieve adaptive head selection. Then, for each head h, its corresponding query vector is calculated. , key vector Sum vector The information gain with the input feature X is obtained by calculating the change in entropy. For the query vector, the information gain , similarly we can get , , and then get the selection score : ,in is the weight coefficient. Finally, between calculating the selection score and setting the selection threshold, it is necessary to determine the correlation between the heads to adjust the calculated score. The mutual information is used to measure the correlation between the heads. For each head h, the calculation of its selection score can be adjusted as follows: ,in The function is the mutual information function.

[0029] In addition, between calculating the selection score and setting the selection threshold, it is necessary to judge the correlation between the heads to adjust the calculation score. The mutual information is used to measure the correlation between the heads. For each head h, the calculation of its selection score can be adjusted as follows: ,in The function is the mutual information function. It is used to control the influence of head correlation on the selection score. Thus, higher inter-head correlation has a greater impact on the selection score, which can more reasonably select heads for subsequent attention calculations, improving the accuracy and effectiveness of the model in power grid topology analysis.

[0030] Attention calculation: Perform scaled dot product attention calculation on each head participating in the calculation.

[0031] Specifically, the raw attention score is calculated as follows: first calculate and The transposed dot product of Since the numerical value of the dot product result will affect the performance of the subsequent softmax function, in order to avoid it being too large and causing the gradient to disappear or become unstable, the dot product result is divided by Finally, the raw attention score is obtained.

[0032] This contextual information is obtained using a pre-trained grid-related model. This pre-trained grid-related model can be an LSTM trained based on historical grid data. A large amount of historical grid operation data is collected. This data is organized into a chronological sequence and input into the LSTM model. The model continuously learns the time series features and dependencies within the historical data to capture patterns and trends in grid operation. During training, model parameters are adjusted to minimize the error between predicted and actual values. After training, when new grid data is input, the model uses the learned patterns and combines them with the current input data to output contextual information containing key information such as grid operation trends and potential abnormal risks.

[0033] The adjustment factor of each attention score is calculated based on the context information and the characteristics of the power grid topology data. The adjustment factor is calculated as follows: , where C is the context information, is the adjustment factor of the obtained attention score, is average pooling.

[0034] Finally, the original attention scores are element-wise multiplied by the adjustment factor to obtain the adjusted attention scores. This element-wise multiplication allows the attention scores to fully account for the real-time state and topological characteristics of the power grid. The softmax function is then applied to the adjusted attention scores to obtain the final attention weights. The softmax function converts the scores into probability distributions, ensuring that the sum of all attention weights is 1. This clearly reflects the relative importance of different features within the overall picture. The final attention weights for each head are multiplied by the value vector to obtain the output of each head. Each head focuses on data features from different angles. By multiplying the attention weights by the value vector, the results are transformed into information screening and integration results. This ensures that each head's output contains valuable information related to its focus, providing rich and targeted data for subsequent multi-head stitching.

[0035] Next, all the results of the multi-head attention calculations are spliced and output. Specifically, after completing the attention calculation of each head and obtaining the corresponding output, these outputs are spliced in a specific dimension in sequence, and then the spliced vectors are input into the final learnable weight matrix Perform a linear transformation. The parameters of the learnable weight matrix are continuously optimized and adjusted during model training. The purpose is to further fuse and transform the concatenated feature vectors to better meet the actual needs of power grid topology analysis. This linear transformation operation yields the final output.

[0036] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any person skilled in the art may utilize the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes for application in other fields. However, any simple modification, equivalent change, and modification of the above embodiments made in accordance with the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A multi-head attention dynamic optimization method for power grid topology adaptation, characterized by: The following steps are involved: S1. Data collection: collecting various data related to the power grid topology, including connection relationship data of power grid nodes, transmission parameter data of lines, power distribution data of each node, and real-time status data of power grid operation; S2, data processing, preprocessing the collected data; S3. Construct a multi-head attention structure, including input data processing, multi-head partitioning adaptive selection, cross-head information interaction, and attention calculation; Input data processing: linear transformation of preprocessed data, through the learnable weight matrix W Q 、W K 、W V Get the query vector Q, key vector K and value vector V respectively; Multi-head partitioning and adaptive selection: The query vector Q, key vector K, and value vector V are divided into multiple heads, and adaptive head selection is introduced to dynamically determine whether each head participates in subsequent attention calculations; Attention calculation: perform scaled dot product attention calculation on each head involved in the calculation; S4. Splice the outputs of all heads involved in the calculation; S5, through the final learnable weight matrix W O The final output is obtained by linear transformation; The division of the query vector Q, key vector K and value vector V into multiple heads in the multi-head partitioning adaptive selection in step S3 can be expressed as: Q i =Q[:,(h-1)d:id] i=1,2,...h; V i =V[:,(h-1)d:id] i=1,2,...,h; K i =K[:,(h-1)d:id] i=1,2,...,h; Where d represents the feature dimension of each head, h represents the total number of heads, and the query vector Q, key vector K, and value vector V are divided into h heads; The specific operations for adaptive head selection after obtaining h head weight matrices are as follows: Use convolutional neural networks to extract features from power grid topology data; For each head, a selection score is calculated; Set a selection threshold. If the selection score is greater than the selection threshold, the head will participate in the subsequent attention calculation; otherwise, the head will not participate in the calculation. The selection score calculation adopts the information gain method. For each head h, the corresponding query vector Q is calculated. h , key vector K h Sum value vector V h Information gain with input feature X; The information gain is obtained by calculating the change in entropy. For the query vector, the information gain IG(X,Q h )=H(X)-H(X|Q h ), and similarly we can get IG(X,K h )=H(X)-H(X|K h ), IG(X,V h )=H(X)-H(X|V h ), and then get the selection score Score h :Score h =α1·IG(X,Q h )+α2·IG(X,K h )+α3·IG(X,V h ), where α1, α2, α3 are weight coefficients; Between calculating the selection score and setting the selection threshold, it is also necessary to judge the correlation between the heads to adjust the calculation score. The mutual information is used to measure the correlation between the heads. For each head h, the calculation of its selection score can be adjusted as follows: Score′ h =Score h -(λ1∑ i≠h MI(Q h ,Q i )+λ2∑ i≠h MI(K h ,K i )+λ3∑ i≠h MI(V h ,V i )), where MI() function is the mutual information function.

2. The multi-head attention dynamic optimization method for power grid topology adaptation according to claim 1 is characterized in that: The specific steps of attention calculation in step S3 are as follows: Calculate the raw attention score; Introducing contextual information about the power grid's operating status, where the contextual information is obtained through a pre-trained model related to the power grid; According to the context information and the characteristics of the power grid topology data, the adjustment factor of each attention score is calculated; the adjustment factor is calculated as: β h =σ(W β Concat(C,AvgPool(Q h ), AvgPool(K h ), where C is the context information, β h is the adjustment factor of the obtained attention score, AvgPool is the average pooling; Multiply the original attention score element-by-element by the adjustment factor to obtain the adjusted attention score, and then apply the softmax function to the adjusted attention score to obtain the final attention weight; Finally, the final attention weight of each head is multiplied by the value vector V h Get the output for each head.

Images