Power knowledge graph construction method and system based on multi-head attention, storage medium

Abstract

The application discloses a power knowledge graph construction method and system based on multi-head attention, and a medium, and belongs to the technical field of power system automation. The method comprises the following steps: constructing graph structure data containing power equipment, components, defects and environment entities, and encoding node features; secondly, inputting the graph data into a graph attention module, compressing a key-value vector through low-rank compression to reduce storage overhead, separately performing decoupled rotation position coding on a query vector to inject hierarchical and time position information, and respectively modeling equipment hierarchical relationship and dynamic time sequence relationship through a multi-head attention mechanism to obtain node vector representation; finally, performing hierarchical processing and dynamic updating on the node representation, and constructing and continuously updating the power knowledge graph through knowledge reasoning after fusion. The application solves the problems of low storage efficiency, insufficient spatiotemporal relationship modeling and difficulty in dynamic updating of traditional methods, and significantly improves the graph construction efficiency, accuracy and real-time performance.

Description

Technical Field

[0001] This invention relates to the field of power system automation control technology, specifically a method, system, and storage medium for constructing a power knowledge graph based on multi-head attention. Background Technology

[0002] Graph representation learning, especially Graph Attention Networks (GAT), has been introduced into the construction of power knowledge graphs due to its powerful relation modeling capabilities: vector representations of nodes are obtained from graph structure data containing nodes and node features; the power knowledge graph is then constructed based on these node vector representations. However, directly applying existing graph attention mechanisms to the power domain, which has strict physical topology and strong temporal characteristics, for power knowledge graph construction faces the following irreconcilable technical drawbacks:

[0003] Traditional graph attention mechanisms, when processing relationships between nodes, require caching complete key-value vectors for all nodes, with storage overhead increasing quadratically with the number of nodes. When dealing with a power grid graph containing massive numbers of nodes, memory requirements explode, making large-scale knowledge graph construction impractical in engineering. General attention mechanisms lack the ability to effectively encode and perceive strict hierarchical connections between power equipment (such as "substation → transformer → winding"), failing to guide the model to prioritize and accurately model such crucial physical topologies. This results in chaotic entity hierarchy relationships and high error rates in the knowledge graph. Furthermore, existing methods typically treat time-series data as ordinary features, lacking dedicated mechanisms to capture and utilize temporal location, order, and dependencies within the data. This leads to poor modeling capabilities for dynamic knowledge such as equipment state evolution and fault sequence correlations.

[0004] Traditional process output node vector representations, due to the aforementioned efficiency and accuracy issues, fail to effectively integrate key attributes, precise spatial locations, and deep temporal context information of power equipment. This under-integrated, low-quality intermediate representation directly results in a loosely structured, semantically ambiguous, and weakly reasoning-capable knowledge graph, which cannot provide reliable knowledge support for advanced business applications (such as accurate fault diagnosis and intelligent risk warning). Summary of the Invention

[0005] To address the problem that existing graph attention networks fail to systematically and structurally adapt to the strict hierarchy, strong temporal sequence, and massive scale unique to power knowledge, resulting in their inability to meet the stringent requirements of the power field in terms of efficiency, accuracy, and practicality, this invention provides a method, system, and storage medium for constructing a power knowledge graph based on multi-head attention.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] In a first aspect, this invention proposes a method for obtaining node vector representations during the construction of a power knowledge graph, comprising:

[0008] (a) Perform linear transformation based on node features to generate query vector, key vector, and value vector;

[0009] (b) Perform dimensionality reduction and compression on the key vector and value vector;

[0010] (c) Decouple the rotation position encoding of the query vector separately;

[0011] (d) Based on the compressed key vector and value vector, and the encoded query vector, the attention weights between nodes are calculated through a multi-head attention mechanism to obtain the vector representation of the node; wherein, the multiple attention heads in the multi-head attention mechanism are divided into at least two groups. The first group of attention heads aggregates the features of adjacent nodes through a graph convolutional network to model the hierarchical relationship of power equipment, and the second group of attention heads is used to process the temporal features in the node features to model the dynamic temporal relationship.

[0012] As a preferred embodiment of the present invention, the dimension reduction compression maps the key vector and value vector from a first original dimension to a second latent dimension, wherein the second latent dimension is smaller than the first original dimension; the dimension reduction compression is achieved through a linear transformation matrix.

[0013] The decoupled rotation position encoding is implemented by applying rotation position encoding to the query vector while keeping the key vector unencoded.

[0014] As a preferred embodiment of the present invention, the rotation position encoding is used to inject position information corresponding to the hierarchical relationship into the query vector of the first set of attention heads, and to inject position information corresponding to the dynamic temporal relationship into the query vector of the second set of attention heads.

[0015] Secondly, this invention also provides a method for constructing an electricity knowledge graph based on multi-head attention, comprising:

[0016] Based on graph structure data with nodes and node features, obtain the vector representation of nodes;

[0017] Based on the vector representation of the nodes, the power knowledge graph is constructed;

[0018] The method for representing nodes by vectors is the same as the method used to obtain node vector representations during the construction of the power knowledge graph described above.

[0019] As a preferred embodiment of the present invention, the nodes include: device nodes, component nodes, defect nodes, and environment nodes; the node features include structured data encoding features, unstructured data semantic features, and time-series data encoding features.

[0020] As a preferred embodiment of the present invention, the structured data encoding features are obtained based on the encoding of power equipment attribute data including voltage level and capacity; the unstructured data semantic features are obtained by extracting text data of the power system including equipment documents and fault reports through the BERT model; and the time-series data encoding features are obtained by encoding the time-series sensor data including the self-power SCADA system through LSTM.

[0021] As a preferred technical solution of the present invention, a power knowledge graph is constructed based on the vector representation of nodes, specifically including the following steps: performing layered processing and fusion on the vector representation of the nodes into a conceptual layer, a business layer, and an instance layer, and dynamically updating the node representation of the instance layer with real-time data; and weightedly fusing the outputs of the conceptual layer, the business layer, and the instance layer to obtain the final node representation for knowledge reasoning.

[0022] The processing of the concept layer includes extracting general device knowledge through graph convolutional networks; the processing of the business layer includes generating semantic features of edges through a pre-trained language model; and the processing of the instance layer includes fusing image and text features to update node representations.

[0023] As a preferred embodiment of the present invention, relation prediction and graph completion are performed based on the final node representation through a knowledge-aware attention layer, and attention embedding is calculated based on the knowledge-aware attention layer. : ,in, For attention network functions, and These are the head entities and relation vectors, respectively, weighted by interaction relevance. is the candidate tail entity vector.

[0024] Thirdly, the present invention provides a power knowledge graph construction system based on multi-head attention, used to implement the power knowledge graph construction method based on multi-head attention as described above, the system comprising:

[0025] The data acquisition module is used to acquire graph structure data containing nodes and node features;

[0026] A graph attention processing module is used to receive the graph structure data and process it to obtain vector representations of the nodes. The graph attention processing module includes:

[0027] A vector generation unit is used to generate query vectors, key vectors, and value vectors based on the features of the node;

[0028] A low-rank compression unit is used to reduce the dimensionality of the key vector and value vector;

[0029] A decoupling encoding unit is used to decouple the query vector and encode its rotational position separately.

[0030] The multi-head attention unit is used to perform attention calculations based on the compressed key vector, value vector, and encoded query vector, and divides the attention heads into at least two groups to model hierarchical relationships and dynamic temporal relationships respectively.

[0031] The graph construction module is used to construct the power knowledge graph based on the vector representation of the nodes.

[0032] Fourthly, the present invention also provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the power knowledge graph construction method based on multi-head attention as described in the first and second aspects.

[0033] The present invention has the following advantages:

[0034] This invention fundamentally reduces the core storage overhead and computational complexity of graph attention mechanisms by dimensionality reduction and compression of key and value vectors. It solves the bottleneck of traditional methods being difficult to deploy practically due to storage explosion.

[0035] By decoupling the query vector through rotational position encoding and combining it with a multi-head attention mechanism with division of labor, accurate spatial topology perception and effective capture of dynamic temporal sequence are achieved. The first set of attention heads is specifically responsible for modeling the strict hierarchical relationships between devices. Combined with the spatial hierarchical information injected by rotational position encoding, the accuracy of understanding the physical connection structure of the power grid is significantly improved. The second set of attention heads is specifically responsible for processing temporal features to model dynamic relationships, enabling the model to effectively integrate and understand the temporal evolution of device states, thereby enhancing the dynamic expressive power of the knowledge graph.

[0036] Working collaboratively through the aforementioned methods, a high-quality, highly information-condensed node vector representation is ultimately output. This representation deeply integrates the attributes, spatial location, and temporal evolution information of power grid equipment, providing a solid and directly usable feature foundation for the subsequent efficient and accurate construction of power knowledge graphs (such as relationship inference and fault diagnosis), thus improving the overall quality and usability of knowledge graph construction.

[0037] In summary, this invention simultaneously overcomes the core challenges of low efficiency, inaccurate modeling, and weak dynamism that have long existed in the construction of power knowledge graphs, achieving a systematic improvement from methodology to effectiveness. Attached Figure Description

[0038] Figure 1 This is a flowchart of the power knowledge graph construction method based on multi-head attention provided in Example 1.

[0039] Figure 2 This is a flowchart of the method for obtaining node vector representations during the construction of a power knowledge graph, as provided in Example 1.

[0040] Figure 3 This is a module diagram of the power knowledge graph construction system based on multi-head attention provided in Example 2. Detailed Implementation

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] It should be noted that when a component is said to be "installed on" another component, it can be directly on the other component or it may be in a component that is centered on it. When a component is said to be "set on" another component, it can be directly set on the other component or it may also be in a component that is centered on it. When a component is said to be "fixed to" another component, it can be directly fixed to the other component or it may also be in a component that is centered on it.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0044] Example 1

[0045] This invention provides a method for constructing a power knowledge graph based on multi-head attention, aiming to build a knowledge graph for power systems that can understand the physical connections of devices, perceive dynamic changes in state, and perform efficient computation and real-time updates. Please refer to... Figure 1 The method includes the following steps:

[0046] S1. Graph Structure Data Construction; This step aims to transform multi-source heterogeneous power data into structured input data that can be processed by the graph attention module. Specifically, it includes defining nodes and edges, and defining the skeleton of the graph based on the physical structure and business logic of the power grid.

[0047] Node definition:

[0048] Equipment nodes: power equipment such as substations, transformers, circuit breakers, disconnect switches, and surge arresters.

[0049] Component nodes: equipment components such as windings, iron cores, contacts, conductors, and insulators.

[0050] Defect nodes: equipment defects such as cracks, corrosion, overheating, electrical discharge, and wear.

[0051] Environmental factors include: pole location, meteorological data, and geographic information.

[0052] Edge definition:

[0053] Topology connection edges: equipment hierarchy such as substation → transformer, transformer → winding.

[0054] Dynamically associated edges: temporal relationships such as sensor → fault record, equipment → state change.

[0055] Attribute relationship edges: Attribute associations such as equipment → voltage level, component → material, etc.

[0056] Business relationship edge: Equipment → Maintenance plan, Defect → Handling measures, and other business rules.

[0057] Node feature encoding: Each node generates a feature vector that integrates information from multiple sources.

[0058] Structured data coding features: Attributes such as voltage level, rated capacity, and years of operation of equipment are extracted from the Production Management System (PMS). These numerical data are normalized, and categorical data are one-hot encoded, ultimately concatenating them into a 128-dimensional feature vector.

[0059] Semantic features of unstructured data: Technical manuals, historical fault reports, and inspection records of the collected equipment were analyzed. A pre-trained BERT model was used to extract 768-dimensional vectors corresponding to the text's tags as semantic features. For example, overheating-related semantic information was extracted from the report "Infrared temperature at the cap of phase B sleeve reaches 95℃".

[0060] Time-series data encoding characteristics: Minute-level or hourly sensor time-series data of load current, oil temperature, and dissolved gases in oil are acquired from SCADA / online monitoring systems. A single-layer LSTM network is used to encode the sequence within a fixed time window (e.g., the past 24 hours) into a 256-dimensional time feature vector to capture the trend of state evolution.

[0061] The 128-dimensional structured vector, 768-dimensional semantic vector, and 256-dimensional temporal vector are combined and concatenated to obtain the 1152-dimensional original feature vector of each node, which constitutes the node feature matrix of the graph structure data.

[0062] S2. Perform graph attention processing on the original feature vectors to obtain the vector representations of the nodes. Please refer to [link / reference]. Figure 2 .

[0063] Specifically, it includes:

[0064] S21, Vector Generation and Low-Rank Compression;

[0065] Among them: Vector generation uses a learnable linear transformation matrix to map the original feature vector of the node to the query vector Q, key vector K, and value vector V respectively. This step converts the comprehensive attribute information of the node into three dedicated vectors used in the attention mechanism to calculate relevance and extract information.

[0066] Low-rank compression: Traditional graph attention mechanisms require caching complete high-dimensional key and value vectors for all node pairs, resulting in storage and computational complexity as high as O(n²·d), where n is the number of nodes and d is the feature dimension, which is not feasible in large-scale power graphs.

[0067] This invention introduces a low-rank compression technique to compress the key vector K and value vector V. The motivation for this is that high-dimensional key-value matrices contain redundant information, which can be approximated by mapping to a low-dimensional latent space, thereby significantly reducing the number of parameters. Specifically, this is achieved by introducing a low-rank transformation matrix. The original key and value vectors are projected from the relatively high-dimensional first original dimension to the relatively low-dimensional second latent dimension; and to accommodate subsequent multi-head attention calculations, matrices are introduced and used. Projection is performed to transform the low-dimensional vector back into a specific multi-head format;

[0068] Through this compression, the storage requirement for the key-value cache is reduced from O(n²·d) to O(n·k·d), where k is the compressed potential dimension. In the above way, the low-rank compression step compresses the dimension from 1152 to 512, reducing this part of the storage overhead by more than 50%. This directly solves the defect of "storage explosion and inability to expand" in the background technology, making the construction of a large-scale power knowledge graph feasible in engineering.

[0069] S22, Decoupled rotational position encoding;

[0070] Rotation Position Encoding (RoPE) is applied only to the query vector Q. Specifically: for device and component nodes, the rotation angle is proportional to their level depth in the topological edge; for nodes strongly associated with time-series data (such as defect nodes), the rotation angle is associated with the position of the timestamp corresponding to that data in the sequence; the key vector K does not participate in rotation encoding to maintain the purity of its semantic features and avoid position encoding interfering with semantic information, thus achieving "decoupling".

[0071] S23, Multi-headed attention calculation;

[0072] Specifically: Set the total number of attention heads H to 128; divide the multiple attention heads into two groups;

[0073] A set of attention heads (1-64): Modeling hierarchical relationships.

[0074] The input is a RoPE-encoded query vector Q with hierarchical positions, along with the compressed key vector K and value vector V. Before calculating the attention score, a two-layer graph convolutional network (GCN) is invoked.

[0075] The GCN takes the compressed features of the nodes adjacent to the current node (based on the topological connection edges) as input and outputs a context vector that aggregates the local topological structure. This context vector is then fused with the query vector Q with hierarchical position encoded by RoPE, and then the attention weights are calculated with the compressed key vector K. This forces the attention heads to focus on learning physical inclusion relationships such as "substation-transformer".

[0076] The second set of attention heads (65-128) models dynamic temporal relationships.

[0077] The input is a RoPE-encoded query vector Q with time location, along with compressed key vector K and value vector V. When calculating attention, the model focuses on nodes with similar temporal feature vectors or temporal correlations. For example, the state nodes of the same device at different time points, or fault events and early warning signal nodes; thus, it specifically captures the evolution of state over time and the temporal propagation chain of faults.

[0078] Finally, the output vectors of the two sets of 128 heads are concatenated and passed through a linear projection layer to generate a 384-dimensional vector representation of each node. This vector representation of the node integrates spatial topology and temporal dynamic information.

[0079] S3. Hierarchical Processing and Dynamic Updates: The vector representations of nodes obtained from the multi-head attention module, which integrate spatial topology and temporal dynamic information, are refined and continuously evolved in a hierarchical manner by introducing domain knowledge structures and real-time signals. This enables hierarchical processing of the power knowledge graph and efficient updating of dynamic data.

[0080] Layered processing:

[0081] The concept layer aggregates the vector representations of nodes of the same category using a graph convolutional network (GCN) to extract general knowledge for that category. The specific steps are as follows:

[0082] Node Classification and Subgraph Construction: Based on knowledge from the power industry, nodes are classified according to equipment / component type, such as grouping transformer-related nodes into a transformer set and circuit breaker nodes into a circuit breaker set. For each category set, an induced subgraph is extracted based on the topological connection edges of the original full graph. A Graph Convolutional Network (GCN) is applied to each category subgraph. GCN smooths and enhances the vector representation of nodes by aggregating the features of their adjacent nodes, outputting an enhanced node vector representation for each specific node that incorporates general knowledge such as equipment type and voltage level.

[0083] The business layer is responsible for converting unstructured business rule text into computable semantic features of relation edges in the graph. The actual processing is to extract action entities (such as "defect elimination" operation) in the maintenance plan through the BERT model and generate semantic feature vectors of the edges.

[0084] The instance layer is responsible for fusing the exclusive and latest multimodal data for each specific node to achieve personalized and real-time updates of the node representation: when receiving the latest relevant images of the main transformer, such as (showing slight oil leakage in the bushing) and the corresponding work order text, CNN and BERT are used to extract visual and text features respectively, and the fusion is used to update the representation of the node in the instance layer.

[0085] Dynamic update: When the SCADA system receives the latest data on the sudden rise in oil temperature of the main transformer, the system only needs to use the trained MLA module to map the new time series data and perform fast attention fusion with the historical representation of the node to complete the node state update within seconds, without reconstructing the entire graph.

[0086] Feature fusion and knowledge reasoning: A gated fusion unit is designed to dynamically learn weights for the outputs of the concept layer, business layer, and instance layer, and then perform weighted summation to obtain the final comprehensive representation of each node; Dynamic attention adjustment: The attention weights of each layer are dynamically adjusted according to the query task. For example, in device fault diagnosis, the weights of the business layer and instance layer are increased. Specific implementation: The weights of each layer are dynamically adjusted through a learnable parameter θ, with the formula: W=σ(θ), where σ is the Sigmoid activation function.

[0087] Knowledge Reasoning and Completion: Relation prediction is performed using a knowledge-aware attention layer based on the final synthesized representation. For example, given the head entity master transformer and the relation, the prediction is made through the formula: ,in, For attention network functions, and These are the head entities and relation vectors, respectively, weighted by interaction relevance. The candidate tail entity vector is used to calculate the attention score of all candidate tail entities (such as various defects) and recommend the most likely defect type, thereby realizing automatic graph completion and root cause reasoning.

[0088] The power knowledge graph constructed using the method in this embodiment, compared to traditional methods,

[0089] 1. Significantly improved storage efficiency: Through MLA's low-rank compression technology, the storage overhead of the Key-Value cache is reduced by 93.3%. When building a power knowledge graph containing 1 million nodes, the storage requirement is reduced from O(n²•d) of the traditional method to O(n•k•d), where k=512 and d=128, which greatly reduces the hardware requirements;

[0090] 2. Significantly Enhanced Expressive Capabilities: MLA's multi-head mechanism captures device hierarchical relationships and dynamic temporal relationships in parallel, while decoupling RoPE technology enhances spatial topology awareness, improving entity relationship recognition accuracy from 80.19% of traditional methods to over 92%. In power operation and maintenance knowledge graphs, entity relationship recognition accuracy can be improved from 80.19% to 94%, and attribute relationship recognition accuracy from 86.77% to 96%.

[0091] 3. Significantly Improved Inference Speed: Leveraging the latent spatial mapping characteristics of MLA, inference speed is improved by over 30%. In power load forecasting tasks, MLA achieves an inference speed of 0.8ms per sample, far superior to traditional methods.

[0092] 4. Improved Human-Machine Collaboration Efficiency: MLA's dynamic display features and attention weight visualization reduce manual review steps by 70%. In power dispatch fault handling, by using a knowledge computing engine to find knowledge paths, the machine can provide dispatchers with key information after filtering, greatly improving the efficiency of fault diagnosis and handling.

[0093] 5. Enhanced Knowledge Reasoning Ability: MLA's multi-head division of labor and decoupling RoPE technology enable knowledge graphs to better represent the complex topological relationships and dynamic temporal characteristics of power systems. In the diagnosis of thermal faults in power equipment, the diagnostic accuracy is significantly improved by combining superpixel segmentation and low-rank representation techniques.

[0094] Example 2

[0095] Based on the scheme in Example 1, this example further provides a power knowledge graph construction system based on multi-head attention, used to implement the power knowledge graph construction method based on multi-head attention, specifically as follows: Figure 3 As shown, the system includes: a data acquisition module, a graph attention processing module, and a graph construction module.

[0096] The specific data acquisition module is used to automatically identify power entities and relationships from multi-source data such as SCADA, PMS, and fault reports. It directly encodes structured data (such as voltage level 220kV); extracts semantic vectors from structured data (such as inspection text "B-phase bushing overheating") using the BERT model; and encodes time-series data (such as temperature sensor curves) into time feature vectors using an LSTM network. Graph structure assembly: the identified entities are used as nodes, relationships as edges, and the encoded features are used as node features to assemble a computer-readable, initial graph structure data. This module unifies power data scattered across different systems and formats into a standardized graph data model, providing the graph representation learning module with pre-cleaned and semantically understood input.

[0097] The vector generation unit in the graph attention processing module transforms point features into standard inputs for the attention mechanism, completing the data format conversion. The low-rank compression unit reduces the dimensionality of key and value vectors, lowering storage and computational complexity and overcoming the fundamental efficiency bottleneck of traditional methods in handling large-scale power graphs, making large-scale applications possible. The coupled encoding unit applies Rotation Position Encoding (RoPE) only to the query vector. This allows for the injection of prior knowledge from the power domain into the model, encoding "hierarchical positions" for spatial entities and "temporal positions" for temporal entities, thereby greatly enhancing the model's ability to perceive the unique structure of the power system.

[0098] Multi-head attention unit: The attention heads are divided into two groups to work in parallel. The first group (spatial group): Combined with a graph convolutional network (GCN), it is specifically used to learn the hierarchical containment relationships between devices (such as substation-transformer); the second group (temporal group): It is specifically used to process temporal features and capture the dynamic evolution of states. It simultaneously and accurately overcomes the two major challenges of spatial topology modeling and dynamic temporal modeling, so that the output node representation contains rich spatiotemporal information and generates high-quality node vector representations. Within a single architecture, it collaboratively solves the problems of efficiency through low-rank compression, spatial awareness (through decoupling RoPE and GCN), and temporal awareness (through decoupling RoPE and division of labor).

[0099] The knowledge graph construction module is used to process node representations by feeding them into the concept layer, business layer, and instance layer, integrating general knowledge, business rules, and real-time status, and then merging them into the final representation. When new data arrives, the module uses the previously trained model parameters to quickly and incrementally update relevant nodes, rather than reconstructing the entire graph. Based on the final node representation, the module predicts missing relationships between entities (such as inferring the most likely cause of a defect) through knowledge-aware attention layers, automatically enriching and completing the knowledge graph. The completed and continuously updated knowledge graph is stored in a graph database, providing query, reasoning, and computation services to upper-layer business applications (such as fault diagnosis and intelligent inspection), producing a dynamically evolving power sector knowledge graph that can be directly used to support business operations.

[0100] Example 3

[0101] In order to apply the solution in Example 1, the power knowledge graph construction method based on multi-head attention provided in Example 1 also provides a computer-readable storage medium for executing the solution. The medium stores program instructions that can be run by a processor. After connecting to the power system data source, it can automatically and efficiently complete the construction and maintenance of the knowledge graph.

[0102] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for obtaining node vector representations during the construction of a power knowledge graph, characterized in that, include: (a) Perform linear transformation based on node features to generate query vector, key vector, and value vector; (b) Perform dimensionality reduction and compression on the key vector and value vector; (c) Decoupled rotational position encoding of the query vector: Apply rotational position encoding to the query vector while keeping the key vector free from rotational position encoding; (d) Based on the compressed key vector and value vector, and the encoded query vector, the attention weights between nodes are calculated through a multi-head attention mechanism to obtain the vector representation of the nodes; wherein, the multiple attention heads in the multi-head attention mechanism are divided into two groups. The first group of attention heads aggregates the features of adjacent nodes through a graph convolutional network to model the hierarchical relationship of power equipment, and the second group of attention heads is used to process the temporal features in the node features to model dynamic temporal relationships. The total number of attention heads H is set to 128. Attention heads 1-64 form the first group, which models hierarchical relationships, and the remaining attention heads 65-128 form the second group, which models dynamic temporal relationships. The input is a query vector Q with hierarchical positions encoded by rotation, as well as a compressed key vector K and value vector V. Before calculating the attention score, a two-layer graph convolutional network is invoked. The graph convolutional network takes the compressed features of the nodes adjacent to the current node as input and outputs a context vector that aggregates the local topology. The context vector is fused with the query vector Q and then the attention weights are calculated with the compressed key vector K.

2. The method for obtaining node vector representations during the construction of a power knowledge graph according to claim 1, characterized in that, The dimensionality reduction compression maps the key vector and value vector from a first original dimension to a second latent dimension, wherein the second latent dimension is smaller than the first original dimension; the dimensionality reduction compression is achieved through a linear transformation matrix.

3. The method for obtaining node vector representations during the construction of a power knowledge graph according to claim 2, characterized in that, The rotational position encoding is used to inject positional information corresponding to the hierarchical relationship into the query vector of the first set of attention heads, and to inject positional information corresponding to the dynamic temporal relationship into the query vector of the second set of attention heads.

4. A method for constructing an electricity knowledge graph based on multi-head attention, comprising: Based on graph structure data with nodes and node features, obtain the vector representation of nodes; Based on the vector representation of the nodes, the power knowledge graph is constructed; The key feature is that the method for representing the vectors of nodes adopts the method for obtaining the vector representations of nodes during the construction of the power knowledge graph as described in claim 1.

5. The method for constructing an electricity knowledge graph based on multi-head attention according to claim 4, characterized in that, The nodes include: device nodes, component nodes, defect nodes, and environment nodes; the node features include structured data encoding features, unstructured data semantic features, and time-series data encoding features.

6. The method for constructing an electricity knowledge graph based on multi-head attention according to claim 5, characterized in that, The structured data encoding features are obtained based on the encoding of power equipment attribute data including voltage level and capacity; the unstructured data semantic features are obtained by extracting text data of the power system including equipment documents and fault reports through the BERT model; and the time-series data encoding features are obtained by encoding the time-series sensor data including the power SCADA system through LSTM.

7. The method for constructing an electricity knowledge graph based on multi-head attention according to claim 4, characterized in that, The construction of a power knowledge graph based on node vector representations includes a layered processing and fusion process: the vector representations of the nodes are processed in layers of concept, business, and instance, and real-time data is dynamically updated to the node representations of the instance layers; the outputs of the concept, business, and instance layers are weighted and fused to obtain the final node representations for knowledge reasoning. The processing of the concept layer includes extracting general device knowledge through graph convolutional networks; the processing of the business layer includes generating semantic features of edges through a pre-trained language model. The processing of the instance layer includes fusing image and text features to update node representations.

8. The method for constructing an electricity knowledge graph based on multi-head attention according to claim 7, characterized in that, Based on the final node representation, relation prediction and graph completion are performed through a knowledge-aware attention layer, and attention embedding is calculated based on the knowledge-aware attention layer. : ,in, For attention network functions, and These are the head entities and relation vectors, respectively, weighted by interaction relevance. is the candidate tail entity vector.

9. A power knowledge graph construction system based on multi-head attention, characterized in that, The system is used to implement the power knowledge graph construction method based on multi-head attention as described in any one of claims 4 to 8, the system comprising: The data acquisition module is used to acquire graph structure data containing nodes and node features; A graph attention processing module is used to receive the graph structure data and process it to obtain vector representations of the nodes. The graph attention processing module includes: A vector generation unit is used to generate query vectors, key vectors, and value vectors based on the features of the node; A low-rank compression unit is used to reduce the dimensionality of the key vector and value vector; A decoupling encoding unit is used to perform decoupling rotational position encoding on the query vector separately: applying rotational position encoding to the query vector while keeping the key vector from undergoing rotational position encoding. The multi-head attention unit is used to perform attention calculations based on the compressed key vector, value vector, and encoded query vector, and divides the attention heads into two groups to model hierarchical relationships and dynamic temporal relationships respectively. The graph construction module is used to construct the power knowledge graph based on the vector representation of the nodes.

10. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the power knowledge graph construction method based on multi-head attention as described in any one of claims 4 to 8.

Images