Device fault text implicit relationship prediction method based on graph structure learning model

Abstract

The application discloses to the technical field of power grid equipment operation and maintenance, and discloses a device fault text implicit relationship prediction method based on a graph structure learning model, which is based on the obtained text data set of power grid equipment fault maintenance; the target is to model the entities and relationships in the device fault text as a graph structure, combine the entity vectors generated by the large model with the graph neural network for message passing and relationship inference, design a graph structure learning model to predict unknown fault relationships between different devices; finally, a classifier is used to predict the unknown relationships between entities. The prediction method uses the characteristics of the large language model to mine the vector representation of the entity in the sentence, combines the graph neural network model, uses the subgraph sampling mechanism considering the random walk of fault influence, constructs the subgraph structure between multiple entities, and finally uses the message passing mechanism based on the pyramid attention mechanism to obtain the implicit relationship prediction result in the power grid equipment fault text.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid equipment operation, maintenance and overhaul, and in particular to a method for predicting implicit relations in equipment fault texts based on a graph structure learning model. Background Art

[0002] Currently, with the coordinated development of power generation, grid loads, and storage, and the large-scale integration of new energy equipment, the fault texts for grid equipment operation, maintenance, and inspection have undergone significant changes. New energy equipment introduces a more diverse range of fault types, such as photovoltaic inverter failures and wind power equipment outages. These faults also have more complex interactions with traditional grid equipment. The correlation between devices has increased, and more cross-device and cross-system records have appeared in fault texts. Traditional text relationship extraction methods are unable to identify the potential processes of mutual influence between device failures and the implicit fault correlation relationships, such as the complex causal chain relationship of voltage anomalies caused by wind power fluctuations. In addition, the inspection and maintenance measures for new energy equipment are more specialized. For example, the text involves the inspection and maintenance of abnormal photovoltaic array grid connection, wind turbine maintenance, and other new energy-specific operations.

[0003] Power companies currently have well-established equipment operation and maintenance (O&M) and troubleshooting systems, accumulating a rich resource of textual records of various equipment failures. These fault texts contain a wealth of implicit relationships. Analyzing these implicit relationships can help automatically infer fault causes, shorten troubleshooting time, discover multi-level connections between equipment, and identify fault propagation patterns. Simultaneously, mining implicit relationships facilitates alarm and fault prediction, providing early warning of potential problems, supporting preventive maintenance, and reducing the risk of equipment damage. Furthermore, implicit relationships can optimize fault handling processes, automatically recommending solutions through correlation analysis, and improving fault handling efficiency. Therefore, predicting implicit relationships in power grid equipment fault texts plays a vital role in power grid equipment O&M and fault handling.

[0004] When processing complex power grid equipment fault text, traditional relationship prediction methods face three major shortcomings and bottlenecks:

[0005] (1) Inadequate handling of domain-specific terms and polysemy is a major problem with traditional methods. Power grid fault texts are filled with a large number of technical terms and equipment names, and general natural language processing models have difficulty accurately identifying these domain-specific terms. In addition, polysemy in power grid texts is often overlooked. For example, "tripping" can indicate both a fault phenomenon and normal operation. Traditional models have difficulty correctly distinguishing these based on contextual information, resulting in inaccurate entity recognition and relationship prediction.

[0006] (2) Insufficient modeling of long texts and complex contextual dependencies also limits the effectiveness of traditional relationship prediction methods. Power grid fault texts are typically long and involve multiple entities and their relationships. Traditional models often ignore long-range contextual associations when processing long texts and fail to capture global dependency information, especially in texts that span sentences or have complex structures. Simple sequence models or shallow models have difficulty capturing these long-range dependencies, resulting in poor prediction of complex relationships between entities.

[0007] (3) The diversity of complex relationships and the lack of cross-domain dependency capture are also key bottlenecks. The entity relationships in power grid equipment failure texts include diverse relationships such as causal relationships, temporal dependencies, and functional dependencies. Traditional methods rely on simple rules or shallow models and cannot effectively capture these complex, multi-dimensional relationships. In addition, with the integration of new energy devices, the relationship between devices and systems in the power grid has become more complex. Traditional methods have difficulty handling complex dependencies across devices and systems, resulting in low relationship prediction accuracy and insufficient generalization capabilities.

[0008] In summary, relationship prediction technology for power grid equipment fault text faces numerous challenges. Traditional methods primarily rely on manually defined rules and templates, which are not only inefficient but also struggle to capture diverse entity relationships, especially when dealing with complex and long texts. As the length of fault text increases, relationships within sentences tend to become diluted, making it difficult to capture long-range dependencies between entities. Furthermore, traditional methods typically focus only on the relationships between fault entities within a single sentence, ignoring entity relationships across sentences and even paragraphs, which severely limits their ability to capture global information. Information sparsity and weakened dependencies further reduce the accuracy and generalization of relationship predictions. Summary of the Invention

[0009] This paper proposes a method for predicting implicit relationships in equipment fault text based on a graph structure learning model, aiming to address the challenges of identifying and predicting entity relationships in complex fault texts of power grid equipment. The method is based on a text dataset of power grid equipment fault inspections. The text contains entities such as equipment alarms, fault phenomena, fault causes, and treatment measures. Relationships include eight relationships: solutions, generation, inclusion, location, and impact. The goal is to model the entities and relationships in the equipment fault text as a graph structure, combine the entity vectors generated by the large model with graph neural networks for message passing and relationship inference, and design a graph structure learning model to predict unknown fault relationships between different devices. Finally, a classifier is used to predict unknown relationships between entities.

[0010] The technical solution adopted in the present invention is:

[0011] A method for predicting implicit relationships in equipment fault text based on a graph structure learning model. This method includes: an entity encoding layer that integrates a large language model (GPT) and entity type labels; a random walk subgraph construction method that considers fault impact; a graph message passing layer based on a pyramid attention mechanism; and fault relationship prediction based on node features.

[0012] This method for predicting implicit relationships in equipment fault text based on a graph structure learning model is based on a text dataset of power grid equipment fault inspection and maintenance. By modeling the entities and relationships in the equipment fault text as a graph structure, the entity vectors generated by the large model are combined with the graph neural network for message passing and relationship inference. The graph structure learning model is used to predict unknown fault relationships between different devices. Finally, a classifier is used to predict unknown relationships between entities.

[0013] Furthermore, the method for predicting implicit relations in equipment fault text based on the graph structure learning model includes the following steps:

[0014] Step 1: Deep text encoding: Use a large language model (LLM) to encode the text and, combined with entity type labels, convert the entities into word vectors. Entities mentioned in the text are mapped to graph structures using position indexes. Finally, deep encoding is performed on the fault text to construct the entity graph structure.

[0015] Step 2: Graph Structure Modeling and Random Walk Subgraph Sampling: The entities and relationships in Step 1 are modeled as undirected graphs, and an improved random walk mechanism is introduced to account for the impact of node failures. Dynamic weighting is performed to reduce redundant information and extract key relationship information, constructing multiple compact subgraph structures. Finally, representative subgraph structures are extracted from the fully connected graph.

[0016] Step 3: Feature aggregation and relationship prediction: Combined with the pyramid attention mechanism, the subgraph structure information is converted into a global entity vector representation through message passing. Finally, entity features are aggregated in the subgraph and relationship prediction is performed.

[0017] Step 4: Fault relationship prediction and classification: Perform relationship prediction on the generated entity vectors through the decoding layer; map the global entity vectors to specific relationship classifications.

[0018] Furthermore, the entity encoding layer that integrates the large language model GPT and the entity type label includes the following steps:

[0019] Step 1, word embedding of entity type:

[0020] When processing text, the entity type label is embedded on both sides of the entity name; for the text after the above processing, the digital representation of the text is extracted through the dictionary in the pre-trained model, that is, for {x1,x2,...,x n}, use the following formula to generate the corresponding {n1,n2,...,n n}∈R represents;

[0021] Embedding(x i )=E(x i )

[0022] Among them, x i Represents each word; the result generated by the Embedding function is {n1,n2,...,n n}∈R;

[0023] Step 2, position encoding:

[0024] Introducing positional encoding to preserve word order;

[0025] The trigonometric function position encoding is used, as shown in the following formula:

[0026]

[0027] Among them, PE represents the encoding function; pos represents the position information of the word in the text; i represents the sequence information; d represents the generation dimension of the model vector;

[0028] Step 3, multi-head self-attention mechanism:

[0029] The key component of Transformer is the multi-head self-attention mechanism, which mainly calculates the attention scores of different heads and multiplies them with the value matrix to output representation;

[0030] The output vector formula of the i-th head is:

[0031]

[0032] Where T represents the transpose operation; d k Indicates the length of the sequence; after obtaining the attention of multiple independent heads, the final output vector is concatenated after the linear transformation of the parameters;

[0033] Finally, the outputs of all heads are concatenated through linear transformation and the comprehensive representation is:

[0034] A=concat(head1,head2,...,head n )W+b

[0035] Where A represents the vector matrix; concat represents the concatenation of vectors, head represents different heads, W and b represent the parameter matrix and bias vector in the linear transformation respectively;

[0036] Step 4: Construction of graph structure:

[0037] Construct a graph structure G = (V, E), where the edge set E is represented by an upper triangular matrix with a default value of 1, and the graph forms a fully connected undirected graph with entities as nodes;

[0038] The initial vector for each node is:

[0039] node i =mean(get_entity(A,start,end))

[0040] Among them, node i ∈H (0) , represents the initial vector of a node; mean represents the average operation on the vector; get_entity represents the function of extracting the entity vector from the vector matrix; entity i A vector representing entity i; start and end are node indices of the entity in the sentence;

[0041] Through the above steps, an encoding layer is constructed that combines entity types and effectively processes sequence data.

[0042] Furthermore, the random walk subgraph construction method considering the impact of faults includes the following steps:

[0043] Step 1: Graph model construction:

[0044] Using the random walk-based subgraph sampling mechanism, the graph G = (V, E) is sampled into subgraph G * =(V * , E * ), where V * ∈V,E * ∈E;

[0045] Step 2, random walk strategy:

[0046] The walking strategy is expressed as the following formula:

[0047]

[0048] Among them, P(path) represents the probability of a specific path; L is the length of the wandering path; d(v i ) represents the degree of node i; v i is the node at step i; d is the degree function of the node;

[0049] Step 3: Dynamically adjust the roaming strategy:

[0050] By introducing the fault influencing factor, the number of steps and paths of the walk are dynamically adjusted according to the local structural information of the graph:

[0051] First, we use breadth-first and depth-first strategies to capture horizontal and depth locality information respectively. We use the head entity as the first node of the graph and the rest as expansion nodes to achieve the purpose of subgraph sampling.

[0052] The wandering strategy is adjusted by introducing two transition probabilities, as shown below:

[0053]

[0054] Among them, P(v i+1 |v i ,v i-1 ) means that when the current node v is known i and the previous node v i-1 In the case of i+1 ;d(v i ) is the node v i degree; p base is the basic probability, p is the return parameter; q base is another basic probability, and q is the breadth-first parameter of the walk.

[0055] Furthermore, the random walk subgraph construction method considering the impact of faults further includes the following steps:

[0056] Step 4: Fault influencing factors:

[0057] Fault impact FI(v i ) is defined as:

[0058]

[0059] Among them, d(v i ) is the node v i Degree; D max The maximum degree of all nodes in the graph;

[0060] Consider the impact of the fault and adjust the parameter pb accordingly ase and qb ase , pb ase and qb ase The calculation formula is as follows:

[0061]

[0062] q base =q base ·(1+FI(vi ))

[0063] Finally, p base and q base Substitute the fault impact FI(v i ), and obtain the random walk method considering the impact of faults.

[0064] Furthermore, the graph message passing layer based on the pyramid attention mechanism includes the following steps:

[0065] Step 1: Node message collection and attention weight calculation:

[0066] At each level of the pyramid, node i collects messages from its neighbor node set N and applies the attention mechanism to calculate the weighted feature vector;

[0067] The attention weights between nodes are expressed as follows:

[0068]

[0069] in, Indicates the attention weight of node i to node j in layer l; exp represents the exponential function; LeakyReLU represents the activation function; α is a learnable vector; T represents the transposition operation; W (l) is a parameter matrix; represents the initial embedding vector of node i, i∈v; || represents the concatenation between features; Represents the feature representation vector of node j in layer l; k is the index of the neighbor node; Ν(i) represents the set of neighbor nodes of node i; Represents the feature representation vector of node k in layer l;

[0070] Step 2, feature aggregation and pooling:

[0071] For the extraction of feature information of each layer, three pooling operations of different scales are used to obtain feature information. The multi-scale pooling output P is expressed as follows: (l) ;

[0072] P (l) =Pool(H (l) )

[0073] Among them, P (l) Represents the multi-scale pooling output obtained at layer l; Pool is a function of pooling operation; H (l) is the feature representation matrix at layer l;

[0074] Step 3, node feature update:

[0075] In GNN, the information transmission of each node is completed through the aggregation of neighboring nodes; after combining the attention mechanism, the node feature update formula is expressed as:

[0076]

[0077] in, is the feature representation vector of node v at layer l+1; σ is a nonlinear activation function; u represents the index of the neighbor node; N(v) represents the set of neighbors of node v; is the attention weight, which indicates the importance between node v and its neighbor node u; W (l) is a parameter matrix; is the feature representation vector of node u in layer l.

[0078] Step 4, comprehensive representation:

[0079] Combining the above parts, the overall representation of the pyramid attention mechanism and GNN is shown in the following formula:

[0080]

[0081] Among them, H (l+1) Represents the l+1th layer node vector; Fusion represents the feature splicing operation; is the feature representation vector of neighbor node u in layer l; P (l) Represents the multi-scale pooling output obtained at layer l.

[0082] Furthermore, the fault relationship prediction based on node features includes the following steps:

[0083] Step 1, node feature representation:

[0084] Using the message passing mechanism, the feature vectors of node i and node j are aggregated to generate relationship features;

[0085] Aggregation methods include maximum and mean operations, and the formula is as follows:

[0086] R(node i ,node j )=concat(max(node i ,node j ),mean(node i ,node j ))

[0087] Among them, R(node i ,node j ) represents the relationship feature between node i and node j; node iA vector representing node i; node j represents the vector of node j; concat represents the concatenation function; max and mean represent the mean function;

[0088] Step 2, relationship prediction:

[0089] After generating the relationship features between nodes, they are mapped to the number of relationship types through a linear layer for relationship prediction. The formula is:

[0090] r ij =softmax(W R ·R(node i ,node j )+b R )

[0091] Among them, r ij Represents the relationship classification between node i and node j; softmax represents the softmax function, which is used to convert the input vector into a probability distribution; W R and b R Represent the parameter matrix and bias vector of the linear layer respectively;

[0092] Step 3, loss function:

[0093] Use the Focal Loss function to increase the model's attention to a small number of positive samples;

[0094] Focal Loss is defined as follows:

[0095]

[0096] Among them, Focal Loss refers to the value of the cross entropy loss function; is the weight factor that controls the balance of positive and negative samples; p t is the probability of positive samples predicted by the model; β is a regulating factor; log(p t ) is part of the cross entropy loss.

[0097] Furthermore, the text dataset obtained during the power grid equipment fault inspection and maintenance contains information from multiple aspects, including but not limited to:

[0098] Equipment alarm: A warning signal issued by the equipment, indicating a fault or abnormal state;

[0099] Fault phenomenon: Specific manifestations of the fault, such as abnormal working status of the equipment or error information;

[0100] Fault cause: the cause of the fault;

[0101] Action: The repair or maintenance measures taken in response to the fault.

[0102] Furthermore, the relationships in the equipment failure text are different types of connections between entities in the power grid equipment failure text; the relationships include:

[0103] Solution: measures on how to deal with the fault;

[0104] Cause: the cause or source of a fault;

[0105] Includes: Parts or related equipment included in a certain equipment;

[0106] Location: The specific location where the fault occurred or the relevant geographical location;

[0107] Impact: The extent to which the fault affects the system or other equipment.

[0108] The beneficial effects of the present invention are:

[0109] This method for predicting implicit relationships in power grid equipment fault text based on a graph structure learning model leverages the characteristics of a large language model to mine the vector representations of entities in sentences. Combined with a graph neural network model, it utilizes a random walk subgraph sampling mechanism that considers fault impact to construct a subgraph structure between multiple entities. Finally, a message passing mechanism based on a pyramid attention mechanism is used to predict implicit relationships in power grid equipment fault text. Experimental results show that this method for predicting implicit relationships in power grid equipment fault text based on a graph structure learning model significantly improves the relationship prediction performance compared to other baseline models. This method for predicting implicit relationships in power grid equipment fault text based on a graph structure learning model demonstrates the potential for predicting relationships within relationships through the combination of a large language model and a graph neural network, providing valuable reference and direction for future research. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0111] Figure 1 This is a flow chart of the method for predicting implicit relations in equipment fault text based on a graph structure learning model of the present invention;

[0112] Figure 2 A diagram showing the walking strategy of the present invention;

[0113] Figure 3 This is a diagram of the subgraph sampling mechanism of the present invention;

[0114] Figure 4 Graph showing the impact of different α parameters on the model. DETAILED DESCRIPTION

[0115] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0116] Due to the technical challenges of relationship prediction in power grid equipment fault text, traditional methods rely on manual rules and struggle to cover diverse entity relationships. In complex and lengthy texts, long-range dependencies between entities are difficult to capture, resulting in reduced accuracy. Traditional methods typically focus only on relationships within a sentence, ignoring relationships across sentences and paragraphs, resulting in insufficient global information capture. Therefore, this embodiment provides a method for predicting implicit relationships in equipment fault text based on a graph-structured learning model, aiming to address the challenges of identifying and predicting entity relationships in complex power grid equipment fault text. This method is based on a dataset of text from power grid equipment fault repairs. The text contains entities such as equipment alarms, fault symptoms, fault causes, and solutions. The relationships include eight relationships: solution, generation, inclusion, location, and impact. The goal is to model the entities and relationships in equipment fault text as a graph structure, combine the entity vectors generated by a large model with a graph neural network for message passing and relationship inference, and design a graph-structured learning model to predict unknown fault relationships between different devices. Ultimately, a classifier is used to predict unknown relationships between entities.

[0117] Assume that the power grid equipment fault text dataset X={x1,x2…,x n}, each text x i Contains multiple entities E = {e1, e2, ..., e n}, such as equipment, fault phenomenon, fault cause, treatment measures, treatment results, etc.

[0118] The goal of this equipment fault text implicit relationship prediction method based on graph structure learning model is to predict the relationship between any two entities e i and e j Relationship between ij , and judge the type of relationship between entities based on the four types of implicit relationships; construct a graph structure G = (V, R) from the power grid equipment fault text, where V is the entity set and E is the edge set, convert the problem into the task of predicting the edges between nodes in the graph, and design a graph structure learning model to capture the implicit entity relationships and complete the prediction.

[0119] This method for predicting implicit relationships in equipment fault text based on a graph structure learning model mainly includes the following four parts: an entity encoding layer that integrates the large language model (GPT) and entity type labels; a random walk subgraph construction method that considers the impact of faults; a graph message passing layer based on a pyramid attention mechanism; and fault relationship prediction based on node features.

[0120] 1. Entity encoding layer that integrates the large language model GPT and entity type labels:

[0121] Large language models are primarily used to process text sequence data. They use a masking mechanism to predict the probability of the next word in a sentence. Trained on massive amounts of text, they identify relationships between words and sentences within context and generate coherent and semantically sound text. As a representative large language model, the GPT model outperforms other models in terms of both the number of parameters and training methods. Existing entity encoding methods typically rely on fixed embedding vectors, which fail to fully capture the diverse meanings of entities in different contexts. The semantics of entities can vary in different scenarios, and existing models may not be able to accurately handle this ambiguity.

[0122] The encoding process of this embodiment based on the GPT pre-training model is mainly divided into three parts: word embedding based on entity type, position encoding, and multi-head self-attention mechanism; the specific steps are as follows:

[0123] Step 1, word embedding of entity type:

[0124] The word embedding method based on entity type encodes the entity by taking the entity type into consideration. The traditional encoding method based on various pre-trained models inputs a paragraph into the pre-trained model. This method processes the entity part and the sentence part in a unified way and does not provide more hints for the classification of downstream relationships.

[0125] This embodiment embeds the entity type label on both sides of the entity, as shown below:

[0126] The new mayor of <city> New York< / city> City <name> De Blasio< / name> isnative-born.

[0127] Add entity types city and name to both New York and De Blasio to achieve the purpose of entity type feature prompts when pre-training the model.

[0128] Then, for the processed text, the digital representation of the text is extracted through the dictionary in the pre-trained model, that is, for {x1,x2,...,x n}, use formula 1 to generate the corresponding {n1,n2,...,nn}∈R represents.

[0129] Embedding(x i )=E(x i ) (1)

[0130] Among them, x i Represents each word; the result generated by the Embedding function is {n1,n2,...,n n}∈R.

[0131] Step 2, position encoding:

[0132] Since the Transformer does not have the position information of the words in the text, it is necessary to introduce position encoding to preserve the order of the text. Position encoding usually uses trigonometric function position encoding, which uses sine and cosine functions to generate a unique position encoding for the position in the text, helping the Transformer model to process sequence data. Specifically, it is shown in Equation 2:

[0133]

[0134] Among them, PE represents the encoding function; pos represents the position information of the word in the text; i represents the sequence information; and d represents the generation dimension of the model vector.

[0135] Step 3, multi-head self-attention mechanism:

[0136] The most representative structure in the Transformer is the multi-head self-attention mechanism, which processes the input text. By calculating the attention score of each head on the value vector and multiplying it by the value matrix V, a vector output representation of each head is obtained. Similarly, after calculations are performed on multiple heads, they are finally spliced ​​together to output the vector representation of the sequence. Formula 3 represents the output vector of the i-th head:

[0137]

[0138] Where T represents the transpose operation; d k Indicates the length of the sequence;

[0139] After obtaining multiple independent head attention splices, the final output vector is obtained after the linear transformation of the parameters. Specifically, it is shown in Formula 4:

[0140] A=concat(head1,head2,...,head n )W+b (4)

[0141] Wherein, A represents a vector matrix; concat represents the concatenation of vectors; head represents different heads; W and b represent the parameter matrix and bias vector in linear transformation, respectively.

[0142] Step 4, construction of graph structure G=(V, E):

[0143] The edge set E of the graph can be represented as an upper triangular matrix with a default value of 1, represented in the graph as a fully connected undirected graph with entities as nodes. The internal node vector of the graph is obtained by the vector matrix and the position index of the entity in this article, and the specific vector representation of a node is shown in formula 5:

[0144] node i =mean(get_entity(A,start,end)) (5)

[0145] Wherein, node i ∈H (0) , represents the initial vector of a certain node; mean represents the average operation on the vector to obtain a unified shape representation; get_entity represents the function of extracting entity vector from the vector matrix; entity i represents the vector of entity i; start and end are the node indexes of the entity in the sentence.

[0146] Through the above steps, an encoding layer that combines entity types and effectively processes sequence data can be constructed, providing more accurate feature representation for downstream tasks.

[0147] II. Random walk subgraph construction method considering fault influence:

[0148] With the increase of sentence length, the dependency relationship between entities tends to weaken, thus increasing the difficulty of relationship prediction. Graph neural network (GNN) is a deep learning model specially designed for processing graph structure data, which can effectively capture and propagate the dependency relationship between nodes, and thus performs well in handling relationship prediction problems.

[0149] Step 1, graph model construction:

[0150] The present invention models the entities in a sentence and the relationships between them as a graph structure, where nodes represent entities and edges represent possible relationships between them. Since the graph type is a fully connected undirected graph structure, it is difficult to directly pass messages through a direct graph neural network. At the same time, since the relationship prediction is performed in the entire data set, the entities in each sentence will be saved in a large graph. If the prediction calculation is performed directly on the entire graph, it will not only be highly complex and memory-intensive, but also make the propagation of important features unclear. This method adopts a subgraph sampling mechanism based on random walks, sampling the graph G = (V, E) as a subgraph G * =(V * , E * ), where V * ∈V,E * ∈E, the subgraph can better preserve the local and overall information of the graph and is suitable for learning embedded vectors;

[0151] Step 2, random walk strategy:

[0152] Through the subgraph sampling mechanism, GNN can also quickly integrate information from neighboring nodes, remove redundant nodes, and help more accurately predict complex relationships in sentences.

[0153] The walking strategy can be expressed as shown in Equation 6:

[0154]

[0155] Among them, P(path) represents the probability of a specific path; L is the length of the wandering path; d(v i ) represents the degree of node i; v i is the node at step i; d is the degree function of the node.

[0156] Step 3: Dynamically adjust the roaming strategy:

[0157] Specifically, since existing random walk strategies all have fixed step lengths and predefined sampling depths, this embodiment introduces fault impact factors on this basis and dynamically adjusts the number of steps and paths of the walk according to the local structural information of the graph.

[0158] First, we standardize the random walk strategy by using breadth-first and depth-first strategies, which capture lateral locality and depth locality, respectively. We select the head entity as the first node of the graph, and the remaining nodes as expansion nodes, achieving subgraph sampling. Equation 7 adjusts the walk strategy by introducing two transition probabilities: the forward transition probability and the breadth transition probability.

[0159]

[0160] Among them, P(v i+1 |vi ,v i-1 ) means that when the current node v is known i and the previous node v i-1 In the case of i+1 ;d(v i ) is the node v i The degree, i.e. the number of neighbors; p base is the basic probability, p is the return parameter, which affects the probability of returning to the previous node; q base is another basic probability, and q is the breadth-first parameter of the walk, which affects the probability of visiting new neighbors.

[0161] The way of roaming strategy, such as Figure 2 shown.

[0162] Step 4: Fault influencing factors:

[0163] In order to adaptively adjust p and q, this embodiment defines parameters based on the local characteristics of the node, that is, the fault impact. For the fault impact factor, the higher the fault impact of the node, the more critical the node is in the graph. Therefore, the fault impact FI (v i ) is defined as shown in Equation 8:

[0164]

[0165] Among them, d(v i ) is the node v i Degree; D max The maximum degree of all nodes in the graph, used for normalization.

[0166] Secondly, this factor is integrated into the parameters p and q. This represents a random walk exploring more neighboring nodes, which decreases as the impact of the fault increases. The parameter q represents the movement from the node to other neighboring nodes, which increases as the impact of the fault increases. This is shown in Equation 9:

[0167]

[0168] q base =q base ·(1+FI(v i )) (9)

[0169] Finally, substituting Equation 9 into Equation 7, we can obtain the random walk method considering the impact of faults.

[0170] Based on the above steps, adaptively adjusting the walk strategy to better capture graph structure information helps improve the accuracy of relationship prediction.

[0171] 3. Graph Message Passing Layer Based on Pyramid Attention Mechanism:

[0172] After obtaining a subgraph sample based on the head entity, node vectors need to be updated through the graph's message passing mechanism. GNN-based message passing directly exchanges information based on a node's neighboring nodes. This approach may have difficulty capturing long-range dependencies or complex graph structures in relationship prediction, resulting in information loss or incomplete features.

[0173] To this end, this paper selectively exchanges information with neighboring nodes by using a pyramid attention mechanism. The pyramid-based attention mechanism mainly utilizes three different levels of node parameters, which can be abstracted to capture the features of the bottom layer, middle layer, and top layer. Specifically, a different attention matrix is ​​generated for each layer.

[0174] The specific steps are as follows:

[0175] Step 1: Node message collection and attention weight calculation:

[0176] At each level of the pyramid, node i collects messages from its neighbor node set N and applies the attention mechanism to calculate the weighted feature vector. The attention weight between nodes is shown in Equation 10:

[0177]

[0178] in, Indicates the attention weight of node i to node j in layer l; exp represents the exponential function; LeakyReLU represents the activation function; α is a learnable vector; T represents the transposition operation; W (l) is a parameter matrix; Represents the initial embedding vector of node i, i∈v; || represents the splicing between features; represents the feature representation vector of node j in layer l; k is the index of the neighbor node; N(i) represents the set of neighbor nodes of node i; Represents the feature representation vector of node k in layer l.

[0179] Step 2, feature aggregation and pooling:

[0180] For the extraction of feature information of each layer, this embodiment adopts three different scales of pooling operations to obtain feature information, and uses formula 11 to express the multi-scale pooling output P (l) ;

[0181] P (l) =Pool (H (l) ) (11)

[0182] Among them, P (l) Represents the multi-scale pooling output obtained at layer l; Pool is a function of pooling operation; H(l) is the feature representation matrix at layer l.

[0183] Step 3, node feature update:

[0184] In GNN, the information transmission of each node is completed by aggregating neighboring nodes. After combining the attention mechanism, the node feature update formula can be expressed as:

[0185]

[0186] in, is the feature representation vector of node v at layer l+1; σ is a nonlinear activation function; u represents the index of the neighbor node; N(v) represents the neighbor set of node v; is the attention weight, which indicates the importance between node v and its neighbor node u; W (l) is a parameter matrix; is the feature representation vector of node u in layer l.

[0187] Step 4, comprehensive representation:

[0188] Combining the above parts, the overall representation of the pyramid attention mechanism and GNN is shown in Equation 13:

[0189]

[0190] Among them, H (l+1) represents the node vector of the l+1th layer; σ is a nonlinear activation function; u represents the index of the neighbor node; N(v) represents the neighbor set of node v; is the attention weight, which indicates the importance between node v and its neighbor node u; W (l) is a parameter matrix; is the feature representation vector of node u in layer l; Fusion represents the feature splicing operation; is the feature representation vector of neighbor node u in layer l; P (l) Represents the multi-scale pooling output obtained at layer l.

[0191] The pyramid attention mechanism enables the model to more effectively capture long-range dependencies and complex structural information when processing graph-structured data, enabling more accurate node feature updates and relationship predictions. This mechanism provides a powerful tool for researching updates in information flow, improving the performance of graph neural networks on a variety of tasks.

[0192] 4. Fault relationship prediction based on node characteristics:

[0193] Step 1, node feature representation:

[0194] The node vectors after the message transmission are aggregated to obtain the vector representation between nodes, which is the feature of the relationship. This paper uses two aggregation methods: maximum and mean. The maximum operation can preserve the features of the important parts of the relationship prediction, while the mean operation corresponds to the overall characteristics of the features.

[0195] Specifically as shown in Formula 14, the characteristics of the relationship between entities are finally obtained.

[0196] R(node i ,node j )=concat(max(node i ,node j ),mean(node i ,node j )) (14)

[0197] Among them, R(node i ,node j ) represents the relationship feature between node i and node j; node i A vector representing node i; node j Represents the vector of node j; concat represents the concatenation function; max and mean represent the averaging function.

[0198] Step 2, relationship prediction:

[0199] After obtaining the vectors between nodes, we map them to the number of relationship types through a linear layer to perform relationship prediction. Specifically, it is expressed as:

[0200] r ij =softmax(W R ·R(node i ,node j )+b R ) (15)

[0201] Among them, r ij Represents the relationship classification between node i and node j; softmax represents the softmax function, which is used to convert the input vector into a probability distribution; W R and b R Represents the parameter matrix and bias vector of the linear layer respectively; R(node i ,node j ) represents the relationship feature between node i and node j.

[0202] Step 3, loss function:

[0203] In the output of graph neural networks, the imbalance between positive and negative samples makes it difficult for the model to effectively learn the features of a small number of positive samples during training. Therefore, we use the Focal Loss function to address this problem. Focal Loss is a weighted cross-entropy loss function designed to focus more on difficult-to-classify samples (positive samples) rather than easily classified negative samples when dealing with class imbalance. It is shown in the following formula:

[0204]

[0205] Among them, Focal Loss refers to the value of the cross entropy loss function; is the weight factor that controls the balance of positive and negative samples; p t is the probability of positive samples predicted by the model; β is a regulating factor used to reduce the loss contribution of those easy-to-classify samples; log(p t ) is part of the cross entropy loss.

[0206] By adjusting Focal Loss, the model can pay more attention to samples that are difficult to classify, thereby improving the learning effect of a small number of positive samples.

[0207] By implementing the above steps, we can effectively learn the relationship features between nodes and focus on difficult-to-classify samples during model training, thereby improving the learning effect of positive samples.

[0208] Based on the entity encoding layer that integrates the large language model GPT and entity type labels; the random walk subgraph construction method considering the impact of faults; the graph message passing layer based on the pyramid attention mechanism; and the corresponding description of fault relationship prediction based on node features, the device fault text implicit relationship prediction method based on the graph structure learning model is as follows Figure 1 As shown, the specific steps include:

[0209] Step 1: Deep text encoding: Use a large language model (LLM) to encode the text and, combined with entity type labels, convert the entities into word vectors. Use position indexes to map entities mentioned in the text to graph structures, enhancing the understanding of contextual information and entity features. Finally, perform deep encoding on the fault text to construct a graph structure for the entities.

[0210] During the encoding process, this embodiment not only fully considers the contextual information of the text, but also adds the type characteristics of the entity, converting the entity into a corresponding word vector representation. These word vectors are then modeled as a graph structure, further capturing the complex relationships and global information in the text, laying a solid foundation for subsequent deep feature extraction and relationship modeling. This process effectively enhances the model's ability to understand the interaction between context and entities, and improves the effectiveness of global information integration.

[0211] Step 2: Graph structure modeling and random walk subgraph sampling: The entities and relationships in step 1 are modeled as undirected graphs, and an improved random walk mechanism is introduced to account for the impact of node failures. Dynamic weight setting is used to reduce redundant information and extract key relationship information, constructing multiple compact subgraph structures, thereby enhancing the diverse expression of relationships between entities and the generalization ability of the model. Finally, representative subgraph structures are extracted from the fully connected graph.

[0212] Specifically, based on the entities and relationships obtained in step 1, all entities are modeled as a fully connected undirected graph. This embodiment introduces an improved random walk mechanism in the graph neural network GNN, considers the influence of node failures, and sets dynamic weights based on importance or features for different nodes to capture the local graph structure and the relationship between nodes. A subgraph sampling mechanism for random walks considering dynamic weights is designed, such as Figure 3 As shown in the figure, reducing redundant edges and extracting key relationship information can construct multiple subgraph structures reflecting entity relationships. This not only enhances the diversity of inter-entity relationships but also effectively improves the model's generalization ability. This mechanism enables the model to more comprehensively capture the potential relationships between entities when processing complex text, thereby improving overall prediction performance.

[0213] Step 3: Feature Aggregation and Relationship Prediction: Combined with the pyramid attention mechanism, the subgraph structure information is converted into a global entity vector representation through message passing. The pyramid attention mechanism can capture information at different levels, ensuring that all possible dependencies are considered when generating the global entity vector, thereby enhancing the model's ability to handle long-distance dependencies. Finally, entity features are aggregated in the subgraph and relationship prediction is performed.

[0214] This mechanism effectively transforms subgraph structural information into a global entity vector representation. The pyramid attention mechanism captures information at different levels, ensuring that all possible dependencies are fully considered when generating the global entity vector, regardless of how far apart they are in the context. This not only strengthens the model's integration of local and global information, but also improves its accuracy and robustness when handling long-range dependencies.

[0215] Step 4: Fault relationship prediction and classification: The generated entity vectors are used through the decoding layer to perform relationship prediction to improve the accuracy and stability of relationship prediction. This method combines global representation and subgraph attention to enhance the model's reasoning ability and adaptability when processing complex text data. The global entity vectors are mapped to specific relationship classifications.

[0216] By predicting the fault relationship based on node features, the generated entity vector is mapped to a specific relationship category, which is finally used for entity relationship prediction in complex text. Through the combination of global representation and subgraph attention, the accuracy and reliability of relationship prediction are significantly improved when dealing with complex and diverse text data. This method not only improves the model's reasoning ability for complex relationships, but also enhances its adaptability and generalization in different application scenarios. Through the above steps, the problem of existing relationship prediction methods often ignoring long-distance context association when dealing with long text, which makes it difficult to capture global dependency information, and the problem of low relationship prediction accuracy and insufficient generalization ability caused by the lack of capturing complex relationship diversity and cross-domain dependency are solved.

[0217] To verify the effectiveness of the device fault text implicit relationship prediction method based on the graph structure learning model, the following prediction verification is also performed:

[0218] The experiment uses power grid equipment operation and maintenance fault text dataset to predict the relationship. It contains a total of 4094 entities, 8 types of relationships, and 4414, 2000, and 2000 triplets in the training, validation, and test sets, respectively.

[0219] The evaluation indicators used include: Mean Reciprocal Rank (MRR) and hit@N. Among them, MRR (Mean Reciprocal Rank) is mainly used to measure the ranking quality of the prediction results, reflecting the position of the first correct answer in the prediction list; hit@N is used to evaluate whether the correct answer is included in the top N results. Through the above indicators, the performance of the device fault text implicit relationship prediction method based on the graph structure learning model can be comprehensively evaluated, ensuring its high accuracy and practicality in actual application.

[0220] To verify the effectiveness of the proposed model, comparative experiments were conducted on the dataset with existing mainstream models, including TranSE, TranSR, TranSH, and PRA. The MRR, hit@1, and hit@10 indicators were compared. The comparative experiment is shown in Table 1. The proposed model has obvious improvement in relationship prediction compared with other models, with an ordering quality of 0.683. The hit@1 and hit@10 indicators are not much different from the PRA model, but compared with other models, it still has a strong advantage.

[0221] Table 1 Comparison experiment on WN18 dataset

[0222]

[0223] In order to measure the contribution of different modules to the proposed model, ablation experiments were conducted on the depth-first, breadth-first, and pyramid attention mechanism modules to verify the improvement of model performance by each structure. The impact of different modules on the model is shown in Table 2. Among them, the depth-first and breadth-first modules effectively improved the hit@1 and hit@10 indicator performance. The final MRR of the model was 0.683, Hit@1 was 80.5%, and Hit@10 was 91.5%. It can be seen that the model can effectively predict relationships.

[0224] Table 2 The impact of different modules on the model

[0225]

[0226] Experimental analysis of subgraph relationship prediction:

[0227] In order to further verify the effectiveness of the proposed subgraph sampling mechanism, the length of the path between any nodes in the subgraph is set to find the impact of this parameter on relationship prediction. Figure 4 The left graph shows the impact of different α parameters on the model, the middle graph shows the impact of the Hit@1 indicator on the model, and the right graph shows the impact of the Hit@10 indicator on the model. Figure 4 As can be seen from the figure, when the longest path length of the subgraph is 64, the model achieves the optimal value.

[0228] Because subgraphs are constructed using two random methods—breadth-first search (BFS) and depth-first search (DFS)—with a probability of 0.5, respectively, to fairly expand paths in both directions, the parameter for sibling nodes is also 64 under these parameters. Whether looking at MRR, Hit@1, or Hit@10, we can see that model performance first increases and then decreases with increasing parameters. This demonstrates that when constructing subgraphs using graph neural networks, more nodes are not necessarily better; sometimes, more nodes can introduce noise into entity vectors.

[0229] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for predicting implicit relationships in equipment fault text based on a graph structure learning model, characterized by: This method for predicting implicit relationships in equipment fault text based on a graph structure learning model includes: an entity encoding layer that integrates a large language model (GPT) and entity type labels; a random walk subgraph construction method that considers fault impact; a graph message passing layer based on a pyramid attention mechanism; and fault relationship prediction based on node features. This graph-structured learning model-based method for predicting implicit relationships in equipment fault text is based on a text dataset of power grid equipment fault maintenance. By modeling the entities and relationships in the equipment fault text as a graph structure, the method combines the entity vectors generated by the large model with a graph neural network for message passing and relationship inference. The graph-structured learning model is used to predict unknown fault relationships between different devices. Finally, a classifier is used to predict unknown relationships between entities. The method for predicting implicit relations in equipment fault text based on a graph structure learning model includes the following steps: Step 1: Deep text encoding: Use a large language model (LLM) to encode the text and, combined with entity type labels, convert the entities into word vectors. Entities mentioned in the text are mapped to graph structures using position indexes. Finally, deep encoding is performed on the fault text to construct the entity graph structure. Step 2: Graph Structure Modeling and Random Walk Subgraph Sampling: The entities and relationships in Step 1 are modeled as undirected graphs, and an improved random walk mechanism is introduced to account for the impact of node failures. Dynamic weighting is performed to reduce redundant information and extract key relationship information, constructing multiple compact subgraph structures. Finally, representative subgraph structures are extracted from the fully connected graph. Step 3: Feature aggregation and relationship prediction: Combined with the pyramid attention mechanism, the subgraph structure information is converted into a global entity vector representation through message passing. Finally, entity features are aggregated in the subgraph and relationship prediction is performed. Step 4: Fault relationship prediction and classification: Perform relationship prediction on the generated entity vectors through the decoding layer; map the global entity vectors to specific relationship classifications; The entity encoding layer that integrates the large language model GPT and entity type labels includes the following steps: Step 1, word embedding of entity type: When processing text, the entity type label is embedded on both sides of the entity name; for the text after the above processing, the digital representation of the text is extracted through the dictionary in the pre-trained model, that is, for {x1,x2,...,x n }, use the following formula to generate the corresponding {n1,n2,...,n n }∈R represents; Embedding(x i )=E(x i ) Among them, x i Represents each word; the result generated by the Embedding function is {n1,n2,...,n n }∈R; Step 2, position encoding: Introducing positional encoding to preserve word order; The trigonometric function position encoding is used, as shown in the following formula: Among them, PE represents the encoding function; pos represents the position information of the word in the text; i represents the sequence information; d represents the generation dimension of the model vector; Step 3, multi-head self-attention mechanism: The key component of Transformer is the multi-head self-attention mechanism, which mainly calculates the attention scores of different heads and multiplies them with the value matrix to output representation; The output vector formula of the i-th head is: Where T represents the transpose operation; d k Indicates the length of the sequence; after obtaining the attention of multiple independent heads, the final output vector is concatenated after the linear transformation of the parameters; Finally, the outputs of all heads are concatenated through linear transformation and the comprehensive representation is: A=concat(head1,head2,...,head n )W+b Where A represents the vector matrix; concat represents the concatenation of vectors, head represents different heads, W and b represent the parameter matrix and bias vector in the linear transformation respectively; Step 4: Construction of graph structure: Construct a graph structure G = (V, E), where the edge set E is represented by an upper triangular matrix with a default value of 1, and the graph forms a fully connected undirected graph with entities as nodes; The initial vector for each node is: node i =mean(get_entity(A,start,end)) Among them, node i ∈H (0) , represents the initial vector of a node; mean represents the average operation on the vector; get_entity represents the function of extracting the entity vector from the vector matrix; entity i A vector representing entity i; start and end are node indices of the entity in the sentence; Through the above steps, a coding layer that combines entity types and effectively processes sequence data is constructed; The random walk subgraph construction method considering the impact of faults includes the following steps: Step 1: Graph model construction: Using the random walk-based subgraph sampling mechanism, the graph G = (V, E) is sampled into subgraph G * =(V * , E * ), where V * ∈V,E * ∈E; Step 2, random walk strategy: The walking strategy is expressed as the following formula: Among them, P(path) represents the probability of a specific path; L is the length of the wandering path; d(v i ) represents the degree of node i; v i is the node at step i; d is the degree function of the node; Step 3: Dynamically adjust the roaming strategy: By introducing the fault influencing factor, the number of steps and paths of the walk are dynamically adjusted according to the local structural information of the graph: First, we use breadth-first and depth-first strategies to capture horizontal and depth locality information respectively. We use the head entity as the first node of the graph and the rest as expansion nodes to achieve the purpose of subgraph sampling. The wandering strategy is adjusted by introducing two transition probabilities, as shown below: Among them, P(v i+1 |v i ,v i-1 ) means that when the current node v is known i and the previous node v i-1 In the case of i+1 ;d(v i ) is the node v i degree; p base is the basic probability, p is the return parameter; q base is another basic probability, and q is the breadth-first parameter of the walk; Step 4: Fault influencing factors: Fault impact FI(v i ) is defined as: Among them, d(v i ) is the node v i Degree; D max The maximum degree of all nodes in the graph; Consider the impact of the fault and adjust the parameter p accordingly base and q base , p base and q base The calculation formula is as follows: q base =q base ·(1+FI(v i )) Finally, p base and q base Substitute the fault impact FI(v i ), and obtain the random walk method considering the impact of faults; The graph message passing layer based on the pyramid attention mechanism includes the following steps: Step 1: Node message collection and attention weight calculation: At each level of the pyramid, node i collects messages from its neighbor node set N and applies the attention mechanism to calculate the weighted feature vector; The attention weights between nodes are expressed as follows: in, Indicates the attention weight of node i to node j in layer l; exp represents the exponential function; LeakyReLU represents the activation function; α is a learnable vector; T represents the transposition operation; W (l) is a parameter matrix; Represents the initial embedding vector of node i, i∈v; || represents the splicing between features; represents the feature representation vector of node j in layer l; k is the index of the neighbor node; N(i) represents the set of neighbor nodes of node i; Represents the feature representation vector of node k in layer l; Step 2, feature aggregation and pooling: For the extraction of feature information of each layer, three pooling operations of different scales are used to obtain feature information. The multi-scale pooling output P is expressed as follows: (l) ; P (l) =Pool(H (l) ) Among them, P (l) Represents the multi-scale pooling output obtained at layer l; Pool is a function of pooling operation; H (l) is the feature representation matrix at layer l; Step 3, node feature update: In GNN, the information transmission of each node is completed through the aggregation of neighboring nodes; after combining the attention mechanism, the node feature update formula is expressed as: in, is the feature representation vector of node v at layer l+1; σ is a nonlinear activation function; u represents the index of the neighbor node; N(v) represents the set of neighbors of node v; is the attention weight, which indicates the importance between node v and its neighbor node u; W (l) is a parameter matrix; is the feature representation vector of node u in layer l; Step 4, comprehensive representation: Combining the above parts, the overall representation of the pyramid attention mechanism and GNN is shown in the following formula: Among them, H (l+1) Represents the l+1th layer node vector; Fusion represents the feature splicing operation; is the feature representation vector of neighbor node u in layer l; P (l) represents the multi-scale pooling output obtained at layer l; Fault relationship prediction based on node features includes the following steps: Step 1, node feature representation: Using the message passing mechanism, the feature vectors of node i and node j are aggregated to generate relationship features; Aggregation methods include maximum and mean operations, and the formula is as follows: R(node i ,node j )=concat(max(node i ,node j ),mean(node i ,node j )) Among them, R(node i ,node j ) represents the relationship feature between node i and node j; node i A vector representing node i; node j represents the vector of node j; concat represents the concatenation function; max and mean represent the mean function; Step 2, relationship prediction: After generating the relationship features between nodes, they are mapped to the number of relationship types through a linear layer for relationship prediction. The formula is: r ij =softmax(W R ·R(node i ,node j )+b R ) Among them, r ij Represents the relationship classification between node i and node j; softmax represents the softmax function, which is used to convert the input vector into a probability distribution; W R and b R Represent the parameter matrix and bias vector of the linear layer respectively; Step 3, loss function: The FocalLoss function is used to improve the model's attention to a small number of positive samples; The definition of FocalLoss is as follows: Among them, FocalLoss refers to the value of the cross entropy loss function; is the weight factor that controls the balance of positive and negative samples; p t is the probability of positive samples predicted by the model; β is a regulating factor; log(p t ) is part of the cross entropy loss.

2. The method for predicting implicit relationships in equipment failure text based on a graph structure learning model according to claim 1, characterized in that: The text dataset obtained during the power grid equipment fault inspection and maintenance contains information from multiple aspects, including: Equipment alarm: A warning signal issued by the equipment, indicating a fault or abnormal state; Fault phenomenon: The specific fault manifestation is: abnormal working state of the equipment or error information; Fault cause: the cause of the fault; Action: The repair or maintenance measures taken in response to the fault.

3. The method for predicting implicit relationships in equipment failure text based on a graph structure learning model according to claim 1 is characterized by: The relationships in the equipment failure text are different types of connections between entities in the power grid equipment failure text; the relationships include: Solution: measures on how to deal with the fault; Cause: the cause or source of a fault; Includes: Parts or related equipment included in a certain equipment; Location: The specific location where the fault occurred or the relevant geographical location; Impact: The extent to which the fault affects the system or other equipment.

Images