A power transformation maintenance operation knowledge graph construction method and device and computer equipment

Abstract

This application relates to the field of power data processing technology, and discloses a method, apparatus, and computer equipment for constructing a knowledge graph for substation maintenance operations. The method includes acquiring a training dataset of substation maintenance-related knowledge; training an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model; wherein, after the normalization unit in the attention layer, the knowledge graph model introduces a rotational position encoding unit for optimizing the length of maintenance procedures and a grouped query attention mechanism for several query headers to share the same set of key-value headers; after the normalization unit in the feedforward network sublayer, a gated linear unit activation function is introduced to dynamically control the information transmission ratio according to the input content; the substation maintenance language model is used to perform semantic mining of substation maintenance operations, outputting a candidate entity list; based on the candidate entity list, nodes and edges in the graph database are mapped to generate a knowledge graph. This application has the effect of improving the model's inference efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of power data processing technology, and in particular to a method, apparatus and computer equipment for constructing a knowledge graph of substation maintenance operations. Background Technology

[0002] As a critical infrastructure of modern society, the safe and stable operation of the power system is of paramount importance. The daily operation and maintenance of the power system generates and relies on a massive amount of documentation, including historical equipment maintenance records, maintenance manuals, and industry technical specifications. This knowledge exists in various formats, including structured, unstructured, and semi-structured texts, with complex terminology.

[0003] In existing technologies, Large Language Models (LLMs) have demonstrated enormous application potential in the power system field due to their powerful contextual understanding and text generation capabilities, covering multiple aspects such as anomaly detection, predictive maintenance, intelligent alarm analysis, and knowledge management. However, in traditional methods for constructing knowledge graphs for substation maintenance operations, model training and graph construction are disconnected. When faced with the proliferation of non-standard terminology in massive amounts of documentation, the model lacks the ability to quickly and accurately understand non-standard terms.

[0004] Regarding the aforementioned technologies, the inventors have discovered that existing methods for constructing knowledge graphs for substation maintenance operations suffer from low model reasoning efficiency and insufficient accuracy. Summary of the Invention

[0005] To improve the model reasoning efficiency and accuracy of substation maintenance operation knowledge graph, this application provides a method, apparatus, and computer equipment for constructing a substation maintenance operation knowledge graph.

[0006] Firstly, this application provides a method for constructing a knowledge graph for substation maintenance operations.

[0007] This application is achieved through the following technical solution: A method for constructing a knowledge graph for substation maintenance operations includes the following steps: Obtain a training dataset for knowledge related to substation maintenance; Based on the training dataset, an improved knowledge graph model is trained to obtain a substation maintenance language model. The knowledge graph model is based on the Decoder-only Transformer model. After the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of maintenance procedures and a group query attention mechanism for several query heads to share the same set of key-value heads are introduced. After the normalization unit of the feedforward network sublayer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced. The substation maintenance language model is used to perform semantic mining of substation maintenance operations and output a candidate entity list. Based on the candidate entity list, nodes and edges in the graph database are mapped to generate a knowledge graph.

[0008] In a preferred example, this application can be further configured as follows: the training step of training the improved knowledge graph model based on the training dataset to obtain the substation maintenance language model includes, Obtain the unlabeled text block dataset from the training dataset; A domain-adaptive pre-training strategy based on distillation is used to perform causal language modeling tasks. This involves inputting an unlabeled text block dataset into the knowledge graph model and outputting the probability distribution of the text. The knowledge graph model is constrained by KL divergence to make its prediction distribution fit the prediction probability distribution of the pre-trained teacher model for the context, thus training the first-stage model.

[0009] In a preferred example, this application can be further configured such that the training step of training the improved knowledge graph model based on the training dataset to obtain the substation maintenance language model also includes, Obtain the untested data in the training dataset and the confidence inference logic of the teacher model; A large-scale instruction distillation strategy based on unverified data enables the knowledge graph model to mimic the confidence reasoning logic of the teacher model. The unverified data is input into the first-stage model, and the probability distribution is output to train the second-stage model.

[0010] In a preferred example, this application can be further configured such that the training step of training the improved knowledge graph model based on the training dataset to obtain the substation maintenance language model also includes, Obtain the preference dataset from the training dataset; Based on the direct preference optimization strategy using preference data, the full-parameter model is fine-tuned, and the preference dataset is input into the second-stage model to train the substation maintenance language model.

[0011] In a preferred embodiment, this application can be further configured such that, after the step of performing semantic mining of substation maintenance operations using the substation maintenance language model and outputting a candidate entity list, it also includes, The post-processing operation of introducing regularization rules into the candidate entity list includes correcting characters that do not conform to the power writing standard, removing invalid extractions that do not conform to the entity definition, and adopting the longest form retention principle for similar entities.

[0012] In a preferred embodiment, this application can be further configured such that the step of generating a knowledge graph by mapping nodes and edges in the graph database based on the candidate entity list includes: Map the candidate entity list to nodes and edges in the graph database; By implementing a source anchoring mechanism, the original text fragments corresponding to the input data, the model's inference confidence score, and the model version number are labeled on each relation edge to generate a knowledge graph.

[0013] In a preferred embodiment, this application can be further configured such that the structure of the knowledge graph model includes, An embedding layer is used to receive the input sequence formed by the training dataset; Several decoding layers are connected in series, and each decoding layer is used to decode the input sequence; each decoding layer includes an attention layer and a feedforward network sub-layer, and the output of the attention layer and the output of the feedforward network sub-layer are connected through a residual; The normalization layer is used to normalize the variance of the input vector; The output layer maps the high-dimensional hidden state vectors to the vocabulary dimension and transforms them into a probability distribution output.

[0014] Secondly, this application provides a device for constructing a knowledge graph for substation maintenance operations.

[0015] This application is achieved through the following technical solution: A knowledge graph construction device for substation maintenance operations, comprising, The dataset module is used to build training datasets for knowledge related to substation maintenance; The modeling module is used to train an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model. The knowledge graph model is based on the Decoder-only Transformer model. After the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of the maintenance procedure and a group query attention mechanism for several query heads to share the same set of key-value heads are introduced. After the normalization unit of the feedforward network sublayer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced. The semantic mining module is used to perform semantic mining of substation maintenance operations using the substation maintenance language model and output a candidate entity list. The knowledge graph module is used to generate a knowledge graph by mapping nodes and edges in the graph database based on the candidate entity list.

[0016] Thirdly, this application provides a computer device.

[0017] This application is achieved through the following technical solution: A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods for constructing a knowledge graph for substation maintenance operations.

[0018] Fourthly, this application provides a computer-readable storage medium.

[0019] This application is achieved through the following technical solution: A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for constructing a knowledge graph for substation maintenance operations.

[0020] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following: A training dataset of knowledge related to substation maintenance is obtained; based on the training dataset, an improved knowledge graph model is trained to obtain a substation maintenance language model; wherein, the knowledge graph model is decoder-only. Based on the Transformer model, its lightweight nature demonstrates powerful performance in generative tasks, meeting the needs of fast inference at the edge. A rotational position encoding unit is introduced after the normalization unit in the attention layer to optimize the length of maintenance procedures, enabling the handling of extremely long procedures and enhancing the model's applicability in substation maintenance. A grouped query attention mechanism is introduced for multiple query heads to share the same set of key-value headers, reducing memory usage during inference and improving throughput for long documents, significantly increasing inference speed while maintaining performance close to multi-head attention. A gated linear unit activation function is introduced after the normalization unit in the feedforward network sublayer to dynamically control the information transmission ratio based on the input content, enhancing the model's nonlinear expressive power. This results in better numerical stability and convergence speed during large model training, enabling more efficient capture of complex semantic features in power industry texts. A substation maintenance language model is used for semantic mining of substation maintenance operations, outputting a candidate entity list. Based on the candidate entity list, nodes and edges in the graph database are mapped to generate a knowledge graph, thereby improving the model's inference efficiency and accuracy in the substation maintenance operation knowledge graph. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the main process of a method for constructing a knowledge graph for substation maintenance operations, provided as an exemplary embodiment of this application.

[0022] Figure 2 A flowchart illustrating the model training process for a method of constructing a knowledge graph for substation maintenance operations, provided as an exemplary embodiment of this application.

[0023] Figure 3 This is a model architecture diagram of a method for constructing a knowledge graph for substation maintenance operations, provided as an exemplary embodiment of this application.

[0024] Figure 4 The following is an overall flowchart of a method for constructing a knowledge graph for substation maintenance operations, provided as an exemplary embodiment of this application.

[0025] Figure 5 This is a structural block diagram of a substation maintenance operation knowledge graph construction device provided as an exemplary embodiment of this application. Detailed Implementation

[0026] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0029] How to efficiently and accurately extract and utilize valuable information from scattered substation maintenance documents is a direction for continuous optimization to improve power grid operation efficiency and knowledge management level.

[0030] In entity extraction, traditional technical approaches mainly include: 1) Rule-based and dictionary-based methods, which match entities by manually writing regular expressions and building domain dictionaries. These methods have high accuracy in specific scenarios but poor generalization ability and high maintenance costs. 2) Statistical machine learning-based methods, such as Hidden Markov Models (HMMs) and Conditional Random Fields (CRFs), which learn statistical patterns in data for sequence labeling, improving automation to some extent. 3) Deep learning-based methods, represented by the BERT-BiLSTM-CRF model, which capture contextual semantics through pre-trained language models (such as BERT) and combine them with Bidirectional Long Short-Term Memory (BiLSTM) networks and CRF layers for entity label prediction, achieving good results on many standard datasets. However, these technical approaches share the commonality that their internal decision-making processes are not transparent to users, making them "black box" discrimination methods. They can identify entities but cannot explain the specific reasons for making that judgment.

[0031] In relation extraction (RE), the task is to identify semantic relationships between entities in text. Traditional methods typically employ a "pipeline" processing flow: first, a separate Named Entity Recognition (NER) model is run, and then the identified entity pairs are classified into relationships. The drawback of this approach is that errors generated in the NER step propagate to the relation extraction step, affecting the final result. Specific techniques include: 1) Pattern- and rule-based methods, which rely on domain experts manually writing lexical or syntactic rules (such as dependency tree patterns) to match relationships. These methods have high accuracy but low recall and are difficult to transfer to new domains. 2) Supervised learning methods, which treat relation extraction as a classification task and use Support Vector Machines (SVM), CRF, or early neural networks (such as CNN and RNN) for training. The main bottleneck of these methods is the need for a large amount of high-quality manually labeled data, which is costly. 3) Distant supervision methods, which address the problem of scarce labeled data by automatically generating training data by aligning existing knowledge bases (such as Freebase) with large-scale unlabeled text. Its core assumption is that if a pair of entities has a certain relationship in the knowledge base, then all sentences containing both entities express that relationship. This assumption introduces a large amount of label noise, which is the main challenge facing this method.

[0032] In knowledge alignment (or entity alignment, ontology matching), traditional methods primarily aim to discover entities in different knowledge bases that point to the same real-world object. Their techniques include: 1) Lexical and string similarity-based methods, which match entities by calculating edit distances, Jaccard similarity, and other metrics based on entity names, tags, or descriptive text. 2) Structure-based methods, such as LogMap and PARIS systems, which align entities by analyzing the similarity of their neighborhood structures (e.g., parent-child relationships, attribute relationships) in their respective knowledge graphs, assuming that structurally similar entities are more likely to be equivalent. 3) Embedding representation-based methods, such as GCN-Align, which embed entities from different knowledge graphs into a unified vector space using graph neural networks (GNNs) and then discover aligned entities by calculating the distance or similarity between vectors. These traditional methods often rely on manually designed features or rules when dealing with complex heterogeneous knowledge and struggle to fully utilize the rich semantic information of entities.

[0033] In existing technologies, the General Language Model (General LLM) is widely used in the power system field for anomaly detection, predictive maintenance, intelligent alarm analysis, and knowledge management.

[0034] However, while General LLMs possess strong generalization capabilities, their massive parameter scale (typically in the hundreds of billions) makes them difficult to deploy privately at edge locations such as substations. Traditional lightweight models, while efficient inference, often lack deep logical reasoning capabilities. When dealing with the complex "equipment-component-state" logical chains in substation maintenance, they struggle to understand the physical relationships behind technical terms (such as "heavy gas" and "tap switch"), limiting extraction accuracy. Furthermore, generative models, inherently based on probabilistic prediction, are prone to creating the illusion of non-existent connections, such as fabricating non-existent equipment connections. When constructing rigorous power knowledge graphs, relying solely on the model's generative capabilities cannot guarantee that the output conforms to the physical topology constraints of the power system (such as voltage level matching) and the mutual exclusion of business logic, resulting in a graph riddled with logical paradoxes. Moreover, existing end-to-end extraction methods typically output triplet results directly, lacking documentation of the decision-making basis. When errors occur in the graph data, maintenance personnel cannot distinguish whether the source text description is unclear or the model itself is faulty. Is the problem due to inference errors or insufficient confidence? In the power sector, where safety requirements are extremely high, this "black box" graph, lacking source anchoring and unable to trace the inference process, greatly reduces its credibility and engineering practical value. Traditional graph construction is often a one-way "one-off deal," with model training and graph construction being separate. Faced with the proliferation of non-standard terminology (industry jargon) and long-tail errors in model inference, the system lacks a closed-loop mechanism to automatically capture errors and transform them into training data. This results in the graph failing to evolve with the accumulation of business data, leading to high maintenance costs and difficulty in continuously improving quality.

[0035] To address this, this application constructs a domain-specific lightweight generative model, designs a dedicated lightweight architecture, and adopts a three-stage, full-parameter learning strategy. Utilizing a general-purpose large model as the "teacher," a rigorous process of "pre-training distillation—large-scale instruction distillation—expert DPO fine-tuning" is employed to "compress" and transfer the high-order logical reasoning capabilities of the large model to the lightweight model. This allows the model to maintain extremely low computational consumption and adapt to edge deployment while possessing deeply internalized substation maintenance expertise and logical reasoning capabilities. This improves the model reasoning efficiency and accuracy of the substation maintenance operation knowledge graph, thereby transforming unstructured text within the domain into a structured knowledge graph (KG). The knowledge graph is presented in the form of nodes (entities) and edges (relationships), clearly expressing the concepts, entities, and their interrelationships within the domain.

[0036] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0037] Reference Figure 1This application provides a method for constructing a knowledge graph for substation maintenance operations. The main steps of the method are described below.

[0038] S1: Obtain a training dataset for substation maintenance-related knowledge; S2: Based on the training dataset, train the improved knowledge graph model to obtain the substation maintenance language model; wherein, the knowledge graph model is based on the Decoder-only Transformer model, and after the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of the maintenance procedure and a group query attention mechanism for several query heads to share the same set of key value heads are introduced; after the normalization unit of the feedforward network sublayer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced; S3: Use the aforementioned substation maintenance language model to perform semantic mining of substation maintenance operations and output a candidate entity list; S4: Based on the candidate entity list, map the nodes and edges in the graph database to generate a knowledge graph.

[0039] Reference Figure 2 Specifically, a high-quality corpus is constructed for training the full-parameter model. The datasets in the corpus are divided into three categories: unlabeled pre-trained datasets (domain-adaptive pre-training based on distillation), unproven instruction datasets (large-scale instruction distillation training based on unproven data), and preference datasets (direct preference optimization training based on preference data).

[0040] To construct three types of datasets, we first read relevant knowledge files in the field of substation maintenance, including multi-source documents, specifications, maintenance manuals, and maintenance records, to obtain a massive amount of unstructured documents. (Data source) include: 1) Maintenance manuals and technical specifications: usually in PDF or Word format, with a strict hierarchical structure; 2) Historical maintenance records: contain a large amount of non-standardized handwritten digital text or brief records; 3) Equipment ledger: Semi-structured Excel or CSV data.

[0041] The data source is cleaned using a rule-based layout analysis algorithm for regularization. For PDF documents, the page coordinate system is first defined. Page height is Width is The header and footer areas are grouped together. If text block Center satisfy or (in If the preset threshold is used, then it will be removed; for two-column layouts, based on Coordinates reorder the text stream to ensure semantic coherence, and finally regular expressions are used. Remove meaningless characters; the formula is as follows: (1) The matching patterns include consecutive whitespace \s+, garbled characters, and non-UTF-8 encoded characters.

[0042] Next, in order to train a dedicated model architecture and improve coding efficiency, this approach does not directly reuse the vocabulary of the general model. Instead, it enhances and simulates the cleaned data to expand and build a domain-specific vocabulary. For example, high-frequency, multi-character electrical terms such as "heavy gas", "tap switch", "sulfur hexafluoride", and "partial discharge" are forcibly merged into a single Token (named Token). This allows the model to treat professional concepts as atomic units rather than combinations of words, thereby indirectly expanding the model's context window capacity.

[0043] To adapt to the context window of large models, the cleaned long document needs to be... Segmented into sequence fragments This solution abandons the traditional fixed-length segmentation and adopts a dynamic segmentation algorithm that prioritizes semantic integrity to achieve dynamic semantic block division. The specific steps are as follows: First, divide the document into atomic sentence groups, using paragraph newline characters \n\n as the boundary. Then set the maximum number of tokens. and soft threshold and initialize the current block. Finally, traverse The formula is expressed as: (2) If the above formula is satisfied, then Otherwise, close. Start a new block .

[0044] For text that is not only long but also internally indivisible, an overlapping sliding window segmentation strategy is adopted. First, the window size is set. Step length Then the first A block is defined as a sub-interval of a text sequence, expressed as: (3) The overlapping area is To ensure the contextual integrity of cross-block entities, the pre-training corpus was finally completed. The setup.

[0045] To support subsequent large-scale instruction distillation training based on untested data and direct preference optimization fine-tuning training based on preference data, high-quality data containing the thought chain (CoT) reasoning process and teacher probability distribution is constructed. This scheme adopts a dual-track strategy of "general large model simulation + manual verification". A general large model with full or extremely large parameters is used as the teacher model to temporarily replace the dedicated domain model in the subsequent automated knowledge graph construction process. When the model generates tokens, the unnormalized probability vector (Logits) or Top-k probability distribution of each generation step is recorded simultaneously.

[0046] Subsequently, random sampling is performed from samples with medium or low confidence scores in the "unverified data." Power industry experts then intervene in the subsequent verification process to correct errors in the model's inference, such as correcting incorrect classification reasons and supplementing missing entities. This results in high-quality data, leading to a fully accurate "hard-label" dataset. This dataset ensures the absolute accuracy of domain knowledge, and based on this, a dataset is constructed to support preference-based optimization training. The data structure is in triplet form. (4) in For input instructions, These are the raw outputs generated during simulations of general-purpose large models. These outputs often contain hallucinations, logical fallacies, or formatting flaws, and are therefore rejected by review experts. This indicates the correct output after expert correction.

[0047] Through the above process, a corpus containing a dedicated vocabulary and a pre-training corpus was constructed. ), instruction data ( ) and preference data ( The complete training dataset ensures that model training can make full use of datasets at all levels, achieving a complete evolution from generalization ability to expert value alignment.

[0048] By acquiring the training dataset of the aforementioned substation maintenance-related knowledge, an improved knowledge graph model is trained to obtain a substation maintenance language model. The knowledge graph model is based on the Decoder-only Transformer model. After the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of maintenance procedures and a grouped query attention mechanism for several query headers to share the same set of key-value headers are introduced. After the normalization unit of the feedforward network sublayer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced.

[0049] In one embodiment, the structure of the knowledge graph model includes, An embedding layer is used to receive the input sequence formed by the training dataset; Several decoding layers are connected in series, and each decoding layer is used to decode the input sequence; each decoding layer includes an attention layer and a feedforward network sub-layer, and the output of the attention layer and the output of the feedforward network sub-layer are connected through a residual; The normalization layer is used to normalize the variance of the input vector; The output layer maps the high-dimensional hidden state vectors to the vocabulary dimension and transforms them into a probability distribution output.

[0050] Specifically, refer to Figure 3 Based on the Decoder-only Transformer model, a dedicated network architecture is designed to construct an improved knowledge graph model, including... The embedding layer receives the input sequence Token formed from the training dataset; The Decoder layer decodes the Token, and several Decoder layers are connected in series. The Decoder layer includes an Attention layer and a feedforward sub-layer. The input and output of the Attention layer are connected by a residual, the input and output of the feedforward sub-layer are connected by a residual, and the output of the Attention layer and the output of the feedforward sub-layer are connected by a residual. The attention layer includes serially connected normalization units RMS Norm, dynamically scaled rotation position encoding units, and a grouped query attention mechanism; The feedforward network sublayer consists of serially connected normalized units RMS Norm and gated linear unit activation functions Swiglu; The (final) normalization layer normalizes the variance of the output vectors of several cascaded decoding layers; The (linear) output layer maps the normalized high-dimensional hidden state vector to the vocabulary dimension and transforms it into a probability distribution, outputting the probability distribution.

[0051] like Figure 3 In this context, N represents the total number of decoding layers, and the model parameter count is set between 3B and 7B to meet the needs of fast inference at the edge.

[0052] The SwiGLU gated linear unit activation function is adopted instead of the traditional ReLU or GELU activation functions to improve the nonlinear expressive power of the model. This is because SwiGLU introduces a gating mechanism, allowing the model to dynamically control the proportion of information passed based on the input content. Compared to the hard truncation of ReLU, SwiGLU exhibits better numerical stability and convergence speed in large model training, and can more efficiently capture the complex semantic features in power-related texts. The calculation formula is as follows: (5) in Represents the input vector. Represents element-wise product. , The weight matrix is ​​a learnable matrix. The Swish activation function is defined as follows: ,in For the Sigmoid function, These are learnable parameters.

[0053] In addition, the model introduces Grouped-Query Attention (GQA). To reduce the GPU memory usage of the KV cache during inference and improve the throughput of long document processing, the GQA mechanism is adopted. By allowing multiple query heads to share the same set of key-value heads, the inference speed is significantly improved while maintaining model performance close to that of Multi-Head Attention (MHA). The calculation formula is as follows: (6) in These represent the query matrix, key matrix, and value matrix, respectively. The dimension of the key vector. This is used as a scaling factor to prevent the gradient from vanishing due to excessively large dot product values. Operation representation will Group key-value pair copy expansion for matching .

[0054] To handle excessively long maintenance procedures (exceeding the training length) Introducing dynamic scaling factor Optimize the rotation position encoding for position index. and dimensions The position vector is calculated as follows: (7) in For input vectors In the components of the current two-dimensional subspace, This enables the model to support Context window reasoning without loss of accuracy.

[0055] Furthermore, to improve the training stability of deep networks, the normalization unit of the attention layer and feedforward sub-layers is root mean square layer normalization (RMS Norm). RMS Norm is used for data normalization. Compared with the traditional layer normalm, RMS Norm removes the centering operation and only normalizes the variance of the input vector. This not only reduces the computational overhead and improves the training speed, but also maintains the numerical stability of gradients in deep Transformer architectures, preventing gradient explosion or vanishing.

[0056] Furthermore, to address the degradation problem in deep networks (N layers stacked), residual connections are introduced at the output of each sub-layer (attention layer and feedforward network layer). This involves adding the input and output of the sub-layer element-wise, allowing gradients to flow directly to the shallower network and ensuring that the model can effectively train deeper layers.

[0057] Furthermore, the model introduces a linear output layer and a probability distribution. After processing by N layers of decoders, the data is fed into the linear output layer (Linear Head) through a final normalization layer to map the high-dimensional hidden state vector to the vocabulary size. The model is then transformed into an output probability distribution through the Softmax function, thereby predicting the next most likely token.

[0058] Based on the Transformer Decoder-only architecture, a generative language model for lightweight substation maintenance is built by introducing the SwiGLU activation function, grouped query attention mechanism, and dynamic scaling rotation position encoding mechanism.

[0059] The training dataset is input into the improved knowledge graph model for learning, resulting in a lightweight substation maintenance language model.

[0060] To reproduce the reasoning capabilities of general large models with a limited parameter scale and ensure the accuracy of knowledge in the substation maintenance field, thereby obtaining a lightweight generative language model specifically for the substation maintenance vertical domain, this application conducts full-parameter phased training on the improved knowledge graph model. By designing a "Three-Stage Curriculum Learning" training strategy, which follows the evolution path of "unsupervised distribution alignment - supervised skill generalization - expert value alignment", and combining full-parameter training, the model feature space is deeply reshaped.

[0061] In one embodiment, the training step of training an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model includes: Obtain the unlabeled text block dataset from the training dataset; A domain-adaptive pre-training strategy based on distillation is used to perform causal language modeling tasks. This involves inputting an unlabeled text block dataset into the knowledge graph model and outputting the probability distribution of the text. The knowledge graph model is constrained by KL divergence to make its prediction distribution fit the prediction probability distribution of the pre-trained teacher model for the context, thus training the first-stage model.

[0062] Phase 1: Domain Adaptive Pre-training based on distillation. This phase serves as the "knowledge injection" period for the model, aiming to align the parameter distribution of the randomly initialized student model with the text distribution in the substation maintenance field, enabling it to grasp the semantic representation of professional terms and the long-range dependencies of technical procedures.

[0063] Reference Figure 2 The dataset used for model training in this stage is a domain text sequence, namely the unlabeled text block dataset mentioned above. During training, a fully parameterized general language model is called as the teacher model to generate the Logits distribution for the text sequence, such as DeepSeek, Baichuan, ChatGLM, etc., to accelerate the model's convergence speed.

[0064] During training, a strategy combining self-supervised learning and knowledge distillation is employed. The student model (i.e., the knowledge graph model constructed above) performs a causal language modeling (CLM) task, predicting the next token while simultaneously fitting the teacher model's prediction probability distribution for the context through KL divergence constraints. This ensures that the student model not only learns explicit textual knowledge but also inherits the teacher model's understanding of latent semantic associations within the domain terminology. In other words, the loss function is optimized to minimize a hybrid loss function. : (8) in To balance the weights of task loss and distillation loss, The loss model is applied to a self-supervised causal language to maximize the model's prediction probability for the next token in the substation maintenance document: (9) in, For the current token in the sequence, For context sequence, These are the full parameters of the student model.

[0065] To pre-train the distillation loss, the prediction distribution of the student model is constrained using Kullback-Leibler divergence to approximate the teacher model: (10) in, The probability distribution output by the teacher model. This is the probability distribution output by the student model. This constraint enables the student model to inherit the teacher model's understanding of the potential semantic associations of domain terms.

[0066] In one embodiment, the step of training an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model further includes, Obtain the untested data in the training dataset and the confidence inference logic of the teacher model; A large-scale instruction distillation strategy based on unverified data enables the knowledge graph model to mimic the confidence reasoning logic of the teacher model. The unverified data is input into the first-stage model, and the probability distribution is output to train the second-stage model.

[0067] Phase Two: Large-Scale Instruction Distillation Based on Unverified Data (SFT with Distillation on Unverified Data), see [reference] Figure 2This stage is the "skill acquisition" period of the model, which aims to quickly endow the model with the ability of thought chain (CoT) reasoning, entity extraction and strict JSON format compliance by using massive simulation data (collection of simulation data of general large model throughout the process).

[0068] Training uses the untested data constructed above ( The system includes the Teacher Logits, which retains the confidence inference logic of the teacher model. By forcing the student model to mimic the complete probability distribution output by the teacher model, the student model learns the "confidence pattern" of the teacher model when performing logical inference. This pattern contains richer information than the final text result, which helps improve the model's generalization ability when facing unseen samples. In other words, the loss function is optimized to minimize the mixture loss function. : (11) in Format and content for learning the basics: (12) in Text tokens generated for the teacher model. This is for inputting commands. This section is used to establish the basic JSON format specification and thought process structure.

[0069] To account for the losses from command distillation, a Softmax function with temperature scaling is used for distribution alignment. (13) in and These are the Logits vectors (unnormalized probabilities) output by the teacher model and the student model, respectively. It refers to the temperature coefficient.

[0070] In one embodiment, the step of training an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model further includes, Obtain the preference dataset from the training dataset; Based on the direct preference optimization strategy using preference data, the full-parameter model is fine-tuned, and the preference dataset is input into the second-stage model to train the substation maintenance language model.

[0071] The third stage is Direct Preference Optimization (DPO) based on preference data. This stage is the "value alignment and denoising" period of the model, which aims to solve the teacher illusion problem that the model may inherit in the second stage and to make its logical judgments fully aligned with the decision boundaries of domain experts.

[0072] Reference Figure 2 The input data uses the preference dataset created above. This dataset contains triples. At this stage, the DPO algorithm is applied directly to fine-tune the full-parameter model, and its loss function... Defined as: (14) in This is the student model that is currently being optimized; This is a copy of the model (Reference Model) after the second phase of training, with parameters frozen during the training process; This is the Sigmoid function. Hyperparameters used to control the degree of deviation.

[0073] The loss function at this stage is maximized. and The difference in log-likelihood ratio forces the model to increase the probability of generating expert-approved answers while decreasing the probability of generating incorrect answers. The implicit KL divergence constraint in the formula ensures that the model... It will not deviate drastically from the reference model This allows for the correction of logical errors while preserving the language fluency and JSON format standardization acquired in the second stage.

[0074] Through cascading training using the "Three-Stage Curriculum Learning" training strategy, the model completes a full evolution from "general knowledge imitation" to "skill generalization" and then to "expert alignment." The resulting lightweight model possesses both the logical reasoning and structured output capabilities of a general large model and deeply internalizes the domain rules of substation maintenance, providing a high-precision and highly reliable substation maintenance language model for subsequent construction of an automated knowledge graph.

[0075] To eliminate format noise and ensure the standardization of the model extraction results, in one embodiment, after the step of using the substation maintenance language model to perform semantic mining of substation maintenance operations and outputting a candidate entity list, the method further includes: The post-processing operation of introducing regularization rules into the candidate entity list includes correcting characters that do not conform to the power writing standard, removing invalid extractions that do not conform to the entity definition, and adopting the longest form retention principle for similar entities.

[0076] Reference Figure 4 Specifically, the hybrid-driven entity extraction method employs a hybrid strategy of "breadth-based model recall + precision correction by a rule engine." It utilizes a dedicated lightweight model with fully fine-tuned parameters for initial semantic mining to maximize entity recall. Subsequently, it employs regularization-based correction and filtering mechanisms, including: Construct a prompt with strict formatting constraints and extract all key technical elements from the text. The specific prompt design is as follows: "You are an entity identification expert in the field of substation maintenance. Please analyze the following work text and extract all key technical terms, covering: equipment name (e.g., 'No. 1 main transformer'), equipment components (e.g., 'high voltage side bushing'), actions (e.g., 'replacement'), phenomena (e.g., 'oil leakage'), and parameters (e.g., '10kV')."

[0077] Output requirements: Please do not omit any possible entities.

[0078] The output format must be a strict JSON string list, such as ["Entity 1", "Entity 2"].

[0079] Do not output any unnecessary explanatory text. Text content: <test>" By inputting each line of substation maintenance work text into the model according to the prompt word template, the original list generated by the model can be obtained. Furthermore, it is necessary to address potential boundary noise introduced by the model, such as excessive punctuation marks or non-standard formatting. Therefore, further adjustments are required. Regularization rules are introduced for post-processing, as follows: 1) Use regular expressions to fine-tune entities and correct characters that do not conform to the writing standards for electricity, such as removing residual punctuation marks at the end of entities and ensuring the format of numerical values ​​and units, etc. 2) Remove invalid extractions that clearly do not conform to the entity definition; removal length. Pure Chinese characters (unless they are in the power whitelist, such as "phase" or "gate"), and common process words such as "to proceed", "work", and "task" should be removed; 3) Given that a single maintenance text usually targets a specific piece of equipment and does not involve complex synonym conflicts, this step only addresses the inclusion redundancy that may arise during model extraction. Therefore, the longest form retention principle is adopted for similar entities, meaning that when two entities exist in the list... and ,and and substrings, for example "casing", "High-voltage side bushing" is considered to be It carries more complete semantic information, thus eliminating Only retain .

[0080] After the post-processing operations of the above regularization rules, the final list generated by the model is obtained. Post-processing yields a candidate entity list with a strictly formatted structure. This lays a high-quality data foundation for subsequent disambiguation and relationship building.

[0081] In one embodiment, an entity standardization method based on hybrid similarity and semantic reasoning addresses the common problem of "equipment terminology abbreviations" and "habitual abbreviations" in substation maintenance work texts, such as "main transformer" corresponding to "main transformer" and "grounding switch" corresponding to "grounding switch." A standardization process is constructed that prioritizes algorithm retrieval, uses model reasoning as a fallback, and aggregates long-tail samples. This method aims to map non-standard entity mentions to unique entities in a standard terminology database and automatically captures long-tail data to drive model iteration, improving the model's output accuracy. This includes: For most text abbreviations that conform to naming conventions, and those that typically retain core words or numerical identifiers, mathematical calculations that do not require large models are prioritized for matching to ensure the system's response speed and determinism. Specifically, a hybrid similarity function matching is used, first defining the extracted words. With standard words similarity between It is composed of a weighted average of literal overlap and edit distance: (15) in Represents a character-level N-gram set; Levenshtein Distance; , These are the weighting coefficients; calculate and The similarity score of all terms in the text is calculated, and the maximum value is taken. and its corresponding standard terms ; like (With a high confidence threshold, such as 0.85), the mapping relationship is directly determined to be valid. Such samples are not included in the manual verification list. For example, "1# Main Transformer" and "1# Main Transformer" have a very high similarity and can be mapped directly.

[0082] For entities with similarity below the threshold, it usually means there is a significant difference in abbreviations, such as "CT" referring to "current transformer" or there is ambiguity, such as "switch" referring to circuit breakers of different voltage levels in different contexts. In this case, the model is activated to perform contextual reasoning and trigger the "active learning" data collection mechanism. The Prompt design is as follows: "Determine entities based on text context" The full standard name of '.

[0083] Text content: <test>.

[0084] Standard library candidate words: .

[0085] If no match is found, please complete the full name using your electrical engineering knowledge. Each entity to be judged The standard name after model output inference is When the final entity standard name If an entity is not in the candidate word list, it is considered a "hard sample," and the system automatically encapsulates this processing into a record and stores it in the list awaiting manual verification. The record structure is as follows: { "source_text":" <test>", "mention":"e", "llm_inference":"s_pred", "candidates":"[s_cand1...]", "status":"pending_review" / / Pending expert confirmation } This data, after being verified by experts, will be directly transformed into high-quality DPO preference data, thereby significantly improving the model's ability to understand industry jargon in the next round of training.

[0086] Through the above two-layer architecture, this embodiment not only ensures efficient processing of common abbreviations through algorithms, but also establishes a data pipeline for continuous model learning by automatically capturing "difficult abbreviations", automatically capturing long-tail data, and realizing model self-iteration.

[0087] In one embodiment, a relation extraction method based on ontology constraint pruning and logical reasoning is employed to establish semantic connections between standardized entities. First, all extracted entities are paired to construct a candidate relation set. This is to ensure that no possible relationships are missed, but in order to address the quadratic explosion of computational complexity in pairwise pairing methods as the number of entities increases (…), To address the issue of large models generating logical violations and reduce the risk of such violations, this solution introduces an ontology constraint matrix (Schema Matrix) as a pre-pruning mechanism. Large model thought chain (CoT) reasoning is performed only on logically valid entity pairs, including: Before calling the large model, predefined domain ontology rules are used to filter out entity pairings that violate physical common sense or business logic. The ontology constraint matrix defines a binary matrix. ,in The total number of entity types (e.g., devices, parts, actions, parameters, etc.): (16) For example, if Represents "voltage level" If the voltage represents an "action", and the business logic stipulates that voltage cannot trigger an action, then... .

[0088] for Any two entities in Calculate the legitimacy mask that constitutes the potential relationship. : (17) Only entity pairs with a mask of 1 are retained to form a candidate relation pair set. : (18) in Representing entities The category index to which it belongs. This step expands the search space of the full permutation from... It is significantly compressed into a legal logical subspace, which greatly reduces the number of subsequent LLM calls.

[0089] For sets In this approach, deep semantic analysis is performed on pruned entity pairs using a fine-tuned dedicated model. The Prompt is designed as follows: "Analyze entity A in the text" 'With Entity B' The semantic relationship of '.

[0090] The relation type is selected from: [contains, connects, belongs to, acts on, has state, no relation].

[0091] Requirements: You must first quote a sentence from the original text to explain your reasoning, and then give your conclusion. The model outputs a thought chain and conclusion. If the model determines there is "no relationship," the chain is discarded. If a relationship exists, the model output is parsed into a standardized graph triple structure, and a preliminary confidence level is calculated. Triple construction follows: (19) in These are the terminology names standardized through step five. For predefined relation types.

[0092] In addition, to provide a reference for subsequent auditing, the confidence score of the triples is calculated based on the generation probability. : (20) in Output the token sequence corresponding to the relation label (such as "contains") for the model; For the model in generating the first The softmax probability for each token. This formula calculates the average log-likelihood probability of the relation labels generated. A higher score indicates that the model is more confident in its judgment of the relation.

[0093] In this embodiment, the system, while ensuring logical legality, utilizes a large model to perform deep semantic relation extraction and outputs a set of candidate triples with evidence support and confidence scores.

[0094] In one embodiment, an automated feedback closed-loop mechanism based on multi-dimensional logic auditing and confidence gating is introduced as a key link in achieving "white-box" quality control of the knowledge graph. Specifically, for the output candidate triples, an auditing module is constructed, consisting of hard logic rules (topology and mutual exclusion detection) and soft confidence gating (score threshold). This module is not only responsible for intercepting erroneous knowledge, but also for automatically capturing long-tail errors that the model failed to cover in step three of training, generating DPO preference data to drive model iteration. The specific steps are as follows: First, perform topological loop detection and mutual exclusion detection without relying on the model. Check if there are any relationships in the output triples that violate physical common sense or graph theory constraints. These will be directly marked as "logical errors". 1) Topological loop detection uses the Tarjan algorithm to detect whether any illegal loops have been introduced into the graph: (twenty one) in A binary indicator variable representing topology compliance, with a value of 1 indicating approval and 0 indicating rejection; It indicates the logical relation of "if and only if"; This represents the historical full graph structure stored in the graph database, including all verified nodes and edges; This represents the candidate triplet to be verified (considered as an edge); The union operation represents the graph structure, which adds a new edge to the old graph. This represents a loop detection function (implemented using Tarjan's strongly connected component algorithm or Depth-First Search (DFS)). If the newly constructed graph does not contain any illegal directed cycles (e.g., the logical paradox of "components contain devices"), the function returns True; otherwise, it returns False.

[0095] 2) Mutual exclusion detection checks whether there is a semantically mutually exclusive relationship between the same pair of entities (e.g., both "component" and "voltage level"). (twenty two) in They represent candidate triples respectively; This indicates an existing relationship between the two entities and the current relationship. A set of mutually exclusive relations. For example, if the database already contains... If "belongs to" and "contains" are defined as mutually exclusive, then "contains" belongs to the mutually exclusive set; This indicates a set membership determination, i.e., the current relation. It cannot belong to a known mutually exclusive set to prevent semantic conflicts.

[0096] Once a candidate triple passes the hard logic rule check, it will proceed to the soft confidence gating check, which is based on the confidence score output by the candidate triple. Perform, by setting a high threshold Such as 0.9, and low threshold For example, a value of 0.6 is used for judgment: 1) High confidence interval ( If the result is positive, it will enter the LLM counterfactual verification. If the verification passes, it will be automatically entered into the database. 2) Medium confidence interval ( If it is marked as "questionable", proceed directly to the next step. (Expert verification queue); 3) Low confidence interval ( If the sample is a negative sample, it is either discarded or treated as a potential negative sample.

[0097] And, a synthesis decision function, used to define the final validity decision function. : (twenty three) in Indicates candidate triples The final decision is a comprehensive Boolean statement determining whether automatic data entry is allowed. 1 indicates allow entry, 0 indicates block. Represents the characteristic function.

[0098] After the triplet passes the detection of the first two layers of independent models, in order to prevent the model from generating a high-confidence illusion, i.e., "talking nonsense with a straight face," a counterfactual verification mechanism is introduced as the last line of defense. This is done by constructing a reverse Prompt query on the large model: "If it doesn't matter" ',original <test>Does the description still hold true? If the model answers "true" or "does not affect," it indicates that the relationship may be an overinterpretation and should be removed. If the model answers "false" or "the original text explicitly mentions...", the validation passes.

[0099] Finally, for the triplet samples judged as erroneous for various reasons in the above process, an error capture and DPO negative sample automatic construction step is introduced. This transforms the audit results into the core of the closed loop of model capabilities. The system automatically classifies and constructs DPO triplets based on the reasons for audit rejection. They mainly fall into the following two categories: 1) The trigger condition for capturing logical error samples (Type I Error) is: It's very tall, but or Failure (the model has made a common-sense error with high certainty). Sample construction. (Negative samples) represent the original output of the model that includes this erroneous relation; (Positive samples) are the output after removing the erroneous relation (or the result automatically corrected by the schema rules). These samples have the highest training value for DPO. Firstly, the collection of class samples does not require manual intervention throughout the process. Both positive and negative samples have a designed workflow that automatically guides the model to complete self-correction. Therefore, these errors reflect the defects of the current model to a certain extent, thus effectively suppressing the model's "stubborn illusion".

[0100] 2) The trigger condition for capturing blurred samples at the boundary (Type II Error) is that the sample falls within the middle confidence interval. The samples were then manually reviewed and corrected by experts. Sample Construction (Negative samples) represent the model's original "hesitant" or erroneous outputs; (Positive samples) represent the correct atlas structure ultimately confirmed by experts. These samples can improve the model's ability to discriminate in complex contexts.

[0101] When the cumulative number of samples in both classes reaches a preset threshold, such as 1000, incremental DPO training of the model is triggered to achieve iterative upgrade of the model version.

[0102] Through this embodiment, the system not only ensures the logical consistency of the knowledge stored in the database, but more importantly, it uses the confidence score as a "probe" to accurately identify the cognitive blind spots of the model and transform them into valuable data for subsequent training, thereby realizing full-process automation and continuous evolution.

[0103] To enable dynamic backtracking and quality upgrades of graph data as model capabilities improve, and to meet the stringent requirements of substation maintenance work texts for data security and interpretability, this embodiment introduces a provenance anchoring mechanism. This ensures that every piece of knowledge in the graph can be traced back to its original documentation. In one embodiment, the step of generating a knowledge graph by mapping nodes and edges in the graph database based on the candidate entity list includes... Map the candidate entity list to nodes and edges in the graph database; By implementing a source anchoring mechanism, the original text fragments corresponding to the input data, the model's inference confidence score, and the model version number are labeled on each relation edge to generate a knowledge graph.

[0104] Specifically, based on the ontology constraint pruning and logical reasoning relationship extraction mechanism, structured data audited through multidimensional audits is mapped to nodes and edges in a graph database. Unlike traditional static graph construction, the knowledge graph applied to substation maintenance in this application supports "source traceability" and "full lifecycle management," including: Get the list of valid triples As a candidate entity list for the knowledge graph, i.e., the data source for the knowledge graph database, each data entry in the list is strictly defined as a quadruple: (twenty four) in The standardized full name of the entity, such as "Main Transformer No. 1"; Indicates extracted semantic relationships, such as "contains"; The metadata packet being carried includes: .

[0105] To handle the cumulative writing of massive amounts of historical data during graph construction, the system adopts a PropertyGraph model and executes an idempotent writing strategy, labeling the original text fragments, the model's inference confidence score, and the model version number on each relation edge to prevent duplicate data from causing graph expansion.

[0106] Introducing a node fusion strategy: using standardized entity names ( As the primary key, the node is uniquely identified and a MERGE operation is performed during writing: if a node with the same name already exists in the graph, a new node is not created, and only its attributes, such as the time of its most recent appearance, are updated; if it does not exist, a new node is created and the corresponding type label (such as Equipment, Defect) is assigned.

[0107] The strategy for constructing relationships between entities is to build directed edges between two nodes. However, in order to reflect the characteristics of "multi-source heterogeneity", when there are multiple edges of the same type between two nodes, they are merged into one edge but with multiple source attributes attached. For example, "No. 1 main transformer" is mentioned in both "2023 maintenance record" and "2024 defect report" and includes "high voltage side bushing". Therefore, it is merged into one edge but with two source attributes attached, such as "2023 maintenance record" and "2024 defect report", to support subsequent frequency analysis.

[0108] To meet the stringent requirements for data security and interpretability in substation maintenance work documents, a provenance anchoring mechanism was designed to construct metadata attribute mapping relationships, that is, to map the metadata attributes in the input data... The attributes are fully mapped to the edges in the graph database, rather than the attributes of the nodes. Specifically, the source_text attribute is set for the edges to indicate that the original text fragment is attached. When the user clicks on the connection edge in the graph, the system can directly display a pop-up window: "This relationship is extracted from '...replace the high voltage side bushing of the No. 1 main transformer...'". The confidence_score is set to attach the confidence score calculated in step seven.

[0109] When displaying the information on the front end, the thickness of the edges can be rendered based on the color intensity corresponding to the confidence score, intuitively showing the credibility of the knowledge. Additionally, the `model_version` attribute can be set to record the version number of the language model extracted during the process.

[0110] The knowledge graph constructed by this scheme is not a static library. Through three-stage full-parameter training of the model and DPO feedback, the graph, after its construction, not only can be updated based on the same data extraction method, but also has the ability to self-evolve. Specifically, when the background model completes training using collected DPO negative samples and releases a new version, the system automatically triggers a "low confidence backtracking scan". It searches the graph database for all relation edges with a confidence_score below a certain threshold, such as 0.75, and whose model_version attribute records the old version. At this time, the original text of these questionable relations (obtained from the source_text attribute) is input into the new model for re-inference. If the confidence given by the new model is significantly improved or overturns the old conclusion, the corresponding edges in the graph are corrected or deleted, and the version number attribute is updated.

[0111] Through the above mechanism, the proposed solution constructs not only a knowledge graph database that stores substation maintenance knowledge, but also a living knowledge system with a life cycle of "perception-reasoning-feedback-evolution", which solves the pain point of traditional knowledge graphs being difficult to maintain and update after construction.

[0112] In summary, a method for constructing a knowledge graph for substation maintenance operations involves acquiring a training dataset of substation maintenance-related knowledge; training an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model; wherein, the knowledge graph model is decoder-only. Based on the Transformer model, its lightweight nature demonstrates powerful performance in generative tasks, meeting the needs of fast inference at the edge. A rotational position encoding unit is introduced after the normalization unit in the attention layer to optimize the length of maintenance procedures, enabling the handling of extremely long procedures and enhancing the model's applicability in substation maintenance. A grouped query attention mechanism is introduced for multiple query heads to share the same set of key-value headers, reducing memory usage during inference and improving throughput for long documents, significantly increasing inference speed while maintaining performance close to multi-head attention. A gated linear unit activation function is introduced after the normalization unit in the feedforward network sublayer to dynamically control the information transmission ratio based on the input content, enhancing the model's nonlinear expressive power. This results in better numerical stability and convergence speed during large model training, enabling more efficient capture of complex semantic features in power industry texts. A substation maintenance language model is used for semantic mining of substation maintenance operations, outputting a candidate entity list. Based on the candidate entity list, nodes and edges in the graph database are mapped to generate a knowledge graph, thereby improving the model's inference efficiency and accuracy in the substation maintenance operation knowledge graph.

[0113] A method for constructing a knowledge graph for substation maintenance operations adopts a lightweight model full-parameter training strategy of "three-stage course learning," breaking the limitations of traditional knowledge extraction that relies solely on supervised fine-tuning (SFT). It innovatively designs a three-stage evolution path: "domain-adaptive pre-training—large-scale instruction distillation—direct preference optimization." Knowledge distillation is introduced in the pre-training and instruction fine-tuning stages, forcing the lightweight student model to fit the probability distribution of the teacher model, thereby inheriting the implicit "hidden knowledge" and generalization ability of the general model. In the final stage, distillation is abandoned in favor of DPO (Direct Point Optimization) to align expert decision boundaries. This progressive training paradigm effectively solves the problem of insufficient ability of small models to handle complex long-chain reasoning in substation maintenance. It "compresses" and transfers the high-order logical reasoning ability of large models to lightweight models, achieving high-performance logical reshaping with low resource consumption. This allows the model to maintain extremely low computational consumption for edge deployment while possessing deeply internalized substation maintenance professional knowledge and logical reasoning ability, resolving the contradiction between the high computational cost of general models and the lack of domain logic.

[0114] A knowledge graph construction method for substation maintenance operations introduces an "illusion" suppression mechanism combining ontology constraint pruning and multi-dimensional logical auditing. Addressing the logical uncontrollability issue in knowledge extraction from generative models, a rigorous logical gating system is constructed, including: Pre-pruning: Before model inference, a legality mask is calculated using a domain ontology matrix (Schema Matrix) to directly filter logically mutually exclusive entity pairs, severing impossible entity relationships at the logical level, compressing the solution space, suppressing illusions, significantly reducing computational complexity, and avoiding basic common sense errors; Post-auditing: An audit layer is designed, including topological loop detection, mutual exclusion verification logic rules, and soft confidence gating. Combined with LLM counterfactual verification, the algorithm forcibly intercepts generated results that violate graph theory constraints, ensuring absolute self-consistency of the knowledge stored in the database in both physical and business logic. This significantly outperforms existing technologies that solely rely on model generation capabilities, solving the problems of uncontrollable illusions and lack of logical consistency in generative models.

[0115] A new method for constructing a knowledge graph for substation maintenance operations introduces an automated feedback loop for automatic error capture and an incremental evolution mechanism for the graph. This changes the traditional static "construction-and-fixation" model of knowledge graphs. It innovatively designs a dynamic evolution architecture based on an automated feedback loop. By utilizing error samples and low-confidence samples intercepted in the logical auditing process, these samples are automatically transformed into winner-loser pairs required for Direct Preference Optimization (DPO). The system automatically classifies and constructs these winner-loser pairs, forming an incremental DPO preference dataset. This allows the model to iterate and upgrade itself using "long-tail errors" encountered during operation. Simultaneously, combined with a graph version awareness mechanism, when the model is upgraded, it automatically triggers a retrospective re-judgment of historically questionable data, achieving a self-evolutionary cycle of "automatic error capture—incremental model training—graph retrospective correction." This ensures the system can continuously adapt to non-standard terminology and complex operating conditions, solving the problems of static rigidity and lack of automated evolution mechanisms, and creating a virtuous cycle of spiraling improvement in model capabilities and graph quality.

[0116] A method for constructing a knowledge graph for substation maintenance operations introduces end-to-end white-box attribute mounting and source anchoring technology. Addressing the stringent data traceability requirements in the power sector, this method implements end-to-end source anchoring (Provenance Anchoring). Unlike existing technologies that only store results, this method mandates that the original text fragments, model inference confidence scores, model version numbers, and standardized logs be mounted as metadata on every relation edge during the graph database writing phase. This gives each piece of knowledge in the graph a complete "birth certificate," transforming the "black box" inference results into "white box" verifiable credentials. This allows maintenance personnel to trace the source and confidence level of each piece of knowledge at any time, solving the "black box" and untraceability problems in the knowledge graph construction process. It achieves end-to-end traceability from result to source, enhancing the graph's interpretability and auditability, and significantly improving the system's credibility and engineering practicality.

[0117] A method for constructing a knowledge graph for substation maintenance operations is presented. This method utilizes a lightweight, dedicated model with phased, full-parameter training and a multi-dimensional logical auditing closed loop for automated knowledge graph construction. It achieves deep logical reasoning capabilities comparable to general-purpose large models at low computational cost. Furthermore, it innovatively introduces an automated feedback closed-loop mechanism, upgrading the graph construction process from a traditional unidirectional "pipeline" to an "intelligent agent" with self-evolutionary capabilities. In addition, the built-in ontology constraint pruning and multi-dimensional logical gating mechanisms effectively suppress the "illusion" risk of generative models. Combined with a white-box traceability design throughout the entire process, the final generated graph possesses extremely high logical self-consistency and auditability, precisely meeting the high standards of data security and rigorous business logic requirements of the power system.

[0118] Compared to directly applying general large models or traditional small model cascading methods, a substation maintenance operation knowledge graph construction method achieves fundamental optimization in its technical approach: First, addressing the issues of high cost and lack of domain depth in general large models, it abandons simple fine-tuning (such as LoRA) or direct API calls, and instead adopts a course learning strategy that combines full-parameter training with knowledge distillation and direct preference optimization (DPO). This allows the lightweight model to not only learn domain terminology but also internalize complex logical reasoning patterns, achieving edge-end adaptation with "small parameter count and large logical capability". Secondly, addressing the common problem of generative models being prone to logical illusions and the difficulty that existing technologies often rely on manual cleaning in the later stages, a dual defense system of "pre-schema pruning + post-multidimensional logic auditing" was constructed. By cutting off illegal paths before inference and performing topology and mutual exclusion checks after inference, the model output is forced to conform to physical common sense from the algorithm level. Finally, addressing the pain point of difficulty in maintaining the graph after its construction, an automated DPO feedback loop was established, which automatically transforms errors in operation into nutrients for model evolution, solving the problem that traditional static graphs cannot continuously adapt to business changes.

[0119] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0120] Reference Figure 5 This application also provides a substation maintenance operation knowledge graph construction device, which corresponds one-to-one with the substation maintenance operation knowledge graph construction method described in the above embodiments. The substation maintenance operation knowledge graph construction device includes... The dataset module is used to build training datasets for knowledge related to substation maintenance; The modeling module is used to train an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model. The knowledge graph model is based on the Decoder-only Transformer model. After the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of the maintenance procedure and a group query attention mechanism for several query heads to share the same set of key-value heads are introduced. After the normalization unit of the feedforward network sublayer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced. The semantic mining module is used to perform semantic mining of substation maintenance operations using the substation maintenance language model and output a candidate entity list. The knowledge graph module is used to generate a knowledge graph by mapping nodes and edges in the graph database based on the candidate entity list.

[0121] The modeling module also includes, The first-stage training submodule is used to obtain the unlabeled text block dataset in the training dataset; perform causal language modeling tasks based on the domain-adaptive pre-training strategy of distillation, input the unlabeled text block dataset into the knowledge graph model, output the probability distribution of the text; and constrain the knowledge graph model through KL divergence to make its prediction distribution fit the prediction probability distribution of the pre-trained teacher model for the context, thereby training the first-stage model. The second-stage training submodule is used to acquire untested data in the training dataset and the confidence inference logic of the teacher model; based on the large-scale instruction distillation strategy of the untested data, the knowledge graph model is made to imitate the confidence inference logic of the teacher model, the untested data is input into the first-stage model, the probability distribution is output, and the second-stage model is trained. The third-stage training submodule is used to obtain the preference dataset in the training dataset; based on the direct preference optimization strategy of the preference data, the full-parameter model is fine-tuned, and the preference dataset is input into the second-stage model to train the substation maintenance language model.

[0122] For specific limitations on a substation maintenance operation knowledge graph construction device, please refer to the limitations on a substation maintenance operation knowledge graph construction method mentioned above, which will not be repeated here.

[0123] The modules in the aforementioned substation maintenance operation knowledge graph construction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0124] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements any of the above-described methods for constructing a knowledge graph for substation maintenance operations.

[0125] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described methods for constructing a knowledge graph for substation maintenance operations.

[0126] In one embodiment, a computer program product is provided, which includes a computer program that, when executed by a processor, implements any of the above-described methods for constructing a knowledge graph for substation maintenance operations.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. When executed, the computer program may include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.< / test> < / test> < / test> < / test>

Claims

1. A method for constructing a knowledge graph of power transformation maintenance operation, characterized in that, Includes the following steps, Obtain a training dataset for knowledge related to substation maintenance; Based on the training dataset, an improved knowledge graph model is trained to obtain a substation maintenance language model. Specifically, this includes: acquiring an unlabeled text block dataset from the training dataset; performing a causal language modeling task based on a distillation-based domain-adaptive pre-training strategy, inputting the unlabeled text block dataset into the knowledge graph model, and outputting the probability distribution of the text; and constraining the knowledge graph model through KL divergence to make its prediction distribution fit the prediction probability distribution of the pre-trained teacher model for the context, thereby training the first-stage model. Obtain the untested data in the training dataset and the confidence inference logic of the teacher model; A large-scale instruction distillation strategy based on unverified data enables the knowledge graph model to mimic the confidence reasoning logic of the teacher model. The unverified data is input into the first-stage model, and the probability distribution is output to train the second-stage model. The knowledge graph model is based on the Decoder-only Transformer model. After the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of the maintenance procedure and a group query attention mechanism for several query heads to share the same set of key value heads are introduced. After the normalization unit of the feedforward network sub-layer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced. The substation maintenance language model is used to perform semantic mining of substation maintenance operations and output a candidate entity list. Based on the candidate entity list, nodes and edges in the graph database are mapped to generate a knowledge graph.

2. The power transformation maintenance operation knowledge graph construction method according to claim 1, characterized in that, The training step of training the improved knowledge graph model based on the training dataset to obtain the substation maintenance language model also includes, Obtain the preference dataset from the training dataset; Based on the direct preference optimization strategy using preference data, the full-parameter model is fine-tuned, and the preference dataset is input into the second-stage model to train the substation maintenance language model.

3. The power transformation maintenance operation knowledge graph construction method according to claim 1, characterized in that, After the step of performing semantic mining of substation maintenance operations using the substation maintenance language model and outputting a candidate entity list, the method further includes... The post-processing operation of introducing regularization rules into the candidate entity list includes correcting characters that do not conform to the power writing standard, removing invalid extractions that do not conform to the entity definition, and adopting the longest form retention principle for similar entities.

4. The power transformation maintenance operation knowledge graph construction method according to claim 1, characterized in that, The step of generating a knowledge graph by mapping nodes and edges in the graph database based on the candidate entity list includes: Map the candidate entity list to nodes and edges in the graph database; By implementing a source anchoring mechanism, the original text fragments corresponding to the input data, the model's inference confidence score, and the model version number are labeled on each relation edge to generate a knowledge graph.

5. The substation maintenance operation knowledge graph construction method according to any one of claims 1-4, characterized in that, The structure of the knowledge graph model includes: An embedding layer is used to receive the input sequence formed by the training dataset; Several decoding layers are connected in series, and each decoding layer is used to decode the input sequence; each decoding layer includes an attention layer and a feedforward network sub-layer, and the output of the attention layer and the output of the feedforward network sub-layer are connected through a residual; The normalization layer is used to normalize the variance of the input vector; The output layer maps the high-dimensional hidden state vectors to the vocabulary dimension and transforms them into a probability distribution output.

6. A power transformation maintenance operation knowledge graph construction device, characterized in that, include, The dataset module is used to build training datasets for knowledge related to substation maintenance; The modeling module is used to train an improved knowledge graph model based on the training dataset to obtain a substation maintenance language model. Specifically, it includes: acquiring an unlabeled text block dataset from the training dataset; performing a causal language modeling task based on a distillation-based domain-adaptive pre-training strategy, inputting the unlabeled text block dataset into the knowledge graph model, and outputting the probability distribution of the text; and constraining the knowledge graph model through KL divergence to make its prediction distribution fit the prediction probability distribution of the pre-trained teacher model for the context, thereby training the first-stage model. Obtain the untested data in the training dataset and the confidence inference logic of the teacher model; A large-scale instruction distillation strategy based on unverified data enables the knowledge graph model to mimic the confidence reasoning logic of the teacher model. The unverified data is input into the first-stage model, and a probability distribution is output to train the second-stage model. The knowledge graph model is based on the Decoder-only Transformer model. After the normalization unit of the attention layer, a rotation position encoding unit for optimizing the length of the maintenance procedure and a grouped query attention mechanism for several query heads to share the same set of key-value heads are introduced. After the normalization unit of the feedforward network sublayer, a gated linear unit activation function for dynamically controlling the information transmission ratio according to the input content is introduced. The semantic mining module is used to perform semantic mining of substation maintenance operations using the substation maintenance language model and output a candidate entity list. The knowledge graph module is used to generate a knowledge graph by mapping nodes and edges in the graph database based on the candidate entity list.

7. A computer device, comprising: The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5.

Images