Power distribution network planning standard file knowledge graph construction method and system based on ALBERT-Transform-Attention-CRF model
By constructing a knowledge graph of distribution network planning standard documents based on the ALBERT-Transformer-Attention-CRF model, the problems of disordered storage and low retrieval efficiency of distribution network planning standard data are solved, achieving efficient management and utilization, supporting complex and ever-changing planning needs, and promoting the high-quality development of distribution networks.
Patent Information
- Application Number
- CN202511574515.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-31
- Publication Date
- 2026-03-20
AI Technical Summary
In existing technologies, standard data for power distribution network planning is stored in an unordered and unstructured manner as unprocessed images, documents, etc., lacking systematic and efficient means of storing and utilizing standard knowledge. This results in low retrieval efficiency, difficulty in meeting complex and ever-changing planning needs, difficulty in integrating standard documents, inability to form a structured knowledge resource pool, and low retrieval accuracy.
A knowledge graph of standard documents for power distribution network planning is constructed based on the ALBERT-Transformer-Attention-CRF model. By analyzing the characteristics of the standard documents, knowledge unit models are divided, and knowledge is extracted from the standard documents by combining rules and deep learning methods. The Neo4j graph database is used for storage and application system to achieve efficient management and utilization.
It enables efficient management and utilization of knowledge from standard documents on distribution network planning, improves retrieval accuracy and efficiency, supports complex and ever-changing planning needs, and promotes high-quality development of distribution networks.
Smart Images
Figure CN121706901A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of knowledge graph technology, and in particular relates to a method and system for constructing a knowledge graph for standard documents of power distribution planning based on the ALBERT-Transformer-Attention-CRF model. Background Technology
[0002] As a crucial component of the power system, the power distribution network covers urban and rural areas, connecting countless households and serving as the "last mile" of electricity supply. Under the new circumstances, the high-quality development of the power distribution network plays a vital role in ensuring power supply, supporting economic and social development, and improving people's livelihoods. Power distribution network planning is a key link in achieving high-quality development, and its overall coordination is extremely important; therefore, it is necessary to enhance the service and support capabilities of the power distribution network.
[0003] However, the construction of distribution network planning information is still in its initial stage, and many problems urgently need to be solved. First, a large amount of distribution network planning standard data is stored in an unprocessed, disordered, and unstructured manner, such as images and documents. This method is not conducive to efficient data management and utilization, affecting the scientific nature and accuracy of distribution network planning. Second, the field of distribution network planning lacks systematic and efficient means of storing and utilizing standard knowledge. This limits planners' ability to quickly acquire and apply standard knowledge, affecting the efficiency and quality of distribution network planning. Third, existing standard retrieval methods are limited to indexing by standard name and standard number, lacking more flexible and intelligent retrieval methods. This makes it inefficient for planners to find and apply relevant standards, making it difficult to meet complex and ever-changing planning needs. Simultaneously, with the rapid development of distribution network planning technology, related standard information is exploding, placing enormous information processing pressure on standardization workers and practitioners. They urgently need efficient and accurate means of acquiring standard knowledge to cope with the ever-increasing volume of standard information. Finally, the intelligent construction of distribution networks requires the establishment of a unified operating system to provide unified access standards and specifications for various equipment and sensors. This will help build a unified "data lake," providing support for data value mining. Regarding standard data management, existing technologies lack a dedicated management system tailored to the professional attributes of distribution network planning standard documents. They often employ general document management models, simply categorizing and storing standard documents into folders without breaking down and associating core knowledge such as technical clauses and parameters within the documents. Furthermore, they fail to establish unified data format specifications, resulting in incompatibility and integration of standard documents from different sources and versions. This prevents the formation of a structured knowledge resource pool, leaving them as isolated files and hindering knowledge reuse and collaboration. In terms of search support for professionals, existing technologies lack search designs adapted to the professional needs of distribution network planning. Besides relying on basic indexes such as standard names and numbers, they fail to set specific search dimensions based on the technical scenarios in distribution network planning or consider the logical relationships between standard clauses. This forces professionals to manually open multiple files and filter them when searching for standard references for specific technical scenarios, making it impossible to directly locate clauses relevant to specific tasks. Consequently, search efficiency and accuracy are both low. Summary of the Invention
[0004] To address the problems existing in the prior art, this invention proposes a method and system for constructing a knowledge graph for distribution network planning standard documents based on the ALBERT-Transformer-Attention-CRF model, providing an effective approach for constructing such a knowledge graph. By analyzing the characteristics of distribution network planning standard documents, a three-layer knowledge unit model of "standard document - standard core elements - standard clauses" is proposed. Based on the data characteristics of distribution network planning standard documents, two knowledge extraction methods are determined: template-based rules and deep learning-based methods, improving the efficiency of knowledge extraction and ensuring the accuracy of the knowledge graph. Finally, the knowledge graph is stored in the Neo4j graph database, and a graph application system is built to provide standard information services. The method proposed in this invention aims to be widely applied in the field of distribution network planning and promote the high-quality development of distribution networks.
[0005] To achieve the above-mentioned technical objectives, the present invention provides the following technical solution: A method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model, which specifically includes the following steps: Analyze the standard documents for distribution network planning, and based on the analysis results of the standard documents, combine the knowledge representation forms of ontology and knowledge units with the knowledge logic structure of the standard documents for distribution network planning, and construct a knowledge unit model for the standard documents; Rule-based extraction methods, deep learning methods, and knowledge unit models are used to extract knowledge from each layer of knowledge units in the standard file. The knowledge from each layer of knowledge units in the standard file includes knowledge from the standard file layer, knowledge from the standard core elements layer, and knowledge from the standard clause layer. A knowledge graph is constructed based on the extracted knowledge from the standard document layer, the standard core element layer, and the standard clause layer.
[0006] Furthermore, constructing a knowledge unit model includes: The composition, entities, relations, and attributes of knowledge units are defined using the entity relation definition method of ontology, and examples of knowledge unit content are provided. The specific process is as follows: While ensuring the semantic integrity of the document, knowledge units of different granularities are divided, including: retaining the coarse-grained knowledge form of paragraphs and sentences, while also dividing the text content in paragraphs and sentences into fine-grained units; After analyzing and summarizing the knowledge units, and combining the entity relationship definition method of ontology, we define and describe the entity types and entity relationships in the knowledge units, as well as the relationships between knowledge units. Knowledge unit content instances are obtained from the data in the standard documents of power distribution network planning by knowledge extraction technology, thereby obtaining the knowledge unit content; By analyzing the acquired knowledge unit content, the structure of undiscovered typical knowledge units is expanded into the knowledge unit definition; and the knowledge unit definition and content are maintained when the standard document is revised or replaced.
[0007] Furthermore, rule-based extraction methods, deep learning methods, and knowledge unit models are used to extract knowledge from each layer of knowledge units in standard files, including: dividing standard file data into each layer of knowledge units according to the knowledge unit model, and extracting the standard core element layer knowledge from the standard file data. A rule-based extraction method is used to extract standard document layer knowledge from each knowledge unit in the standard document; A deep learning approach is used to construct an ALBERT-hierarchical Transformer-layered Attention-CRF model to identify standard clause entities in knowledge units at each layer of a standard document. Based on the standard clause entity recognition results, the clause relationships between standard clause entities in knowledge units at each layer of the standard document are extracted using dependency parsing to obtain standard clause layer knowledge. The extracted knowledge from each layer is organized and integrated into a CSV table format; the results after extraction are manually corrected.
[0008] Furthermore, the specific process for extracting the core element layer knowledge from the standard document data is as follows: data acquisition, data segmentation, and construction of a dictionary for the distribution network planning standard domain are carried out; and the extraction of the core element layer knowledge is completed through clause title classification and keyword mapping rules during the data segmentation process. The specific process of extracting standard file layer knowledge from standard file data using a rule-based extraction method is as follows: knowledge extraction is performed on standard file layer data by constructing positional rules and syntactic rules to obtain complete standard file attribute triples, standard file objects and their relationships.
[0009] Furthermore, the ALBERT-Hierarchical Transformer-Hierarchical Attention-CRF model includes ALBERT, Hierarchical Transformer, Hierarchical Attention, and CRF, where: ALBERT, as a lightweight embedding layer, transforms each word in the standard text layer knowledge unit into a vector representation with contextual meaning; Hierarchical Transformers extract features of different granularities from context-meaning vector representations; Hierarchical Attention assigns different semantic weights to features of different granularities; CRF decodes features of different granularities assigned different semantic weights and outputs the optimal label sequence corresponding to each word as the standard text entity.
[0010] Furthermore, the hierarchical Transformer extracts features at different granularities from context-meaning vector representations based on local and global encoding. The specific process is as follows: The hierarchical Transformer extracts features from context-meaning vector representations based on local encoding. This includes: using a sliding window self-attention mechanism to encode context-meaning vector representations of single sentences or paragraphs to capture fine-grained features and obtain the hidden state representation output by the local encoding. The size of the sliding window is dynamically adjusted. The formula for the hidden state representation output by the local encoding is as follows: ; in, The hidden state representation of the local encoding output, i.e., the context vector representation of each word in a sentence / paragraph; M window It is a triangular mask matrix; For layer normalization, It is a normalized exponential function; , , These represent the query, key, and value in the self-attention mechanism. l Indicates the index layer number of the encoding network; The hierarchical Transformer integrates relevant information from other paragraphs based on global encoding, including: pre-calculating keyword weights through a domain terminology library and rule combinations as a bias term for the attention score. The calculation formula is: ; in, W keyword It is a keyword weight matrix. λ For hyperparameters, d For feature dimensions; Computing sparse attention : ; Where N(i) is the set of indexes of technology-related paragraphs, that is, the set of indexes of words that are semantically or structurally related to the keyword; i is the position index of the current keyword, and j is the position index of the context words used to calculate the attention score with the keyword. , These are the query and key for the corresponding bit index; After receiving sparse attention Then, the attention score is combined with the value vector at index j. By performing a weighted summation, we obtain cross-paragraph rule association features that incorporate relevant information from other paragraphs.
[0011] Furthermore, the hierarchical Attention represents the hidden state of the word-level local encoding output. Semantic weights are assigned to features associated with sentence-level cross-paragraph rules, specifically as follows: For each paragraph, the importance weight of the [CLS] tag is calculated, focusing on important sentences with technical constraints in the cross-paragraph rule association features; The formula for calculating importance weight is: ; Where s represents the sentence index; k represents the total number of sentence indices; Within important sentences, attention is focused through scaling dot product. Key terms in the text; the formula for calculating dot product attention is: ; in, M term For the term mask matrix, when the input word w i When it belongs to the parameter class term, M term ( i , j )=+∞.
[0012] Furthermore, the specific process by which CRF decodes multi-granular features assigned different semantic weights and outputs the optimal label sequence corresponding to the word sequence is as follows: The given input is a hierarchically processed sequence of words. The output is the label sequence for each word. CRF assessment score for: ; in, n Indicates sentence length. P The state score matrix is the output of the hierarchical Attention mechanism. For the first i Each position is a tag The probability, A For the transition score matrix, For tags i arrive i +1 transfer score; After normalizing all paths using the Softmax function, the probability of the output sequence is obtained using the maximum likelihood function; the formula is expressed as:
[0013]
[0014] in, For the true labeled value, Given all possible label sequences, the Dimension Bit algorithm decodes them to obtain the output sequence with the highest probability, i.e., the optimal label sequence. .
[0015] Furthermore, the specific process of extracting standard clause relations using the relation extraction method of dependency parsing is as follows: Dependency parsing is performed on standard clauses in standard documents for power distribution network planning to obtain the core terms of the clauses and the dependency relationships between the clause body and clause value terms. Based on the clause body, core terms, clause value terms, and their dependency relationships, the clause relationship types of the standard clauses are determined through clause core relationship dictionary matching. During the dictionary matching process, missing semantics are completed according to relationship allocation rules, and the relationships between clause body and clause value terms are assigned. The relationship allocation rules specifically include: If the core term of the article and other components in the standard article sentence have an adverbial-head relationship, then the other components in the standard article sentence are modifiers of the core term of the article. The core term of the article is completed with the modifier, and the article core relationship dictionary matching is performed again. The article relationship type is determined by the type of the completed core term. If the main body and value words of an article directly depend on the core words of the article, the article relationship type is the core word type. If the main body of an article directly depends on a non-core component in the standard article, and the two are in a parallel relationship, then the main body of the article, the non-core component, and the core word of the article are specifically consistent with the article's relationship; if the two are in a noun-head relationship, then the main body of the article modifies the non-core component, and the main body of the article and the non-core component are merged into a complete main body of the article, which is then re-entered into dictionary matching; the same treatment applies when the article value word directly depends on a non-core component in the standard article. If the main body of the article and the core word of the article are in an adverbial-head relationship, and there are other subject components in the standard article that serve as the subject of the core word of the article, then the main body of the article is a conditional modifier of other subject components. The main body of the article and other subject components are merged to obtain the complete main body of the article, which is then re-entered into dictionary matching. If there are no other subject components, the missing subject in the sentence is manually filled in, and then the sentence is re-entered into dictionary matching.
[0016] Furthermore, the present invention also provides an application system for the above method, comprising: The knowledge unit modeling module is used to build knowledge unit models for standard documents. The knowledge extraction module is used to extract knowledge from knowledge units at each layer of standard files using rule-based extraction methods, deep learning methods, and knowledge unit models, thereby completing the construction of the standard file knowledge graph.
[0017] Based on the above technical solution, the present invention has at least the following beneficial effects: This invention addresses the lack of in-depth and effective methods for storing and utilizing standard document knowledge in distribution network planning standard information services. It applies knowledge graphs to the field of distribution network standards, focusing on three aspects: knowledge modeling, knowledge extraction, and knowledge storage and application. A knowledge modeling method for distribution network planning standard documents is determined, and a three-layer knowledge unit model of "standard document layer - standard core element layer - standard clause layer" is proposed. Knowledge extraction methods for each layer are designed: the ALBERT-hierarchical Transformer-layered Attention-CRF model is used for core task entity extraction, and a dependency parsing-based extraction method is used for relation extraction. Distribution network planning standard document knowledge is stored using the Neo4j graph database, and a distribution network planning standard document knowledge graph application system is constructed. This achieves efficient management and effective utilization of distribution network planning standard document knowledge, providing a new approach for the intelligent construction of distribution network planning standardization. Attached Figure Description
[0018] The present invention will now be described in further detail with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the knowledge modeling process of this invention; Figure 2 This is a flowchart illustrating the knowledge extraction process of this invention; Figure 3 This is an example of the knowledge graph of standard documents for power distribution network planning constructed by this invention. Detailed Implementation
[0019] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.
[0020] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0021] like Figure 1-3 As shown, this invention discloses a method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model, which specifically includes: Analyze the standard documents for distribution network planning, and based on the analysis results of the standard documents, combine the knowledge representation forms of ontology and knowledge units with the knowledge logic structure of the standard documents for distribution network planning, and construct a knowledge unit model for the standard documents; As a preferred embodiment, such as Figure 2 As shown, constructing a knowledge unit model includes: defining the composition, entities, relations, and attributes of a knowledge unit using the entity relation definition method of an ontology, and providing an example of knowledge unit content. The specific process is as follows: The division of knowledge units is crucial for constructing the knowledge model of power distribution network planning guidelines. Therefore, this application first divides knowledge units of different granularities to meet the needs of knowledge graph retrieval and application, while ensuring the semantic integrity of the document. This includes: retaining the coarse-grained knowledge form of paragraphs and sentences, while also dividing the content of clauses in paragraphs and sentences into fine-grained units to support retrieval and question answering applications. This addresses the problem that the general document knowledge graph's method of mining text keywords and their relationships can only reflect the theme or outline of the text, resulting in significant knowledge loss. Ultimately, it presents a comprehensive and complete knowledge content of the power distribution network planning standard document. Analyze and summarize the content of the knowledge units after division, and combine the entity relationship definition method in the ontology to define and describe the entity types and relationships in the knowledge units, as well as the relationships between knowledge units; Knowledge unit content instances are obtained from the data in the standard documents of power distribution network planning by knowledge extraction technology, thereby obtaining the knowledge unit content; By analyzing the acquired knowledge unit content, the structure of undiscovered typical knowledge units is expanded into the knowledge unit definition; and when the standard document is revised or replaced, the knowledge unit definition and content are maintained to ensure the accuracy and effectiveness of the knowledge.
[0022] Rule-based extraction methods, deep learning methods, and knowledge unit models are used to extract knowledge from each layer of knowledge units in the standard file. The knowledge from each layer of knowledge units in the standard file includes knowledge from the standard file layer, knowledge from the standard core elements layer, and knowledge from the standard clause layer. In this embodiment, the rule-based extraction method, deep learning method, and knowledge unit model extract knowledge from each layer of knowledge units in the standard file, including: The standard file data is divided into knowledge units at each level in the standard file according to the knowledge unit model, and the standard core element layer knowledge is extracted from the standard file data. As a preferred embodiment, such as Figure 2As shown, the specific process for extracting the core element layer knowledge from standard document data is as follows: data acquisition, data segmentation (dividing documents into standard document layers, such as name, number, scope, etc., and clause layers, such as chapter titles + specific clauses) and construction of a dictionary for the distribution network planning standard field (including the construction of a terminology dictionary and relation dictionary for the distribution network field); and in the data segmentation process, the extraction of the core element layer knowledge is completed through clause title classification and keyword mapping rules. In this application, the specific implementation method for extracting knowledge from the core element layer of the standard through clause title classification and keyword mapping rules is as follows: During the data segmentation phase, when extracting each clause: regular expressions are used to match the standard clause number + title; the title is the potential core element tag; for example, "3.1 Distribution Network Planning Principles" is labeled as "Planning Principles"; each clause is assigned to a core element theme node, thus completing the construction of the standard core element layer. In other words, the extraction of core elements is implicitly completed during the data segmentation phase through the classification of clause titles and keyword mapping rules.
[0023] A rule-based extraction method is used to extract standard file layer knowledge from each knowledge unit in the standard file. The specific process is as follows: knowledge is extracted from the standard file layer data by constructing positional rules and syntactic rules to obtain complete standard file attribute triples, standard file objects and their relationships.
[0024] In this embodiment, positional rules refer to a method of identifying and extracting entities or keywords by their fixed positions or format structures in the text. This method is mainly applicable to semi-structured texts with clear structures and fixed formats. Syntactic rules are a method of extracting entities and relations based on the grammatical structure of words in a sentence (such as subject-verb-object and dependency relations). This method is often combined with techniques such as syntactic dependency analysis, constituent syntactic analysis, and POS part-of-speech tagging.
[0025] Using deep learning methods, an ALBERT-hierarchical Transformer-layered Attention-CRF model is constructed to identify standard text entities in knowledge units at each layer of a standard document. In a preferred embodiment, the ALBERT-Hierarchical Transformer-Hierarchical Attention-CRF model includes ALBERT, Hierarchical Transformer, Hierarchical Attention, and CRF, wherein: ALBERT (Lightweight Pre-trained Language Model) provides contextual semantic embedding, transforming each word in a standard text into a vector representation with contextual meaning; similar to BERT, it can capture global semantic information of a sentence, but its structure is lighter and training is faster; The hierarchical Transformer is used to construct hierarchical semantic structures, extracting multi-granular features from context-meaning vector representations, and is particularly suitable for semantic understanding of long documents / multi-level paragraphs. In this embodiment, the hierarchical Transformer extracts features of different granularities from context-meaning vector representations based on local and global encoding. The specific process is as follows: A sliding window self-attention mechanism is employed to encode the context-meaning vector representations of single sentences and paragraphs to capture fine-grained features and obtain the hidden state representation of the local encoded output. The size of the sliding window is dynamically adjusted. The formula for the hidden state representation of the local encoded output is as follows: ; in, The hidden state representation of the local encoding output, i.e., the context vector representation of each word in a sentence / paragraph; M window It is a triangular mask matrix; For layer normalization, It is a normalized exponential function; , , These represent the query, key, and value in the self-attention mechanism. l Indicates the index layer number of the encoding network; The hierarchical Transformer integrates relevant information from other paragraphs based on global encoding, including: pre-calculating keyword weights through a domain terminology corpus and rule combinations (primarily using TF-IDF methods, domain terminology corpus weight fusion, and word vector / embedding similarity methods), which serve as a bias term in the attention score. The calculation formula is: ; in, W keyword It is a keyword weight matrix. λ For hyperparameters, d For feature dimensions; Then, attention is diverted. The calculation formula is: ; Where N(i) is the set of indexes of technology-related paragraphs, that is, the set of indexes of words that are semantically or structurally related to the position index; i is the position index of the current keyword, and j is the position index of the context words used to calculate the attention score with the keyword. , These are the query and key for the corresponding bit index; After receiving sparse attention Then, the attention score is combined with the value vector at index j. By performing a weighted summation, we obtain cross-paragraph rule association features that incorporate relevant information from other paragraphs.
[0026] Hierarchical Attention assigns different semantic weights to features of different granularities, enabling hierarchical attention to information of different granularities, such as words, sentences, and paragraphs, thereby improving the accuracy of key entity recognition. It simulates the focus on key terms and relational words when manually reading standard documents. In this embodiment, the hierarchical Attention is the hidden state representation of the local encoding output at the word level. The specific process of assigning semantic weights to features associated with sentence-level cross-paragraph rules is as follows: For each paragraph, the importance weight of the [CLS] tag is calculated, focusing on important sentences with technical constraints in the cross-paragraph rule association features; The formula for calculating importance weight is: ; Where s represents the sentence index; k represents the total number of sentence indices; Within important sentences, attention is focused through scaling dot product. Key terms in the text; the formula for calculating dot product attention is: ; in, M term For the term mask matrix, when the input word w i When it belongs to the parameter class term, M term ( i , j )=+∞.
[0027] CRF (Conditional Random Field) decodes multi-granularity features assigned different semantic weights and outputs the optimal label sequence corresponding to the word sequence; ensuring that the prediction result meets the structural constraints of entity recognition; the specific process of decoding multi-granularity features assigned different semantic weights using CRF to output the optimal label sequence corresponding to the word sequence is as follows: The given input is a hierarchically processed sequence of words. The output is the label sequence for each word. CRF assessment score for: ; in, n Indicates sentence length. P The state score matrix is the output of the hierarchical Attention mechanism. For the first i Each position is a tag The probability, A For the transition score matrix, For tags i arrive i +1 transfer score; After normalizing all paths using the Softmax function, the probability of the output sequence is obtained using the maximum likelihood function; the formula is expressed as:
[0028]
[0029] in, For the true labeled value, Given all possible label sequences, the Dimension Bit algorithm is used to decode the output sequence with the highest probability. .
[0030] In addition, in this embodiment, a rule-based extraction method is used to obtain a small number of standard clause triples and manually annotate them to construct a standard clause training dataset for training the deep learning algorithm model. The dataset is then processed sequentially through the optimal model's ALBERT, hierarchical Transformer, hierarchical Attention, and CRF to complete the entity recognition of the standard clauses.
[0031] Then, combining the standard clause entity recognition results, and based on the dependency parsing method, the clause relationships between standard clause entities in each knowledge unit of the standard document are extracted; As a preferred embodiment, in this example, the specific process of extracting standard clause relations using the relation extraction method of dependency parsing is as follows: Dependency parsing is performed on standard clauses in standard documents for power distribution network planning to obtain the core terms of the clauses and the dependency relationships between the clause body and clause value terms. Based on the clause body, core terms, clause value terms, and their dependency relationships, the clause relationship types of the standard clauses are determined through clause core relationship dictionary matching. During the dictionary matching process, missing semantics are completed according to relationship allocation rules, and the relationships between clause body and clause value terms are assigned. The relationship allocation rules specifically include: If the core term of the article and other components in the standard article sentence have an adverbial-head relationship, then the other components in the standard article sentence are modifiers of the core term of the article. The core term of the article is completed with the modifier, and the article core relationship dictionary matching is performed again. The article relationship type is determined by the type of the completed core term. If the main body and value words of an article directly depend on the core words of the article, the article relationship type is the core word type. If the main body of an article directly depends on a non-core component in the standard article, and the two are in a parallel relationship, then the main body of the article, the non-core component, and the core word of the article are specifically consistent with the article's relationship; if the two are in a noun-head relationship, then the main body of the article modifies the non-core component, and the main body of the article and the non-core component are merged into a complete main body of the article, which is then re-entered into dictionary matching; the same treatment applies when the article value word directly depends on a non-core component in the standard article. If the main body of the article and the core word of the article are in an adverbial-head relationship, and there are other subject components in the standard article that serve as the subject of the core word of the article, then the main body of the article is a conditional modifier of other subject components. The main body of the article and other subject components are merged to obtain the complete main body of the article, which is then re-entered into dictionary matching. If there are no other subject components, the missing subject in the sentence is manually filled in, and then the sentence is re-entered into dictionary matching.
[0032] So far Figure 2 As shown, this application extracts standard clause layer knowledge through deep learning algorithm model and dependency parsing; after the knowledge of each layer is extracted, the extracted knowledge of each layer is organized and integrated into CSV table format; and the results after extraction are manually corrected.
[0033] This embodiment also provides an application system for the above method, including: The knowledge unit modeling module is used to build knowledge unit models for standard documents. The knowledge extraction module is used to extract knowledge from knowledge units at each layer of standard files using rule-based extraction methods, deep learning methods, and knowledge unit models, thereby completing the construction of the standard file knowledge graph.
[0034] In addition, after the knowledge graph is constructed, in order to facilitate storage and application, this embodiment selects the neo4j database to store the knowledge graph of the power distribution network planning standard documents, providing a systematic and effective way to manage standard document knowledge; As a preferred implementation method, the specific process of using the neo4j database to store the knowledge graph of power distribution network planning standard documents includes: Neo4j graph database is selected for knowledge storage, and knowledge is represented in the form of labels, nodes, edges, and attributes. Labels correspond to the hierarchical knowledge unit model of the power distribution network planning standard document, nodes and attributes correspond to entities and their attributes, and edges correspond to the relationships between entities. Construct a knowledge graph application system for distribution network planning standard documents to realize knowledge management functions for distribution network planning standard documents and provide standard information services including standard retrieval, standard visualization and indicator comparison. The system architecture includes: Data layer: The system's underlying database, including Neo4j and MySQL databases; the MySQL database is used to store user login information. Data access layer: Responsible for data transfer, enabling data access and database operations; including knowledge base management, user information management, and model management; Business logic layer: performs logical judgments and executes operations on user requests, including user login, standard search, document display, graph search, graph display, and indicator comparison. Presentation layer: interacts directly with users, is responsible for accepting user requests and returning request results; performs operations including standard retrieval, graph display and indicator comparison.
[0035] This concludes the description of the entire process of constructing the knowledge graph for distribution network planning standard documents proposed in this invention. Furthermore, this embodiment also provides verification of the performance of the proposed method as follows: The feasibility of this invention was verified by testing on a dataset of clauses built based on the "Q / GDW 10738-2020 Technical Guidelines for Distribution Network Planning and Design". This guideline document is divided into five parts: table of contents, preface, main text, appendix, and compilation instructions. It standardizes aspects such as planning area division, load forecasting and power balance, main technical principles, power grid structure and main wiring methods, equipment selection, basic requirements for intelligent systems, user and power supply access requirements, planning calculation and analysis requirements, and technical and economic analysis. The clause dataset uses the BMES entity annotation method, assigning a label to each character in the text. Entities are labeled with "B-label", "M-label", "E-label", and "S-label", representing the beginning, middle, and end of an entity, respectively, and a single character entity.
[0036] The model evaluation metric uses accuracy. P Recall rate R and F A value of 1 is used for evaluation. Precision is the ratio of the number of samples that correctly predicted the data to the total sample. The calculation formula is as follows:
[0037] in, For correctly predicted positive samples, These are positive samples that were incorrectly predicted.
[0038] Recall is the ratio of the number of samples that were actually correctly predicted to the number of samples that should have been correctly predicted, i.e., the number of correct entities extracted by the model / the number of labeled entities.
[0039]
[0040] in, These are negative samples that were incorrectly predicted.
[0041] The value is an overall evaluation metric determined by both precision and recall.
[0042]
[0043] This invention uses the PyTorch deep learning framework for model training and testing. The experimental environment is shown in Table 1 below: Table 1 Experimental Environment
[0044] The specific parameter settings of the ALBERT-hierarchical Transformer-hierarchical Attention-CRF model constructed in this invention are shown in Table 2: Table 2 Model Parameters
[0045] The test results of the model constructed in this invention are shown in Table 3 below: Table 3 Test Results
[0046] The final knowledge graph for the distribution network planning standard document uses the Neo4j graph database for knowledge storage. The Neo4j database represents knowledge in the form of tags, nodes, edges, and attributes. Tags correspond to the hierarchical structure of the knowledge unit model in the distribution network planning standard document; nodes and their attributes correspond to entities and their attributes; and edges correspond to the relationships between entities. The storage scheme is shown in Table 4 below. Table 4 Storage Scheme
[0047] The knowledge graph of the power distribution network planning standard documents, formed by storing the knowledge unit data of each layer of the acquired standard documents into the Neo4j database, is as follows: Figure 3 As shown in the diagram. Circles represent entities, circle colors represent label categories, and lines connecting circles represent relationships.
[0048] In summary, the method proposed in this invention addresses the lack of in-depth and effective methods for storing and utilizing standard document knowledge in distribution network planning standard information services. It applies knowledge graphs to the field of distribution network standards, and conducts research on three aspects: knowledge modeling, knowledge extraction, and knowledge storage and application. Through the ALBERT-hierarchical Transformer-hierarchical Attention-CRF model, a knowledge graph for distribution network planning standard documents is successfully constructed, realizing efficient management and effective utilization of knowledge in distribution network planning standard documents, and providing a new approach for the intelligent construction of distribution network planning standardization.
[0049] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0050] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).
[0051] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model, characterized in that, Specifically, the following steps are included: Analyze the standard documents for distribution network planning, and based on the analysis results of the standard documents, combine the knowledge representation forms of ontology and knowledge units with the knowledge logic structure of the standard documents for distribution network planning, and construct a knowledge unit model for the standard documents; Rule-based extraction methods, deep learning methods, and knowledge unit models are used to extract knowledge from each layer of knowledge units in the standard file. The knowledge from each layer of knowledge units in the standard file includes knowledge from the standard file layer, knowledge from the standard core elements layer, and knowledge from the standard clause layer. A knowledge graph is constructed based on the extracted knowledge from the standard document layer, the standard core element layer, and the standard clause layer.
2. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 1, characterized in that, The method uses rule-based extraction, deep learning methods, and knowledge unit models to extract knowledge from each layer of knowledge units in a standard file. This includes dividing standard file data into each layer of knowledge units according to the knowledge unit model and extracting the standard core element layer knowledge from the standard file data. A rule-based extraction method is used to extract standard document layer knowledge from each knowledge unit in the standard document; A deep learning approach is used to construct an ALBERT-hierarchical Transformer-layered Attention-CRF model to identify standard clause entities in knowledge units at each layer of a standard document. Based on the standard clause entity recognition results, the clause relationships between standard clause entities in knowledge units at each layer of the standard document are extracted using dependency parsing to obtain standard clause layer knowledge. The extracted knowledge from each layer is organized and integrated into a CSV table format; The results after extraction are manually corrected.
3. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 1, characterized in that, The construction of the knowledge unit model includes: defining the composition, entities, relations, and attributes of the knowledge unit using the entity relation definition method of ontology, and providing an example of the knowledge unit content. The specific process is as follows: While ensuring the semantic integrity of the document, knowledge units of different granularities are divided, including: retaining the coarse-grained knowledge form of paragraphs and sentences, while also dividing the content of clauses in paragraphs and sentences into fine-grained units; After analyzing and summarizing the knowledge units, and combining the entity relationship definition method of ontology, we define and describe the entity types and entity relationships in the knowledge units, as well as the relationships between knowledge units. Knowledge unit content instances are obtained from the data in the standard documents of power distribution network planning by knowledge extraction technology, thereby obtaining the knowledge unit content; By analyzing the acquired knowledge unit content, the structure of undiscovered typical knowledge units is expanded into the knowledge unit definition; and the knowledge unit definition and content are maintained when the standard document is revised or replaced.
4. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 2, characterized in that, The specific process for extracting the core element layer knowledge from standard document data is as follows: data acquisition, data segmentation, and construction of a dictionary for the distribution network planning standard domain are carried out; and the extraction of the core element layer knowledge is completed through clause title classification and keyword mapping rules during the data segmentation process. The specific process of extracting standard file layer knowledge from standard file data using a rule-based extraction method is as follows: knowledge extraction is performed on standard file layer data by constructing positional rules and syntactic rules to obtain complete standard file attribute triples, standard file objects and their relationships.
5. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 2, characterized in that, The ALBERT-Hierarchical Transformer-Hierarchical Attention-CRF model includes ALBERT, Hierarchical Transformer, Hierarchical Attention, and CRF, where: ALBERT, as a lightweight embedding layer, transforms each word in the standard text layer knowledge unit into a vector representation with contextual meaning; Hierarchical Transformers extract features of different granularities from context-meaning vector representations; Hierarchical Attention assigns different semantic weights to features of different granularities; CRF decodes features of different granularities assigned different semantic weights and outputs the optimal label sequence corresponding to each word as the standard text entity.
6. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 5, characterized in that, The hierarchical Transformer extracts features at different granularities from context-meaning vector representations based on local and global encoding. The specific process is as follows: The hierarchical Transformer extracts features from context-meaning vector representations based on local encoding. This includes: using a sliding window self-attention mechanism to encode context-meaning vector representations of single sentences or paragraphs to capture fine-grained features and obtain the hidden state representation output by the local encoding. The size of the sliding window is dynamically adjusted. The formula for the hidden state representation output by the local encoding is as follows: ； in, The hidden state representation of the local encoding output, i.e., the context vector representation of each word in a sentence / paragraph; M window It is a triangular mask matrix; For layer normalization, It is a normalized exponential function; , , These represent the query, key, and value in the self-attention mechanism. l Indicates the index layer number of the encoding network; The hierarchical Transformer integrates relevant information from other paragraphs based on global encoding, including: pre-calculating keyword weights through a domain terminology library and rule combinations as a bias term for the attention score. The calculation formula is: ； in, W keyword It is a keyword weight matrix. λ For hyperparameters, d For feature dimensions; Computing sparse attention : ； Where N(i) is the set of indexes of technology-related paragraphs, that is, the set of indexes of words that are semantically or structurally related to the keyword; i is the position index of the current keyword, and j is the position index of the context words used to calculate the attention score with the keyword. , These are the query and key for the corresponding bit index; After receiving sparse attention Then, the attention score is combined with the value vector at index j. By performing a weighted summation, we obtain cross-paragraph rule association features that incorporate relevant information from other paragraphs.
7. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 6, characterized in that, Hierarchical Attention assigns semantic weights to the hidden state representation of the word-level local encoding output and the sentence-level cross-paragraph rule association features, specifically as follows: For each paragraph, the importance weight of the [CLS] tag is calculated, focusing on important sentences with technical constraints in the cross-paragraph rule association features; The formula for calculating importance weight is: ； Where s represents the sentence index; k represents the total number of sentence indices; Within important sentences, attention is focused through scaling dot product. Key terms in the text; the formula for calculating dot product attention is: ； in, M term For the term mask matrix, when the input word w i When it belongs to the parameter class terminology, M term ( i , j )=+∞.
8. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 5, characterized in that, The specific process by which CRF decodes multi-granular features assigned different semantic weights and outputs the optimal label sequence corresponding to the word sequence is as follows: The given input is a hierarchically processed sequence of words. The output is the label sequence for each word. CRF assessment score for: ； in, n Indicates sentence length. P The state score matrix is the output of the hierarchical Attention mechanism. For the first i Each position is a tag The probability, A For the transition score matrix, For tags i arrive i +1 transfer score; After normalizing all paths using the Softmax function, the probability of the output sequence is obtained using the maximum likelihood function; the formula is expressed as: ； ； in, For the true labeled value, Given all possible label sequences, the Dimension Bit algorithm decodes them to obtain the output sequence with the highest probability, i.e., the optimal label sequence. .
9. The method for constructing a knowledge graph for standard documents of power distribution network planning based on the ALBERT-Transformer-Attention-CRF model according to claim 4, characterized in that, The specific process of extracting standard clause relations using the relation extraction method of dependency parsing is as follows: Dependency parsing analysis is performed on standard clauses in standard documents for distribution network planning to obtain the core terms of the clauses and the dependency relationships between the clause body and the clause value terms. Based on the clause body, core terms of the clauses, clause value terms and the dependency relationships among them, the clause relationship type of the standard clauses is determined by clause core relationship dictionary matching. During dictionary matching, missing semantics are completed according to relation allocation rules, and the relationships between the main body of the article and the value of the article are assigned. The relation allocation rules specifically include: If the core term of the article and other components in the standard article sentence have an adverbial-head relationship, then the other components in the standard article sentence are modifiers of the core term of the article. The core term of the article is completed with the modifier, and the article core relationship dictionary matching is performed again. The article relationship type is determined by the type of the completed core term. If the main body and value words of an article directly depend on the core words of the article, the article relationship type is the core word type. If the main body of an article directly depends on a non-core component in the standard article, and the two are in a parallel relationship, then the main body of the article, the non-core component, and the core word of the article are specifically consistent with the article's relationship; if the two are in a noun-head relationship, then the main body of the article modifies the non-core component, and the main body of the article and the non-core component are merged into a complete main body of the article, which is then re-entered into dictionary matching; the same treatment applies when the article value word directly depends on a non-core component in the standard article. If the main body of the article and the core word of the article are in an adverbial-head relationship, and there are other subject components in the standard article that serve as the subject of the core word of the article, then the main body of the article is a conditional modifier of other subject components. The main body of the article and other subject components are merged to obtain the complete main body of the article, which is then re-entered into dictionary matching. If there are no other subject components, the missing subject in the sentence is manually filled in, and then the sentence is re-entered into dictionary matching.
10. An application system for a knowledge graph construction method for power distribution network planning standard documents according to any one of claims 1-9, characterized in that, include: The knowledge unit modeling module is used to build knowledge unit models for standard documents. The knowledge extraction module is used to extract knowledge from knowledge units at each layer of the standard document based on rule-based extraction methods, deep learning methods, and knowledge unit models, and to complete the construction of the knowledge graph of the power distribution network planning standard document.