Method, apparatus, and storage medium for constructing a knowledge base for power grid infrastructure data based on LLM.

Abstract

This invention proposes a method, apparatus, and storage medium for constructing a power grid infrastructure engineering data knowledge base based on LLM (Limited Language Modeling). The method includes: extracting power grid infrastructure engineering data from massive amounts of journal literature in the power grid field using LLM; integrating power grid GIS platform data and equipment ledgers based on the power grid infrastructure engineering data to construct a dynamic knowledge graph of power grid engineering data containing entity set E, attribute set R, and relation set A; deeply fusing the dynamic knowledge graph of power grid engineering data with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base; and optimizing the initial power grid infrastructure engineering data knowledge base based on Retrieval Enhanced Generation (RAG) to obtain a more accurate power grid infrastructure engineering data knowledge base. This invention achieves a deep fusion of "symbolicism" and "connectivity" in generating the initial power grid infrastructure engineering data knowledge base, followed by optimization to obtain a more accurate power grid infrastructure engineering data knowledge base.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and power grid infrastructure engineering integration technology, specifically to a method, device and storage medium for constructing a power grid infrastructure engineering data knowledge base based on LLM. Background Technology

[0002] Traditional symbolic AI lacks reasoning ability, and traditional LLM lacks logical interpretability. General large models have biases in their understanding of complex entities such as power grid infrastructure, which involve the physical world, engineering specifications, and geographical information, and are prone to producing "illusions".

[0003] Due to the diverse terminology used in power grid infrastructure projects, the general model lacks sufficient semantic representation learning for professional entities such as "tower impedance", "relay protection setting", and "engineering transient stability", resulting in output results lacking engineering physical meaning.

[0004] Furthermore, power grid infrastructure data is highly time-sensitive (such as the latest technical upgrade plans and standard updates), and the offline weights of general models cannot be updated in real time, posing a risk of "knowledge becoming outdated".

[0005] Infrastructure data encompasses multiple modalities, including structured (ledgers, Excel), semi-structured (XML, PDF), and unstructured (survey reports, construction logs), making it difficult to represent them uniformly using a general model. Summary of the Invention

[0006] In view of one or more technical defects in the prior art, the present invention proposes the following technical solution.

[0007] A method for constructing a knowledge base for power grid infrastructure engineering data based on LLM, characterized in that the method includes: The cleaning step involves using LLM to extract power grid infrastructure engineering data from the "Specification for Calculation of Engineering Quantities in Power Grid Construction Projects", "Electric Power Safety Work Regulations", and a large amount of journal literature in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes, and equipment attribute values. Knowledge graph construction steps: Based on the power grid infrastructure engineering data, integrate the power grid GIS platform data and equipment ledger to construct a dynamic knowledge graph of power grid engineering data, which includes entity set E, attribute set R, and relationship set A. The entity set E includes power grid equipment, the attribute set R includes equipment attributes, and the relationship set A is the relationship between power grid equipment in the entity layer. The linking step involves deeply integrating the dynamic knowledge graph of power grid engineering data with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base. The implicit features include the weights and vector space of LLM. The optimization step involves optimizing the initial power grid infrastructure engineering data knowledge base based on Retrieval Enhanced Generation (RAG) to obtain the power grid infrastructure engineering data knowledge base. Where E={e1,e2,...,e n}, covering primary equipment, secondary devices, infrastructure procedures, survey indicators, cost items, and specification clauses; R={r1,r2,...,r m}, including topological relationships, composition relationships, dependency relationships, constraint relationships, and compliance relationships; A={a1,a2,...,a...} k} is used to describe key information such as equipment parameters, engineering indicators, and technical limits, where n, m, and k are all integers ≥ 2.

[0008] Furthermore, the optimization step involves the following steps: after the LLM generates an answer to the user's question based on the initial power grid infrastructure engineering data knowledge base, the generated question and answer are compared with the retrieved standard data. If the generated question and answer are inconsistent with the standard data, the initial power grid infrastructure engineering data knowledge base is updated using the standard data to obtain the power grid infrastructure engineering data knowledge base.

[0009] Furthermore, in the cleaning step: at the input layer of the LLM, the general token and the power grid engineering entity token are embedded in a differentiated entity-aware manner.

[0010] Furthermore, in the linking step, graph attention is used to jointly encode the association information of entity context and dynamic knowledge graph of power grid engineering data to perform entity disambiguation, mapping entities to symbols of dynamic knowledge graph of power grid engineering data, embedding the symbols into entities and projecting the implicit features of LLM context to semantic space, and achieving deep fusion through alignment constraints and dynamic gating to obtain the initial knowledge base of power grid infrastructure engineering data.

[0011] Furthermore, the entity embedding operation of the symbols involves: pre-training the dynamic knowledge graph of power grid engineering data using the TransE algorithm, mapping symbol entities and relations to low-dimensional dense vectors, and obtaining structured feature vectors with logical constraints for symbol entity embedding. e s : ; in, G R A graph showing the relationships between entities. G R =( E, R, E ).

[0012] Furthermore, the operation of projecting the implicit features of the LLM context onto the semantic space is as follows: The implicit features output by the LLM...h LLM Cross-space alignment is achieved by mapping to the same-dimensional space as the symbol embedding via a low-rank adaptation (LoRA) projection layer:

[0013] in, e c For the semantic feature vector of the LLM context embedding of symbols mapped to the same dimensional space as the symbol embedding, W proj It is a low-rank projection matrix. b It is a constant. While retaining the semantic capabilities of LLM, it enables the transformation from a general representation to a professional representation for power grid infrastructure.

[0014] Furthermore, to fine-tune the LLM and reduce the distributional discrepancy between symbolic embeddings and contextual representations, an aligned orthogonal joint loss function is constructed: ; in, This represents the distance between the same entity in two spaces. This represents the orthogonal constraint loss, which makes entities spatially independent. For weights.

[0015] Furthermore, the feature vector ultimately fused by the dynamic gating... e final The calculation method is as follows:

[0016] in, This represents the gated generation function. Indicates dynamic gating weights, Will The value is normalized.

[0017] This invention also proposes a device for constructing a knowledge base for power grid infrastructure engineering data based on LLM (Liquid Module Management), the device comprising: The cleaning unit uses LLM to extract power grid infrastructure engineering data from the "Specification for Calculation of Engineering Quantities in Power Grid Construction Projects", "Electric Power Safety Work Regulations" and a large amount of journal literature in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes and equipment attribute values. Knowledge Graph Construction Unit: Based on the power grid infrastructure engineering data, integrate power grid GIS platform data and equipment ledger to construct a dynamic knowledge graph of power grid engineering data, which includes entity set E, attribute set R, and relationship set A. The entity set E includes power grid equipment, the attribute set R includes equipment attributes, and the relationship set A is the relationship between power grid equipment in the entity layer. The linking unit deeply integrates the dynamic knowledge graph of power grid engineering data with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base. The implicit features include the weights and vector space of LLM. The optimization unit optimizes the initial power grid infrastructure engineering data knowledge base based on retrieval-enhanced generation (RAG) to obtain a power grid infrastructure engineering data knowledge base. Where E={e1,e2,...,e n}, covering primary equipment, secondary devices, infrastructure procedures, survey indicators, cost items, and specification clauses; R={r1,r2,...,r m}, including topological relationships, composition relationships, dependency relationships, constraint relationships, and compliance relationships; A={a1,a2,...,a...} k} is used to describe key information such as equipment parameters, engineering indicators, and technical limits, where n, m, and k are all integers ≥ 2.

[0018] Furthermore, the operation of the optimization unit is as follows: after the LLM generates an answer to the user's question based on the initial power grid infrastructure engineering data knowledge base, it compares the generated question and answer with the retrieved standard data. If the generated question and answer is inconsistent with the standard data, the initial power grid infrastructure engineering data knowledge base is updated using the standard data to obtain the power grid infrastructure engineering data knowledge base.

[0019] Furthermore, in the cleaning unit: at the input layer of the LLM, a differentiated entity-aware embedding is performed between the general token and the power grid engineering entity token.

[0020] Furthermore, in the linking unit, graph attention is used to jointly encode the association information of entity context and dynamic knowledge graph of power grid engineering data to perform entity disambiguation, mapping entities to symbols of dynamic knowledge graph of power grid engineering data, embedding the symbols into entities and projecting the implicit features of LLM context to semantic space, and achieving deep fusion through alignment constraints and dynamic gating to obtain the initial knowledge base of power grid infrastructure engineering data.

[0021] Furthermore, the operation of projecting the implicit features of the LLM context onto the semantic space is as follows: The implicit features output by the LLM... h LLM Cross-space alignment is achieved by mapping to the same-dimensional space as the symbol embedding via a low-rank adaptation (LoRA) projection layer:

[0022] in, e c For the semantic feature vector of the LLM context embedding of symbols mapped to the same dimensional space as the symbol embedding,W proj It is a low-rank projection matrix. b It is a constant. While retaining the semantic capabilities of LLM, it enables the transformation from a general representation to a professional representation for power grid infrastructure.

[0023] Furthermore, to fine-tune the LLM and reduce the distributional discrepancy between symbolic embeddings and contextual representations, an aligned orthogonal joint loss function is constructed: ; in, This represents the distance between the same entity in two spaces. This represents the orthogonal constraint loss, which makes entities spatially independent. For weights.

[0024] Furthermore, the feature vector ultimately fused by the dynamic gating... e final The calculation method is as follows:

[0025] in, This represents the gated generation function. Indicates dynamic gating weights, Will The value is normalized.

[0026] The present invention also proposes a computer-readable storage medium storing computer program code, which, when executed by a computer, performs any of the methods described above.

[0027] The technical effect of this invention is as follows: This invention provides a method, apparatus, and storage medium for constructing a knowledge base for power grid infrastructure engineering data based on LLM. The method includes: a cleaning step S101, using LLM to extract power grid infrastructure engineering data from documents such as the "Specification for Calculation of Engineering Quantities in Power Grid Construction Projects," the "Electric Power Safety Work Regulations," and a large amount of journal literature in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes, and equipment attribute values; and a knowledge graph construction step S102, integrating power grid GIS platform data and equipment ledgers based on the power grid infrastructure engineering data to construct a knowledge graph containing entity set E, attributes, and other data. A dynamic knowledge graph of power grid engineering data, consisting of entity set E (containing power grid equipment), attribute set R (containing equipment attributes), and relation set A (relationships between power grid equipment in the entity layer), is generated. In step S103, the dynamic knowledge graph of power grid engineering data is deeply integrated with the connection layer of the LLM (Limited Link Model) to generate an initial knowledge base for power grid infrastructure engineering data. The implicit features include the weights and vector space of the LLM. In step S104, the initial knowledge base for power grid infrastructure engineering data is optimized based on Retrieval Enhanced Generation (RAG) to obtain a complete knowledge base for power grid infrastructure engineering data. This invention achieves a deep fusion of "symbolicism" and "connectivity" to generate an initial knowledge base for power grid infrastructure engineering data, and then optimizes the initial knowledge base based on Retrieval Enhanced Generation (RAG) to obtain a complete knowledge base for power grid infrastructure engineering data. The resulting knowledge base is relatively accurate and can be used for daily knowledge-based Q&A related to power grid infrastructure engineering. Attached Figure Description

[0028] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart of a method for constructing a knowledge base for power grid infrastructure data based on an embodiment of the present invention.

[0030] Figure 2 This is a structural diagram of a device for constructing a knowledge base for power grid infrastructure engineering data based on an embodiment of the present invention. Detailed Implementation

[0031] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0032] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] Figure 1 This invention illustrates a method for constructing a knowledge base for power grid infrastructure engineering data based on LLM (Liquid Module Management). The method includes: In the cleaning step S101, LLM is used to extract power grid infrastructure engineering data from the "Specification for Calculation of Engineering Quantities of Power Grid Construction Projects", "Electric Power Safety Work Regulations" and a large number of journal articles in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes and equipment attribute values. Knowledge graph construction step S102: Based on the power grid infrastructure engineering data, integrate the power grid GIS platform data and equipment ledger to construct a dynamic knowledge graph of power grid engineering data containing entity set E, attribute set R, and relationship set A. The entity set E contains power grid equipment, the attribute set R includes equipment attributes, and the relationship set A is the relationship between power grid equipment in the entity layer. In step S103, the dynamic knowledge graph of power grid engineering data is deeply integrated with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base. The implicit features include the weights and vector space of LLM. In optimization step S104, the initial power grid infrastructure engineering data knowledge base is optimized based on retrieval enhancement generation (RAG) to obtain the power grid infrastructure engineering data knowledge base. Where E={e1,e2,...,e n}, covering primary equipment, secondary devices, infrastructure procedures, survey indicators, cost items, and specification clauses; R={r1,r2,...,r m}, including topological relationships, composition relationships, dependency relationships, constraint relationships, and compliance relationships; A={a1,a2,...,a...} k} is used to describe key information such as equipment parameters, engineering indicators, and technical limits, where n, m, and k are all integers ≥ 2.

[0034] To address the technical problem in the background art where general large-scale models have biases in their understanding of complex entities like power grid infrastructure involving the physical world, engineering specifications, and geographical information, easily leading to "illusions," this invention creatively proposes the following technical concept: After cleaning power grid engineering data using an LLM model, a dynamic knowledge graph of power grid engineering data is first constructed, which integrates heterogeneous data such as equipment ledgers, engineering topology, design standards, and cost quotas. Then, power grid entities (such as transformers, towers, and lines) are linked with the internal representations of the LLM, achieving a deep fusion of "symbolicism" and "connectivity" to generate an initial power grid infrastructure engineering data knowledge base. This initial knowledge base is then optimized based on Retrieval Enhanced Generation (RAG) to obtain a more accurate power grid infrastructure engineering data knowledge base, making it suitable for daily knowledge-based Q&A in power grid infrastructure engineering. This is one of the key inventive concepts of this invention.

[0035] In one embodiment, the optimization step S104 is performed as follows: after the LLM generates an answer to the user's question based on the initial power grid infrastructure engineering data knowledge base, the generated question and answer are compared with the retrieved standard data. If the generated question and answer are inconsistent with the standard data, the initial power grid infrastructure engineering data knowledge base is updated using the standard data to obtain the power grid infrastructure engineering data knowledge base.

[0036] In this invention, in order to further reduce the illusion of the initial power grid infrastructure engineering data knowledge base generated by LLM, optimization is performed based on RAG technology alignment, so that the updated power grid infrastructure engineering data knowledge base is more accurate. This is another important inventive concept of this invention.

[0037] In one embodiment, in the cleaning step S101: at the input layer of the LLM, the general token and the power grid engineering entity token are embedded using differentiated entity-aware embedding. This invention enables the LLM to distinguish between "general natural language" and "power grid engineering language" during attention calculation by embedding the general token and the power grid engineering entity token using differentiated entity-aware embedding at the input layer of the LLM, thus eliminating semantic confusion. This is another important inventive concept of this invention.

[0038] In one embodiment, in the linking step S103, graph attention is used to jointly encode the association information of entity context and dynamic knowledge graph of power grid engineering data to perform entity disambiguation, mapping entities to symbols of dynamic knowledge graph of power grid engineering data, embedding the symbols into entities and projecting the implicit features of LLM context to semantic space, and achieving deep fusion through alignment constraints and dynamic gating to obtain the initial knowledge base of power grid infrastructure engineering data.

[0039] In this invention, an enhanced algorithm for entity linking and embedding is specifically designed to address the dense use of technical terminology in power grid infrastructure. This algorithm first uses graph attention to jointly encode the association information between entity context and the dynamic knowledge graph of power grid engineering data for entity disambiguation. Then, it maps entities to symbols in the dynamic knowledge graph of power grid engineering data. Finally, it embeds these symbols into entities and projects the implicit features of the LLM context onto the semantic space. Through alignment constraints and dynamic gating, deep fusion is achieved to obtain an initial knowledge base for power grid infrastructure engineering data. This algorithm achieves deep fusion of explicit knowledge at the symbolic level (such as power grid ontology, attributes, and relationships) and implicit representations at the connection level (LLM weights, vector space), thereby eliminating semantic ambiguity, improving entity matching accuracy, and addressing the lack of reasoning ability in traditional symbolic AI and the lack of logical interpretability in traditional LLM. This algorithm establishes a one-to-one correspondence between the distributed representations of LLM and explicit logical rules through entity linking, solving the "symbol grounding" problem of LLM and improving the accuracy of the power grid infrastructure engineering data knowledge base as a question-and-answer knowledge base. This is another important inventive concept of this invention.

[0040] In one embodiment, the operation of embedding the symbols into entities involves: pre-training the dynamic knowledge graph of power grid engineering data using the TransE algorithm, mapping symbol entities and relationships into low-dimensional dense vectors, and obtaining structured feature vectors for symbol entity embedding with logical constraints. e s : ; in, G R A graph showing the relationships between entities. G R =( E, R, E ).

[0041] This invention creatively pre-trains the existing TransE algorithm on the dynamic knowledge graph of power grid engineering data, thereby mapping symbolic entities and relations into low-dimensional dense vectors to obtain structured feature vectors with logically constrained symbolic entity embeddings. e s This invention achieves symbolic mapping of entities, thereby enabling each entity in a power grid project to be mapped to a corresponding symbol, thus facilitating reasoning and calculation. This is one of the inventive aspects of the present invention.

[0042] In one embodiment, the operation of projecting the implicit features of the LLM context onto the semantic space is: projecting the implicit features output by the LLM... h LLM Cross-space alignment is achieved by mapping to the same-dimensional space as the symbol embedding via a low-rank adaptation (LoRA) projection layer:

[0043] in, e c For the semantic feature vector of the LLM context embedding of symbols mapped to the same dimensional space as the symbol embedding, W proj It is a low-rank projection matrix. b It is a constant.

[0044] In this invention, by projecting the implicit features of the LLM context into the semantic space, the general representation is transformed into a professional representation of power grid infrastructure while retaining the semantic capabilities of the LLM. This is one of the inventive points of this invention.

[0045] In one embodiment, to fine-tune the LLM and reduce the distributional discrepancy between symbolic embeddings and contextual representations, an aligned orthogonal joint loss function is constructed: ; in, This represents the distance between the same entity in two spaces. This represents the orthogonal constraint loss, which makes entities spatially independent. The weights can be set empirically, such as 0.3 and 0.7 respectively, or they can be obtained by fitting data based on historical training data. i, j Each of the following is an integer greater than 1.

[0046] In order to better fine-tune the LLM to match the constructed power grid infrastructure engineering data knowledge base and form a dedicated LLM for the power grid infrastructure engineering field, the above-mentioned aligned orthogonal joint loss function was specially constructed. This function takes into account the distance and orthogonal constraint loss of the same entity of the power grid in two spaces in a unified manner, so that the fine-tuned LLM is more in line with power grid infrastructure engineering and the reasoning is more accurate. This is another inventive point of this invention.

[0047] In one embodiment, the dynamically gated final fused feature vector e final The calculation method is as follows:

[0048] in, This represents the gated generation function. Indicates dynamic gating weights, Will The value is normalized.

[0049] In this invention, dynamic gating fusion is used. When the input is strong logical text such as standard clauses or equipment parameters, the gating biases towards symbol embedding to ensure the compliance and accuracy of the output. When the input is flexible text such as natural language queries or construction descriptions, the gating biases towards LLM context representation to improve the fluency of understanding and generalization ability, thereby making the reasoning more accurate. This is another inventive point of this invention.

[0050] Figure 2 This invention illustrates a device for constructing a knowledge base for power grid infrastructure data based on LLM (Liquid Module Management). The device includes: Cleaning unit 201 uses LLM to extract power grid infrastructure engineering data from the "Specification for Calculation of Engineering Quantities of Power Grid Construction Projects", "Electric Power Safety Work Regulations" and a large number of journal articles in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes and equipment attribute values. Knowledge Graph Construction Unit 202: Based on the power grid infrastructure engineering data, integrate the power grid GIS platform data and equipment ledger to construct a dynamic knowledge graph of power grid engineering data, which includes entity set E, attribute set R, and relationship set A. The entity set E includes power grid equipment, the attribute set R includes equipment attributes, and the relationship set A is the relationship between power grid equipment in the entity layer. Linking unit 203 deeply integrates the dynamic knowledge graph of power grid engineering data with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base. The implicit features include the weights and vector space of LLM. Optimization unit 204 optimizes the initial power grid infrastructure engineering data knowledge base based on retrieval enhancement generation (RAG) to obtain a power grid infrastructure engineering data knowledge base; Where E={e1,e2,...,e n}, covering primary equipment, secondary devices, infrastructure procedures, survey indicators, cost items, and specification clauses; R={r1,r2,...,r m}, including topological relationships, composition relationships, dependency relationships, constraint relationships, and compliance relationships; A={a1,a2,...,a...} k} is used to describe key information such as equipment parameters, engineering indicators, and technical limits, where n, m, and k are all integers ≥ 2.

[0051] To address the technical problem in the background art where general large-scale models have biases in their understanding of complex entities like power grid infrastructure involving the physical world, engineering specifications, and geographical information, easily leading to "illusions," this invention creatively proposes the following technical concept: After cleaning power grid engineering data using an LLM model, a dynamic knowledge graph of power grid engineering data is first constructed, which integrates heterogeneous data such as equipment ledgers, engineering topology, design standards, and cost quotas. Then, power grid entities (such as transformers, towers, and lines) are linked with the internal representations of the LLM, achieving a deep fusion of "symbolicism" and "connectivity" to generate an initial power grid infrastructure engineering data knowledge base. This initial knowledge base is then optimized based on Retrieval Enhanced Generation (RAG) to obtain a more accurate power grid infrastructure engineering data knowledge base, making it suitable for daily knowledge-based Q&A in power grid infrastructure engineering. This is one of the key inventive concepts of this invention.

[0052] In one embodiment, the operation of the optimization unit 204 is as follows: after the LLM generates an answer to the user's question based on the initial power grid infrastructure engineering data knowledge base, it compares the generated question and answer with the retrieved standard data. If the generated question and answer is inconsistent with the standard data, it updates the initial power grid infrastructure engineering data knowledge base using the standard data to obtain the power grid infrastructure engineering data knowledge base.

[0053] In this invention, in order to further reduce the illusion of the initial power grid infrastructure engineering data knowledge base generated by LLM, optimization is performed based on RAG technology alignment, so that the updated power grid infrastructure engineering data knowledge base is more accurate. This is another important inventive concept of this invention.

[0054] In one embodiment, in the cleaning unit 201: at the input layer of the LLM, a differentiated entity-aware embedding is performed between the general token and the power grid engineering entity token. This invention enables the LLM to distinguish between "general natural language" and "power grid engineering language" during attention calculation by performing differentiated entity-aware embedding between the general token and the power grid engineering entity token at the input layer of the LLM, thus eliminating semantic confusion. This is another important inventive concept of this invention.

[0055] In one embodiment, in the linking unit 203, graph attention is used to jointly encode the association information of entity context and dynamic knowledge graph of power grid engineering data to perform entity disambiguation, mapping entities to symbols of dynamic knowledge graph of power grid engineering data, embedding the symbols into entities and projecting the implicit features of LLM context to semantic space, and achieving deep fusion through alignment constraints and dynamic gating to obtain the initial knowledge base of power grid infrastructure engineering data.

[0056] In this invention, an enhanced algorithm for entity linking and embedding is specifically designed to address the dense use of technical terminology in power grid infrastructure. This algorithm first uses graph attention to jointly encode the association information between entity context and the dynamic knowledge graph of power grid engineering data for entity disambiguation. Then, it maps entities to symbols in the dynamic knowledge graph of power grid engineering data. Finally, it embeds these symbols into entities and projects the implicit features of the LLM context onto the semantic space. Through alignment constraints and dynamic gating, deep fusion is achieved to obtain an initial knowledge base for power grid infrastructure engineering data. This algorithm achieves deep fusion of explicit knowledge at the symbolic level (such as power grid ontology, attributes, and relationships) and implicit representations at the connection level (LLM weights, vector space), thereby eliminating semantic ambiguity, improving entity matching accuracy, and addressing the lack of reasoning ability in traditional symbolic AI and the lack of logical interpretability in traditional LLM. This algorithm establishes a one-to-one correspondence between the distributed representations of LLM and explicit logical rules through entity linking, solving the "symbol grounding" problem of LLM and improving the accuracy of the power grid infrastructure engineering data knowledge base as a question-and-answer knowledge base. This is another important inventive concept of this invention.

[0057] In one embodiment, the operation of embedding the symbols into entities involves: pre-training the dynamic knowledge graph of power grid engineering data using the TransE algorithm, mapping symbol entities and relationships into low-dimensional dense vectors, and obtaining structured feature vectors for symbol entity embedding with logical constraints. e s : ; in, G R A graph showing the relationships between entities. G R =( E, R, E ).

[0058] This invention creatively pre-trains the existing TransE algorithm on the dynamic knowledge graph of power grid engineering data, thereby mapping symbolic entities and relations into low-dimensional dense vectors to obtain structured feature vectors with logically constrained symbolic entity embeddings. e s This invention achieves symbolic mapping of entities, thereby enabling each entity in a power grid project to be mapped to a corresponding symbol, thus facilitating reasoning and calculation. This is one of the inventive aspects of the present invention.

[0059] In one embodiment, the operation of projecting the implicit features of the LLM context onto the semantic space is: projecting the implicit features output by the LLM... h LLM Cross-space alignment is achieved by mapping to the same-dimensional space as the symbol embedding via a low-rank adaptation (LoRA) projection layer:

[0060] in, e c For the semantic feature vector of the LLM context embedding of symbols mapped to the same dimensional space as the symbol embedding, W proj It is a low-rank projection matrix. b It is a constant.

[0061] In this invention, by projecting the implicit features of the LLM context into the semantic space, the general representation is transformed into a professional representation of power grid infrastructure while retaining the semantic capabilities of the LLM. This is one of the inventive points of this invention.

[0062] In one embodiment, to fine-tune the LLM and reduce the distributional discrepancy between symbolic embeddings and contextual representations, an aligned orthogonal joint loss function is constructed: ; in, This represents the distance between the same entity in two spaces. This represents the orthogonal constraint loss, which makes entities spatially independent. The weights can be set empirically, such as 0.3 and 0.7 respectively, or they can be obtained by fitting data based on historical training data. i, j Each of the following is an integer greater than 1.

[0063] In order to better fine-tune the LLM to match the constructed power grid infrastructure engineering data knowledge base and form a dedicated LLM for the power grid infrastructure engineering field, the above-mentioned aligned orthogonal joint loss function was specially constructed. This function takes into account the distance and orthogonal constraint loss of the same entity of the power grid in two spaces in a unified manner, so that the fine-tuned LLM is more in line with power grid infrastructure engineering and the reasoning is more accurate. This is another inventive point of this invention.

[0064] In one embodiment, the dynamically gated final fused feature vector e final The calculation method is as follows:

[0065] in, This represents the gated generation function. Indicates dynamic gating weights, Will The value is normalized.

[0066] In this invention, dynamic gating fusion is used. When the input is strong logical text such as standard clauses or equipment parameters, the gating biases towards symbol embedding to ensure the compliance and accuracy of the output. When the input is flexible text such as natural language queries or construction descriptions, the gating biases towards LLM context representation to improve the fluency of understanding and generalization ability, thereby making the reasoning more accurate. This is another inventive point of this invention.

[0067] One embodiment of the present invention provides a computer storage medium storing a computer program. When the computer program on the computer storage medium is executed by a processor, the above-described method is implemented. The computer storage medium may be a hard disk, DVD, CD, flash memory, or other storage device.

[0068] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0069] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the apparatus described in various embodiments or some parts of the embodiments of this application.

[0070] Finally, it should be noted that the above embodiments are for illustration only and not for limiting the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention without departing from the spirit and scope of the present invention. Any modifications or partial substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for constructing a knowledge base for power grid infrastructure engineering data based on LLM, characterized in that, The method includes: The cleaning step involves using LLM to extract power grid infrastructure engineering data from the "Specification for Calculation of Engineering Quantities in Power Grid Construction Projects", "Electric Power Safety Work Regulations", and a large amount of journal literature in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes, and equipment attribute values. Knowledge graph construction steps: Based on the power grid infrastructure engineering data, integrate the power grid GIS platform data and equipment ledger to construct a dynamic knowledge graph of power grid engineering data, which includes entity set E, attribute set R, and relationship set A. The entity set E includes power grid equipment, the attribute set R includes equipment attributes, and the relationship set A is the relationship between power grid equipment in the entity layer. The linking step involves deeply integrating the dynamic knowledge graph of power grid engineering data with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base. The implicit features include the weights and vector space of LLM. The optimization step involves optimizing the initial power grid infrastructure engineering data knowledge base based on Retrieval Enhanced Generation (RAG) to obtain the power grid infrastructure engineering data knowledge base. Where E={e1,e2,...,e n }, covering primary equipment, secondary devices, infrastructure procedures, survey indicators, cost items, and specification clauses; R={r1,r2,...,r m }, including topological relationships, composition relationships, dependency relationships, constraint relationships, and compliance relationships; A={a1,a2,...,a...} k } is used to describe key information such as equipment parameters, engineering indicators, and technical limits, where n, m, and k are all integers ≥ 2.

2. The method according to claim 1, characterized in that, The optimization step is as follows: after LLM generates answers to user questions based on the initial power grid infrastructure engineering data knowledge base, it compares the generated questions and answers with the retrieved standard data. If the generated questions and answers are inconsistent with the standard data, the initial power grid infrastructure engineering data knowledge base is updated using the standard data to obtain the power grid infrastructure engineering data knowledge base.

3. The method according to claim 2, characterized in that, In the cleaning step: at the input layer of the LLM, the general token and the power grid engineering entity token are embedded in a differentiated entity-aware manner.

4. The method according to claim 3, characterized in that, In the linking step, graph attention is used to jointly encode the association information of entity context and dynamic knowledge graph of power grid engineering data to perform entity disambiguation, mapping entities to symbols of dynamic knowledge graph of power grid engineering data, embedding the symbols into entities and projecting the implicit features of LLM context to semantic space, and achieving deep fusion through alignment constraints and dynamic gating to obtain the initial knowledge base of power grid infrastructure engineering data.

5. The method according to claim 4, characterized in that, The operation of embedding the symbols into entities involves: pre-training the dynamic knowledge graph of power grid engineering data using the TransE algorithm, mapping symbol entities and relations into low-dimensional dense vectors, and obtaining structured feature vectors for symbol entity embedding with logical constraints. e s : ； in, G R A graph showing the relationships between entities. G R =( E, R, E ).

6. A device for constructing a knowledge base for power grid infrastructure engineering data based on LLM, characterized in that, The device includes: The cleaning unit uses LLM to extract power grid infrastructure engineering data from the "Specification for Calculation of Engineering Quantities in Power Grid Construction Projects", "Electric Power Safety Work Regulations" and a large amount of journal literature in the power grid field. The power grid infrastructure engineering data includes power grid equipment, equipment attributes and equipment attribute values. Knowledge Graph Construction Unit: Based on the power grid infrastructure engineering data, integrate power grid GIS platform data and equipment ledger to construct a dynamic knowledge graph of power grid engineering data, which includes entity set E, attribute set R, and relationship set A. The entity set E includes power grid equipment, the attribute set R includes equipment attributes, and the relationship set A is the relationship between power grid equipment in the entity layer. The linking unit deeply integrates the dynamic knowledge graph of power grid engineering data with the connection layer of LLM to generate an initial power grid infrastructure engineering data knowledge base. The implicit features include the weights and vector space of LLM. The optimization unit optimizes the initial power grid infrastructure engineering data knowledge base based on retrieval-enhanced generation (RAG) to obtain a power grid infrastructure engineering data knowledge base. Where E={e1,e2,...,e n }, covering primary equipment, secondary devices, infrastructure procedures, survey indicators, cost items, and specification clauses; R={r1,r2,...,r m }, including topological relationships, composition relationships, dependency relationships, constraint relationships, and compliance relationships; A={a1,a2,...,a...} k } is used to describe key information such as equipment parameters, engineering indicators, and technical limits, where n, m, and k are all integers ≥ 2.

7. The apparatus according to claim 6, characterized in that, The operation of the optimization unit is as follows: after LLM generates answers to user questions based on the initial power grid infrastructure engineering data knowledge base, it compares the generated questions and answers with the retrieved standard data. If the generated questions and answers are inconsistent with the standard data, the initial power grid infrastructure engineering data knowledge base is updated using the standard data to obtain the power grid infrastructure engineering data knowledge base.

8. The apparatus according to claim 7, characterized in that, In the cleaning unit: at the input layer of the LLM, a differentiated entity-aware embedding is performed between the general token and the power grid engineering entity token.

9. The apparatus according to claim 8, characterized in that, In the linking unit, graph attention is used to jointly encode the association information of entity context and dynamic knowledge graph of power grid engineering data to perform entity disambiguation, mapping entities to symbols of dynamic knowledge graph of power grid engineering data, embedding the symbols into entities and projecting the implicit features of LLM context to semantic space, and achieving deep fusion through alignment constraints and dynamic gating to obtain the initial knowledge base of power grid infrastructure engineering data.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

Images