Bridge management and maintenance knowledge graph representation method based on large language model collaborative graph neural network

Abstract

The invention belongs to the technical field of intelligent models, and particularly relates to a bridge management and maintenance knowledge graph representation method based on a large language model collaborative graph neural network, which comprises the following steps: S1, converting an OWL ontology into a knowledge graph RDF graph of a triple, generating a plurality of sub-graphs with different relationships according to a preset migration scheme, and turning to S2; s3, adopting a preset hybrid walk strategy, starting from a plurality of entities of each sub-graph, generating an entity sequence based on a specific step number, and turning to S3; constructing an inverse relation and a self-loop relation on the basis of the initial RDF graph to form a new RDF graph, and turning to S4; and S5, carrying out splicing and feature conversion on the local structure embedding, the semantic embedding and the global structure embedding obtained in the steps S2-S4 to obtain an embedded representation of the fused entity and relationship. According to the method, complex semantic information, text information and structure information in the OWL ontology can be fully utilized, automatic completion and link prediction of knowledge are realized, and the accuracy and integrity of the bridge management and maintenance knowledge graph are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent models, and in particular to a bridge maintenance knowledge graph representation method of a large language model collaborative graph neural network. Background Art

[0002] By the end of 2023, the number of highway bridges in my country has reached an astonishing 1.0793 million, including 10,239 super-large bridges and 177,700 large bridges, which constitute an important part of my country's highway infrastructure network. The safety and stability of bridges are not only related to the smooth flow of traffic, but also directly related to the safety of people's lives and property. Therefore, bridge structure health monitoring has become a key link in ensuring bridge safety. It provides a strong guarantee for the safe operation of bridges by real-time monitoring of the operation status of bridges, timely discovering and warning potential problems. However, with the rapid growth of the number of bridges and the diversification of monitoring methods, the processing and analysis of bridge health monitoring data face unprecedented challenges. Traditional methods have many shortcomings in data processing, analysis methods, and monitoring information fusion, which makes it difficult to improve monitoring efficiency and management level. Especially in the context of the big data era, how to extract valuable information from massive data and realize intelligent monitoring and management has become a problem that needs to be solved urgently.

[0003] The era of big data has promoted the transformation of knowledge engineering to big data knowledge engineering. As a typical representative of this new type of knowledge engineering, knowledge graph provides a new solution for the field of bridge management and maintenance. The bridge management and maintenance knowledge graph stores a wealth of basic bridge information, health monitoring information, detection information, and time information, providing strong data support for intelligent bridge management and maintenance. However, the bridge management and maintenance knowledge graph also faces the problem of information loss. Manual information completion is not only inefficient and costly, but also has uneven quality and is difficult to meet actual needs.

[0004] To address this problem, although existing knowledge graph representation learning methods have achieved automatic knowledge completion and link prediction to a certain extent, there are still many shortcomings. As the basic construction tool of the bridge maintenance knowledge graph, OWL (Web Ontology Language) ontology contains a large amount of complex semantic information, text information and structural information. However, existing representation learning methods often ignore the comprehensive utilization of this information, resulting in poor representation learning effects. At the same time, although the representation learning method based on subgraph classification can extract some structural information, it is easy to lose logical information and it is difficult to accurately represent the relationship between each triple. In addition, due to the limited node link length, these methods also have great difficulties in representing unseen entities and long-tail relationships.

[0005] The reason is that the existing representation learning methods fail to fully consider the complexity and diversity when processing the bridge maintenance knowledge graph. The bridge maintenance knowledge graph not only contains rich structural information, but also involves a large amount of semantic information and text information. These information are intertwined and influence each other, forming a complex network of knowledge graphs. Therefore, how to effectively integrate this information and achieve more accurate and comprehensive representation learning has become a major challenge currently faced.

[0006] Therefore, how to make full use of the complex semantic information, text information and structural information in the OWL ontology to achieve automatic knowledge completion and link prediction, and improve the accuracy and completeness of the bridge maintenance knowledge graph has become an urgent problem to be solved. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a bridge maintenance knowledge graph representation method based on a large language model and a collaborative graph neural network, which can make full use of the complex semantic information, text information and structural information in the OWL ontology to achieve automatic knowledge completion and link prediction, and improve the accuracy and completeness of the bridge maintenance knowledge graph.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network includes the following steps:

[0010] S1, convert the OWL ontology into a triple knowledge graph RDF graph through the ontology mapping rules, and generate multiple subgraphs with different relationships according to the preset walking scheme, and go to S2; adopt the preset mixed walking strategy, starting from multiple entities in each subgraph, generate entity sequences based on a specific number of steps, and go to S3; construct inverse relationships and self-loop relationships based on the initial RDF graph to form a new RDF graph, and go to S4;

[0011] S2, use the graph structure encoder to encode each subgraph to capture the local structural features of entities and relations, and perform average pooling on the representations of the same entity and relationship to obtain the local structural embedding of the entity and relationship; then go to S5;

[0012] S3, map the instance token of the entity sequence to the description text corresponding to the entity; use the large language model as the entity text encoder to convert the obtained description text into the initial semantic embedding; and go to S5;

[0013] S4, use the graph structure encoder to perform graph convolution operation on the new RDF graph, extract the global structure embedding of entities and relationships, and go to S5;

[0014] S5, concatenate the local structure embedding, semantic embedding and global structure embedding obtained in S2-S4 to form a unified representation, and then perform further feature transformation through a linear layer to obtain the fused embedding representation of entities and relations;

[0015] S6, optimizing the embedded representation of the fused entities and relations obtained in S5 to obtain the final representation of the entities and relations;

[0016] S7. Use the final representation of entities and relationships obtained in S6 for subsequent practical applications.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. Comprehensive information utilization and representation learning. Traditional methods often only focus on the structural information of the knowledge graph, ignoring the value of semantic and textual information. Compared with the existing technology, this method not only considers the structural information in the bridge maintenance knowledge graph, but also makes full use of the complex semantic information and textual information in the OWL ontology. Through the synergy of the graph structure encoder and the large language model, multi-dimensional representation learning of entities and relationships is achieved. By integrating this information, this method improves the accuracy and comprehensiveness of representation learning.

[0019] 2. Capturing subgraphs and global structural features. This method effectively captures the local structural features of entities and relationships by generating multiple subgraphs of different relationships and encoding each subgraph using a graph structure encoder. At the same time, a graph convolution operation is performed on the new RDF graph to extract the global structural embedding. Compared with methods based only on the global graph or a single subgraph, this method can capture the local and global structural features in the knowledge graph in a more detailed manner and improve the sophistication of representation learning.

[0020] 3. Fusion of semantic embedding and text information. This method uses a large language model to convert the description text of the entity sequence into a semantic embedding, and splices it with the local structure embedding and the global structure embedding to form a unified representation. This fusion method makes the representation of entities and relationships richer and more accurate. Traditional methods usually only consider structural information or simple text information, while this method achieves a deep understanding and utilization of text information through a large language model, improving the semantic richness of representation learning.

[0021] 4. Optimization and application of final representation. This method optimizes the embedded representation of fused entities and relations to obtain a more accurate and robust final representation. These final representations can be directly used in subsequent practical applications, such as knowledge reasoning, entity linking, relationship prediction, etc. Through the optimization steps, this method can further improve the quality of representation learning and provide more reliable and effective support for practical applications.

[0022] In summary, this method can make full use of the complex semantic information, text information and structural information in the OWL ontology, and realize the multi-dimensional representation learning of entities and relationships through the synergy of the graph structure encoder and the large language model. By capturing local and global structural features, integrating semantic embedding and text information, and performing optimization processing, this method obtains a more accurate and complete bridge maintenance knowledge graph representation. These representations can be directly used in subsequent practical applications, such as knowledge reasoning, entity linking, relationship prediction, etc., thereby improving the accuracy and completeness of the bridge maintenance knowledge graph.

[0023] Preferably, in S1, the subgraphs of different relationships include a class hierarchy subgraph, a class object attribute subgraph, a class instance subgraph, an instance object attribute subgraph and an instance data attribute subgraph.

[0024] This setting, by decomposing the knowledge graph into multiple subgraphs with specific relationships, each subgraph can more finely represent a certain type of information in the OWL ontology. For example, the class hierarchy subgraph can clearly show the inheritance relationship between classes, while the class object property subgraph can reveal the association between the class and its object properties. This refined representation helps to more accurately capture the characteristics of entities and relationships, and provide more valuable information for subsequent graph structure encoding and semantic embedding.

[0025] Since each subgraph focuses on a certain type of relationship, the features of that type of relationship can be extracted more accurately when encoding the subgraph. This highly targeted encoding method helps improve the accuracy of representation learning. In addition, by averaging the representations of the same entity and relationship in multiple subgraphs, noise and redundant information can be further reduced, improving the robustness of the final representation.

[0026] Preferably, in S1, the process of generating a plurality of subgraphs with different relationships according to a preset walking scheme includes:

[0027] Randomly select entities with current relationship links as the starting point, and walk along the current relationship to generate multiple subgraphs:

[0028] First, for the relationship rdfs:subClassOf, the axioms are converted into RDF triples according to the ontology mapping rules, the head entity and the tail entity are restricted to concept classes, and all triples with subClassOf relationships are extracted to form a class hierarchy subgraph;

[0029] Secondly, according to the object attribute relationship, the head entity and the tail entity are restricted to the concept class, and complex axiom constraints are added to extract all triples that meet the conditions except the subClassOf relationship to form the class object attribute subgraph;

[0030] Thirdly, extract the types of all instances according to the rdf:type relationship, and combine the triples containing rdf:typ into a class instance subgraph;

[0031] Then, all triples whose relations are object attributes and whose head and tail entities are instances are extracted to form an instance object attribute subgraph;

[0032] Finally, all triples whose relations are data attributes and whose tail entities are literals are extracted to form an instance data attribute subgraph.

[0033] This setting, 1. Relationship segmentation and clarification. By segmenting the relationships in the knowledge graph into class hierarchy relationships, class object attribute relationships, instance type relationships, instance object attribute relationships, and data attribute relationships, this process makes each type of relationship clearly reflected in the subgraph. This segmentation helps to more clearly understand the entities and relationships in the knowledge graph, and facilitates subsequent analysis and processing.

[0034] 2. Improve the accuracy of representation learning. Each subgraph focuses on a certain type of relationship, so that when encoding and representing the subgraph, the characteristics of this type of relationship can be extracted more accurately. This highly targeted processing method helps to improve the accuracy of representation learning and provides a more reliable foundation for subsequent practical applications.

[0035] 3. Enhance the integrity and interpretability of the knowledge graph. By generating multiple subgraphs with different relationships, this process can more comprehensively cover all types of information in the knowledge graph. This helps to make up for the information loss problem that may exist in a single global graph and enhance the integrity of the knowledge graph. At the same time, the generation process of each subgraph also provides a detailed explanation of each type of relationship, enhancing the interpretability of the knowledge graph.

[0036] Preferably, in S1, the preset hybrid wandering strategy includes:

[0037] Based on the five subgraphs, the entities with the highest connectivity are selected as the starting points of regular walks and random walks respectively. If the highest connectivity is the same, the entity with the highest connectivity is randomly selected as the starting point;

[0038] For the class hierarchy subgraph, rules are formulated to perform specific walks in specific directions, and preset random walks are introduced;

[0039] Among them, specific walk steps, regular walk steps, and random walk steps are all hyperparameters.

[0040] Such a setting can achieve efficient traversal and sampling. By selecting the entity with the highest connectivity as the starting point of the walk, the strategy can prioritize the traversal of the core nodes in the knowledge graph, thereby more effectively covering the important information in the graph. At the same time, the introduction of random walks increases the randomness and diversity of the traversal, which helps to avoid falling into the local optimal solution.

[0041] 2. For the class hierarchy subgraph, specific walking rules in specific directions are formulated to more accurately capture the hierarchical and inheritance relationships between classes. This highly targeted walking method helps to improve the accuracy and depth of representation learning, and provides a more reliable basis for subsequent analysis and processing. The specific walking steps, regular walking steps, and random walking steps are all used as hyperparameters and can be adjusted according to the characteristics of specific tasks and data sets. This adjustability enables the strategy to adapt more flexibly to different application scenarios and needs, improving its versatility and practicality.

[0042] 3. Balance global and local information. By combining regular walks and random walks, the strategy is able to strike a balance between global traversal and local depth. Regular walks help capture global structures and relationships, while random walks can explore more local information and details. This balance helps improve the comprehensiveness and accuracy of representation learning.

[0043] Preferably, the CompGCN model of the graph neural network GNN is used as the graph structure encoder.

[0044] This setting, using the CompGCN model of the graph neural network GNN as the graph structure encoder, can efficiently process multi-relational graphs, enhance the model's expressiveness, optimize the representation learning process, improve the actual application effect, and promote the development of GNN technology. It has broad application prospects and important research value in the fields of knowledge graphs, social network analysis, and recommendation systems.

[0045] Preferably, in S3, when mapping the instance token of the entity sequence to the description text corresponding to the entity, if the entity has multiple annotation attribute information, the annotation attribute is selected as the text representative according to the preset standard; if the entity has no annotation attribute, the word representative containing semantic information in the entity URI is used as the text representative.

[0046] This setting, 1. Improves information accuracy. When an entity has multiple annotation attribute information, selection based on preset standards can ensure that the selected annotation attribute can accurately and comprehensively represent the core characteristics of the entity. This selection method avoids information redundancy and misleading, and improves the accuracy of subsequent processing and analysis.

[0047] 2. Enhance the ability of semantic expression. Whether selecting annotation attributes or using semantic information words in entity URI, it is to better express the semantic information of the entity. This method makes the entity description more specific and vivid, which helps the subsequent tasks to accurately understand and process the entity semantics.

[0048] 3. Improve processing efficiency. Mapping entity descriptions through preset standards and rules can greatly simplify the processing process and reduce the time cost of manual intervention and judgment. This automated processing method not only improves processing efficiency, but also reduces the risk of human error.

[0049] Preferably, in S3, before using the large language model to convert the description text, the large language model is also trained through the Masked Language Model task to optimize and improve the semantic quality of text embedding.

[0050] In this setting, the MLM (Masked Language Model) task requires the model to predict masked or replaced words, which prompts the model to understand the contextual information of the text more deeply. Through training on a large number of MLM tasks, the large language model can learn the dependencies between words in a sentence, as well as the intrinsic structure and semantic information of the language, thereby significantly improving its language understanding ability.

[0051] Text embedding is the process of converting text into vector representation, and its quality directly affects the results of subsequent tasks. The large language model trained by the MLM task can generate higher quality text embeddings. These embedding vectors can more accurately capture the semantic features of the text and provide more valuable information for subsequent tasks.

[0052] The MLM task uses a large amount of different text data for training, which enables the large language model to learn a variety of language phenomena and expressions. Therefore, when faced with new, unseen text, the model can better generalize its learned knowledge and generate reasonable predictions or outputs.

[0053] Preferably, in S4, the process of extracting global structural embedding of entities and relationships from the new RDF graph includes:

[0054] For the initial RDF graph Add inverse relations and self-loop relations to the relationship set and obtain a new RDF graph by adding inverse relations and self-loop relations to the relationship set Combine the head entity and the relationship as shown below:

[0055]

[0056] in, is a composite operator; (h, r, t) represents the starting point, relationship type, and end point; ε represents the set of entities, represents a collection of relations, Represents a set of triples;

[0057] Perform convolution operation on the entity representation after the combination operation:

[0058]

[0059] Among them, W k is the parameter matrix for the relationship type, which is divided as follows:

[0060]

[0061] Transform the relation embedding as shown below:

[0062]

[0063] Among them, W rel is a learnable transformation matrix used to project all relation feature vectors into the same embedding space as node+ and allow them to be utilized in the next CompGCN layer;

[0064] Finally, the global structural embedding of entities and relations is obtained.

[0065] Glossary: ​​"Node+" refers to the comprehensive representation of entities and relationships in the embedding space. Specifically, when encoding the global graph structure, the model will combine the head entity and the relationship. The result of this combination represents the comprehensive information of the starting point, relationship type, and end point. Subsequently, this comprehensive information will be used for convolution operations and other processing steps to obtain the global structure embedding. In this process, "Node+" can be understood as a comprehensive node that contains entity and relationship information, which interacts and projects with the relationship feature vector in the embedding space.

[0066] This setting, 1. Enhances the model's ability to understand the global structure. By combining the head entity and the relationship and introducing the convolution operation, the model can capture the complex interactions between entities and relationships, thereby understanding the global structure more accurately. This global structure embedding helps the model better understand and utilize the information in the graph in subsequent tasks.

[0067] 2. Improve the expressiveness of relation embeddings. By transforming relation embeddings and projecting them into the same embedding space as nodes, the model can use relation information more flexibly. This transformation not only enhances the expressiveness of relation embeddings, but also makes relation embeddings more consistent with node embeddings, which helps improve the performance of subsequent tasks.

[0068] 3. Support for processing complex relationships. After introducing inverse relationships and self-loop relationships, the new RDF graph can more comprehensively represent the relationships between entities. This representation method helps the model process complex relationships, such as transitive relationships, inverse relationships, etc., so as to more accurately understand the entities and relationships in the graph.

[0069] Preferably, in S6, an L2 distance loss function is used to optimize the embedded representation of the fused entity and relationship obtained in S5.

[0070] This setting uses the L2 distance loss function to optimize the embedded representation of the fused entities and relations, which can improve the accuracy of the embedded representation, enhance the generalization ability of the model, optimize the computational efficiency and stability, promote the semantic consistency of the embedded representation, and support a variety of subsequent tasks. These effects together provide a strong guarantee for the performance of the model in practical applications.

[0071] Preferably, in S6, the expression of the L2 distance loss function is:

[0072]

[0073] In the formula, represents a new RDF graph; (h, r, t) is a positive triple, (h - ,r - ,t - ) is a negative example triplet; γ is the distance between positive and negative samples and is a hyperparameter.

[0074] In this setting, 1. The loss function is calculated by comparing the positive triplet (h, r, t) with the negative triplet (h - ,r - ,t - ) and introduces a positive-negative sample interval γ to distinguish positive and negative samples. This helps the model learn the correct associations between entities and relations while distinguishing those that do not exist.

[0075] 2. During the training process, the model continuously adjusts the embedding vectors of entities and relations to minimize the loss function. This will cause the distance between the embedding vectors of positive triples to decrease, while the distance between the embedding vectors of negative triples to increase, thereby optimizing the quality of the embedded representation.

[0076] 3. The introduction of negative example triplets and the distance between positive and negative samples γ allows the model to focus not only on positive example triplets during training, but also on negative example triplets that are mistakenly considered to be positive examples. This helps improve the robustness of the model in the face of noisy and abnormal data. The distance between positive and negative samples γ is a hyperparameter that can be adjusted according to the characteristics of specific tasks and data sets. This provides additional flexibility, allowing the model to adapt to different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0078] Figure 1 A flowchart of the method is shown in FIG.

[0079] Figure 2 Schematic diagram of ontology mapping rules in the embodiment. DETAILED DESCRIPTION

[0080] The following is a further detailed description through specific implementation methods:

[0081] Example:

[0082] Aiming at the knowledge graph representation learning method for the field of bridge maintenance, this method is based on the powerful semantic feature capture capability of the large language model and the powerful structural feature capture capability of the graph convolutional neural network. It integrates the complex semantic information, heterogeneous attribute information, descriptive text information, global graph structure information of the RDF graph, and local graph structure information after subgraph division in the ontology, so that the embedding results of entities and relationships contain richer semantic and structural information, and enhances the generalization ability of the model by constructing negative examples, so that it can better serve downstream tasks such as link prediction.

[0083] Compared with traditional methods, this method makes full use of the semantic and structural information in the ontology. First, the OWL ontology is mapped to an RDF (Resource Description Framework) graph through ontology mapping, and the inverse relationship and self-loop relationship of the RDF graph are constructed to form a new RDF graph, and multiple subgraphs are generated according to different relationships; secondly, for the semantic information part, starting from multiple entities in each subgraph, an entity sequence is generated with a specific number of steps based on a hybrid walk strategy, and the entity is represented by the description text in the ontology, and the initial text embedding is generated using a large language model; thirdly, CompGCN is used to encode the graph structure of each entity sequence, and CompGCN is used to encode the graph structure of the entire RDF graph; finally, the obtained embeddings of each entity (class, instance) and relationship (object attribute, data attribute) are spliced, and a comprehensive entity and relationship embedding is obtained through feature fusion, and finally the final embedding result is obtained through the designed loss function. The specific process of this method is as follows.

[0084] like Figure 1 As shown, this embodiment discloses a bridge maintenance knowledge graph representation method of a large language model collaborative graph neural network, including the following steps:

[0085] S1. Convert the OWL ontology into a triple-based knowledge graph RDF graph through ontology mapping rules (in specific implementation, the mapping method proposed by OWL2Vec* is adopted. The specific rules are as follows Figure 2 As shown in the figure, multiple subgraphs with different relationships are generated according to the preset walking scheme, and then go to S2. The preset mixed walking strategy is adopted, starting from multiple entities in each subgraph, entity sequences are generated based on a specific number of steps, and then go to S3. On the basis of the initial RDF graph, inverse relationships and self-loop relationships are constructed to form a new RDF graph, and then go to S4.

[0086] Among them, the subgraphs of different relationships include class hierarchy subgraphs, class object attribute subgraphs, class instance subgraphs, instance object attribute subgraphs, and instance data attribute subgraphs. In this way, by decomposing the knowledge graph into multiple subgraphs with specific relationships, each subgraph can more finely represent a certain type of information in the OWL ontology. For example, the class hierarchy subgraph can clearly show the inheritance relationship between classes, while the class object attribute subgraph can reveal the association between classes and their object attributes. This refined representation helps to more accurately capture the characteristics of entities and relationships, and provide more valuable information for subsequent graph structure encoding and semantic embedding. Since each subgraph focuses on a certain type of relationship, when encoding the subgraph, the characteristics of the class relationship can be more accurately extracted. This highly targeted encoding method helps to improve the accuracy of representation learning. In addition, by averaging the representations of the same entity and relationship in multiple subgraphs, noise and redundant information can be further reduced, and the robustness of the final representation can be improved.

[0087] The existing walking methods start from the entities of the ontology, walk based on specific relationships, and generate subgraphs with the same relationships. Although this method can fully explore the hierarchical relationships between entities, it may ignore other important relationships and entities in the knowledge graph. Therefore, this method combines specific walks, regular walks and random walks to form a hybrid walk scheme, which introduces a certain degree of contingency to mitigate other important relationships and entities ignored by the subgraph. First, a basic subgraph is generated based on a specific walk to fully explore the hierarchical relationship information between entities. Then, based on the generated subgraph, the entity with the most relationships is used as the starting point to perform a predefined regular walk, and a small number of random walks are performed on the remaining relationships except the current relationship. Examples of entities, attributes, and triples are shown in Table 1.

[0088] Table 1 Examples of entities, attributes and their related triples

[0089]

[0090] Specifically, the triples of the knowledge graph can be expressed as (h, r, t), which stands for (head entity, relation, tail entity). For example, (Chongqing Yangtze River Bridge, completion time, "1980") means that the Chongqing Yangtze River Bridge was completed in 1980.

[0091] In specific implementation, the process of generating multiple subgraphs with different relationships according to the preset walking scheme includes:

[0092] Randomly select entities linked by the current relationship r as the starting point, and walk along the current relationship r to generate multiple subgraphs:

[0093] First, for the relationship rdfs:subClassOf, the axioms are converted into RDF triples according to the ontology mapping rules, the head entity and the tail entity are restricted to concept classes, and all triples with subClassOf relationships are extracted to form a class hierarchy subgraph;

[0094] Secondly, according to the object attribute relationship, the head entity and the tail entity are restricted to the concept class, and complex axiom constraints are added to extract all triples that meet the conditions except the subClassOf relationship to form the class object attribute subgraph;

[0095] Thirdly, extract the types of all instances according to the rdf:type relationship, and combine the triples containing rdf:typ into a class instance subgraph;

[0096] Then, all triples whose relations are object attributes and whose head and tail entities are instances are extracted to form an instance object attribute subgraph;

[0097] Finally, all triples whose relations are data attributes and whose tail entities are literals are extracted to form an instance data attribute subgraph.

[0098] In this way, by subdividing the relationships in the knowledge graph into class hierarchy relationships, class object attribute relationships, instance type relationships, instance object attribute relationships, and data attribute relationships, this process makes each type of relationship clearly reflected in the subgraph. This subdivision helps to understand the entities and relationships in the knowledge graph more clearly, which facilitates subsequent analysis and processing. It can also improve the accuracy of representation learning. Each subgraph focuses on a certain type of relationship, so that when encoding and representing the subgraph, the characteristics of this type of relationship can be more accurately extracted. This highly targeted processing method helps to improve the accuracy of representation learning and provides a more reliable foundation for subsequent practical applications. In addition, by generating multiple subgraphs of different relationships, this process can more comprehensively cover various types of information in the knowledge graph. This helps to make up for the possible information loss problem of a single global graph and enhance the integrity of the knowledge graph. At the same time, the generation process of each subgraph also provides a detailed explanation of each type of relationship, enhancing the interpretability of the knowledge graph.

[0099] In specific implementation, the preset hybrid roaming strategies include:

[0100] Based on the five subgraphs, the entities with the highest connectivity are selected as the starting points of regular walks and random walks respectively. If the highest connectivity is the same, the entity with the highest connectivity is randomly selected as the starting point;

[0101] For the class hierarchy subgraph, rules are formulated to perform specific walks in specific directions, and preset random walks are introduced.

[0102] When implementing it, the specific rules are shown in Table 2.

[0103] Table 2 Rule completion and roaming scheme

[0104]

[0105] The specific walk steps, regular walk steps, and random walk steps are all set as hyperparameters. In order to avoid introducing too much randomness and thus failing to fully explore the hierarchical relationship between entities, the regular walk steps and random walk steps should not be set too large, so as to generate entity sequences that match the input of the large language model.

[0106] In this way, efficient traversal and sampling can be achieved. By selecting the entity with the highest connectivity as the starting point of the walk, the strategy can prioritize the traversal of the core nodes in the knowledge graph, thereby more effectively covering the important information in the graph. At the same time, the introduction of random walks increases the randomness and diversity of the traversal, which helps to avoid falling into the local optimal solution. In addition, for the class hierarchy subgraph, formulating specific walk rules in specific directions can more accurately capture the hierarchical relationship and inheritance relationship between classes. This highly targeted walk method helps to improve the accuracy and depth of representation learning, and provides a more reliable foundation for subsequent analysis and processing. The specific walk steps, the regular walk steps, and the random walk steps are all used as hyperparameters, which can be adjusted according to the characteristics of specific tasks and data sets. This adjustability enables the strategy to adapt to different application scenarios and needs more flexibly, improving its versatility and practicality. It can also balance global and local information. By combining regular walks and random walks, the strategy can strike a balance between global traversal and local depth. Regular walks help capture global structures and relationships, while random walks can explore more local information and details. This balance helps improve the comprehensiveness and accuracy of representation learning.

[0107] S2. Use a graph structure encoder to encode each subgraph to capture the local structural features of entities and relations, and perform average pooling on the representations of the same entity and relationship in multiple subgraphs or in the same subgraph to obtain the local structural embedding of entities and relationships; then go to S5.

[0108] In the specific implementation, the CompGCN model of the graph neural network GNN is used as the graph structure encoder. In this way, the CompGCN model of the graph neural network GNN is used as the graph structure encoder, which can efficiently process multi-relational graphs, enhance the model's expressiveness, optimize the representation learning process, improve the actual application effect, and promote the development of GNN technology. It has broad application prospects and important research value in the fields of knowledge graphs, social network analysis, and recommendation systems.

[0109] S3: Map the instance token of the entity sequence to the description text corresponding to the entity; use the large language model as the entity text encoder to convert the obtained description text into the initial semantic embedding; and go to S5.

[0110] In specific implementation, when mapping the instance token of the entity sequence to the description text corresponding to the entity, if the entity has multiple annotation attribute information, the annotation attribute is selected as the text representative according to the preset standard; if the entity has no annotation attribute, the word representative containing semantic information in the entity URI is used as the text representative.

[0111] In this way, when there are multiple annotation attribute information for an entity, selection by preset standards can ensure that the selected annotation attributes can accurately and comprehensively represent the core characteristics of the entity. This selection method avoids information redundancy and misleading, and improves the accuracy of subsequent processing and analysis. In addition, it can enhance the ability of semantic expression. Whether selecting annotation attributes or using semantic information words in the entity URI, it is to better express the semantic information of the entity. This method makes the entity description more specific and vivid, which helps the subsequent tasks to accurately understand and process the entity semantics. In addition, it can also improve processing efficiency. Mapping entity descriptions through preset standards and rules can greatly simplify the processing process and reduce the time cost of manual intervention and judgment. This automated processing method not only improves processing efficiency, but also reduces the risk of human error.

[0112] In specific implementation, before using the large language model to convert the description text, the large language model is also trained through the Masked Language Model task to optimize and improve the semantic quality of text embedding.

[0113] In this way, the MLM (Masked Language Model) task requires the model to predict masked or replaced words, which prompts the model to understand the contextual information of the text more deeply. Through the training of a large number of MLM tasks, the large language model can learn the dependencies between words in a sentence, as well as the intrinsic structure and semantic information of the language, thereby significantly improving its language understanding ability. Text embedding is the process of converting text into a vector representation, and its quality directly affects the effect of subsequent tasks. The large language model trained by the MLM task can generate higher quality text embeddings. These embedded vectors can more accurately capture the semantic features of the text and provide more valuable information for subsequent tasks. The MLM task uses a large amount of different text data for training, which enables the large language model to learn a variety of language phenomena and expressions. Therefore, when faced with new and unseen text, the model can better generalize its learned knowledge and generate reasonable predictions or outputs.

[0114] S4: Use the graph structure encoder to perform graph convolution on the new RDF graph, extract the global structure embedding of entities and relationships, and go to S5.

[0115] In specific implementation, the process of extracting global structural embedding of entities and relationships from the new RDF graph includes:

[0116] Using CompGCN as the graph structure encoder for entities and relations, the RDF knowledge graph is defined as where ε represents the set of entities, represents a collection of relations, Represents a set of triples. On this basis, we add inverse relations and self-loop relations, and add the inverse relations and self-loop relations to the relation set, as shown below:

[0117]

[0118] Get a new RDF graph

[0119] New RDF Graph Combine the head entity and the relationship as shown below:

[0120]

[0121] in, is a composite operator; (h, r, t) represents the starting point, relationship type, and end point; ε represents the set of entities, represents a collection of relations, Represents a set of triples;

[0122] Perform convolution operation on the entity representation after the combination operation:

[0123]

[0124] Among them, W k is the parameter matrix for the relationship type, which is divided as follows:

[0125]

[0126] Transform the relation embedding as shown below:

[0127]

[0128] Among them, W rel is a learnable transformation matrix used to project all relation feature vectors into the same embedding space as node+ and allow them to be utilized in the next CompGCN layer;

[0129] Finally, the global structural embedding of entities and relations is obtained.

[0130] The local structural embedding of entities and relations in S2 is similar to the above process and will not be repeated here.

[0131] "Node+" refers to the comprehensive form of entity and relationship representation in the embedding space. Specifically, when encoding the global graph structure, the model will combine the head entity and the relationship. The result of this combination represents the comprehensive information of the starting point, relationship type, and end point. Subsequently, this comprehensive information will be used for convolution operations and other processing steps to obtain the global structure embedding. In this process, "node+" can be understood as a comprehensive node that contains entity and relationship information, which interacts and projects with the relationship feature vector in the embedding space.

[0132] In this way, the model's ability to understand the global structure can be enhanced. By combining the head entity and the relationship and introducing convolution operations, the model can capture the complex interactions between entities and relationships, thereby more accurately understanding the global structure. This global structure embedding helps the model better understand and utilize the information in the graph in subsequent tasks. It can also improve the expressiveness of the relationship embedding. By transforming the relationship embedding and projecting it into the same embedding space as the node, the model can more flexibly utilize the relationship information. This transformation not only enhances the expressiveness of the relationship embedding, but also makes the relationship embedding and the node embedding more consistent, which helps to improve the performance of subsequent tasks. In addition, it also supports the processing of complex relationships. After introducing inverse relationships and self-loop relationships, the new RDF graph can more comprehensively represent the relationship between entities. This representation method helps the model handle complex relationships, such as transitive relationships, inverse relationships, etc., so as to more accurately understand the entities and relationships in the graph.

[0133] S5: Concatenate the local structure embedding, semantic embedding, and global structure embedding obtained in S2-S4 to form a unified representation, and then perform further feature transformation through a linear layer to obtain the embedded representation of the fused entities and relations.

[0134] Each embedding captures important information in the knowledge graph from different perspectives. These embedding vectors will be concatenated into a unified representation and then further transformed through a linear layer to obtain the fused embedding representation of entities and relations.

[0135] S6. Optimize the embedded representation of the fused entities and relations obtained in S5 to obtain the final representation of the entities and relations.

[0136] In specific implementation, the L2 distance loss function is used to optimize the embedded representation of the fused entities and relations obtained in S5. In this way, using the L2 distance loss function to optimize the embedded representation of the fused entities and relations can improve the accuracy of the embedded representation, enhance the generalization ability of the model, optimize the computational efficiency and stability, promote the semantic consistency of the embedded representation, and support a variety of subsequent tasks. These effects together provide a strong guarantee for the performance of the model in practical applications.

[0137] The expression of the L2 distance loss function is:

[0138]

[0139] In the formula, represents a new RDF graph; (h, r, t) is a positive triple, (h - ,r - ,t - ) is a negative example triplet; γ is the distance between positive and negative samples and is a hyperparameter.

[0140] In this way, the loss function is calculated by comparing the positive triplet (h, r, t) with the negative triplet (h - ,r - ,t - ), and introduce a positive and negative sample spacing γ to distinguish between positive and negative samples. This helps the model learn the correct associations between entities and relations, while distinguishing those that do not exist. During the training process, the model continuously adjusts the embedding vectors of entities and relations to minimize the loss function. This causes the distance between the embedding vectors of positive triples to decrease, while the distance between the embedding vectors of negative triples to increase, thereby optimizing the quality of the embedded representation. In addition, the introduction of negative triples and the positive and negative sample spacing γ allows the model to focus not only on positive triples during training, but also on negative triplets that are mistakenly considered to be positive. This helps improve the robustness of the model in the face of noisy and abnormal data. As a hyperparameter, the positive and negative sample spacing γ can be adjusted according to the characteristics of specific tasks and datasets. This provides additional flexibility, allowing the model to adapt to different application scenarios.

[0141] S7. Use the final representation of entities and relationships obtained in S6 for subsequent practical applications.

[0142] Traditional methods often only focus on the structural information of the knowledge graph, ignoring the value of semantic and textual information. Compared with the existing technology, this method not only considers the structural information in the bridge maintenance knowledge graph, but also makes full use of the complex semantic information and textual information in the OWL ontology. Through the synergy of the graph structure encoder and the large language model, multi-dimensional representation learning of entities and relationships is achieved. By integrating this information, this method improves the accuracy and comprehensiveness of representation learning. In addition, this method effectively captures the local structural features of entities and relationships by generating multiple subgraphs of different relationships and encoding each subgraph using the graph structure encoder. At the same time, the new RDF graph is subjected to graph convolution operations to extract the global structural embedding. Compared with the method based only on the global graph or a single subgraph, this method can capture the local and global structural features in the knowledge graph more carefully and improve the precision of representation learning. In addition, this method uses a large language model to convert the description text of the entity sequence into a semantic embedding, and splices it with the local structure embedding and the global structure embedding to form a unified representation. This fusion method makes the representation of entities and relationships richer and more accurate. Traditional methods usually only consider structural information or simple text information, while this method achieves in-depth understanding and utilization of text information through a large language model, improving the semantic richness of representation learning. In addition, this method optimizes the embedded representation of fused entities and relationships to obtain a more accurate and robust final representation. These final representations can be directly used in subsequent practical applications, such as knowledge reasoning, entity linking, relationship prediction, etc. Through the optimization steps, this method can further improve the quality of representation learning and provide more reliable and effective support for practical applications.

[0143] In summary, this method can make full use of the complex semantic information, text information and structural information in the OWL ontology, and realize the multi-dimensional representation learning of entities and relationships through the synergy of the graph structure encoder and the large language model. By capturing local and global structural features, integrating semantic embedding and text information, and performing optimization processing, this method obtains a more accurate and complete representation of the bridge maintenance knowledge graph. These representations can be directly used in subsequent practical applications, such as knowledge reasoning, entity linking, relationship prediction, etc., thereby improving the accuracy and completeness of the bridge maintenance knowledge graph.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A bridge maintenance knowledge graph representation method based on a large language model and a graph neural network, characterized in that: The following steps are involved: S1, convert the OWL ontology into a triple knowledge graph RDF graph through the ontology mapping rules, and generate multiple subgraphs with different relationships according to the preset walking scheme, and go to S2; adopt the preset mixed walking strategy, starting from multiple entities in each subgraph, generate entity sequences based on a specific number of steps, and go to S3; construct inverse relationships and self-loop relationships based on the initial RDF graph to form a new RDF graph, and go to S4; S2. Use a graph structure encoder to encode each subgraph to capture the local structural features of entities and relations, and perform average pooling on the representations of the same entity and relation to obtain the local structural embedding of the entity and relation. Then go to S5; S3, map the instance token of the entity sequence to the description text corresponding to the entity; use the large language model as the entity text encoder to convert the obtained description text into the initial semantic embedding; and go to S5; S4, use the graph structure encoder to perform graph convolution operations on the new RDF graph, extract the global structure embedding of entities and relationships, and go to S5; S5, concatenate the local structure embedding, semantic embedding and global structure embedding obtained in S2-S4 to form a unified representation, and then perform further feature transformation through a linear layer to obtain the fused embedding representation of entities and relations; S6, optimizing the embedded representation of the fused entities and relations obtained in S5 to obtain the final representation of the entities and relations; S7. Use the final representation of entities and relationships obtained in S6 for subsequent practical applications.

2. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 1 is characterized by: In S1, the subgraphs of different relationships include a class hierarchy subgraph, a class object attribute subgraph, a class instance subgraph, an instance object attribute subgraph, and an instance data attribute subgraph.

3. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network as claimed in claim 2 is characterized by: In S1, the process of generating multiple subgraphs with different relationships according to the preset walking scheme includes: Randomly select entities with current relationship links as the starting point, and walk along the current relationship to generate multiple subgraphs: First, for the relationship rdfs:subClassOf, the axioms are converted into RDF triples according to the ontology mapping rules, the head entity and the tail entity are restricted to concept classes, and all triples with subClassOf relationships are extracted to form a class hierarchy subgraph; Secondly, according to the object attribute relationship, the head entity and the tail entity are restricted to the concept class, and complex axiom constraints are added to extract all triples that meet the conditions except the subClassOf relationship to form the class object attribute subgraph; Thirdly, extract the types of all instances according to the rdf:type relationship, and combine the triples containing rdf:typ into a class instance subgraph; Then, all triples whose relations are object attributes and whose head and tail entities are instances are extracted to form an instance object attribute subgraph; Finally, all triples whose relations are data attributes and whose tail entities are literals are extracted to form an instance data attribute subgraph.

4. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network as claimed in claim 3 is characterized by: In S1, the preset hybrid roaming strategies include: Based on the five subgraphs, the entities with the highest connectivity are selected as the starting points of regular walks and random walks respectively. If the highest connectivity is the same, the entity with the highest connectivity is randomly selected as the starting point; For the class hierarchy subgraph, rules are formulated to perform specific walks in specific directions, and preset random walks are introduced; Among them, specific walk steps, regular walk steps, and random walk steps are all hyperparameters.

5. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 1 is characterized by: The CompGCN model of the graph neural network GNN is used as the graph structure encoder.

6. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 1 is characterized by: In S3, when mapping the instance token of the entity sequence to the description text corresponding to the entity, if the entity has multiple annotation attribute information, the annotation attribute is selected as the text representative according to the preset standard; if the entity has no annotation attribute, the word representative containing semantic information in the entity URI is used as the text representative.

7. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 6 is characterized by: In S3, before using the large language model to convert the description text, the large language model is also trained through the Masked Language Model task to optimize and improve the semantic quality of text embedding.

8. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 1 is characterized by: In S4, the process of extracting global structural embeddings of entities and relations from the new RDF graph includes: For the initial RDF graph Add inverse relations and self-loop relations to the relationship set and obtain a new RDF graph by adding inverse relations and self-loop relations to the relationship set Combine the head entity and the relationship as shown below: in, is a composite operator; (h, r, t) represents the starting point, relationship type, and end point; ε represents the set of entities, represents a collection of relations, Represents a set of triples; Perform convolution operation on the entity representation after the combination operation: Among them, W k is the parameter matrix for the relationship type, which is divided as follows: Transform the relation embedding as shown below: Among them, W rel is a learnable transformation matrix used to project all relation feature vectors into the same embedding space as node+ and allow them to be utilized in the next CompGCN layer; Finally, the global structural embedding of entities and relations is obtained.

9. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 1 is characterized by: In S6, the L2 distance loss function is used to optimize the embedded representation of the fused entities and relations obtained in S5.

10. The bridge maintenance knowledge graph representation method of the large language model collaborative graph neural network according to claim 9 is characterized by: In S6, the expression of the L2 distance loss function is: In the formula, represents a new RDF graph; (h, r, t) is a positive triple, (h - ,r - ,t - ) is a negative example triplet; γ is the distance between positive and negative samples and is a hyperparameter.

Images