Knowledge graph completion method based on subgraph sampling and pre-training language model
By combining subgraph sampling and pre-trained language models, entity and relation representations are optimized, addressing the problem of insufficient entity association mining in complex relation processing of existing methods. This improves the knowledge graph completion performance and enhances the model's generalization ability.
Patent Information
- Application Number
- CN202510967256.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-14
- Publication Date
- 2025-10-28
AI Technical Summary
Existing knowledge graph completion methods fail to fully explore the relationships between entities when dealing with complex relationships, resulting in poor completion performance.
We employ a method based on subgraph sampling and pre-trained language models. By acquiring entity information and preprocessing it, we perform second-order subgraph sampling and graph transformation. We then use the BERT model to encode entity relationships and combine contrastive learning to optimize the embedding vectors, thereby completing the knowledge graph.
It significantly improves the performance of knowledge graph completion tasks, enhances the accuracy of triple prediction, and improves the training efficiency and generalization ability of the model on large-scale datasets.
Smart Images
Figure CN120851166A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of knowledge graph completion technology, specifically to a knowledge graph completion method based on subgraph sampling and pre-trained language models. Background Technology
[0002] Knowledge graphs, as a structured form of knowledge representation, store massive amounts of human knowledge and play a crucial role in downstream fields such as recommender systems, question-answering systems, and information retrieval. By organizing entities and their relationships into a graph structure, knowledge graphs facilitate machine learning and reasoning about complex knowledge. However, due to limitations in knowledge acquisition and the lag in data updates, knowledge graphs often suffer from incomplete information, severely limiting their effectiveness and performance in downstream fields. Therefore, how to effectively and automatically complete knowledge graphs has attracted widespread attention.
[0003] Existing knowledge graph completion methods primarily rely on graph embedding techniques. The core idea is to learn vector representations of entities and relations in a low-dimensional embedding space and then predict them using machine learning algorithms. For example, traditional TransE models learn the representations of entities and relations in the embedding space and predict knowledge graph triples using distance or similarity between vectors. However, they have limitations when dealing with complex graph structures. Knowledge graph embedding methods based on graph neural networks mainly rely on graph structure information to represent entities and relations, neglecting the rich semantic information in the knowledge graph, making it difficult to effectively utilize the textual descriptions of entities and relations.
[0004] Therefore, text-based knowledge graph completion methods have been proposed, which use pre-trained language models to encode textual descriptions of entities and relationships in order to capture semantic associations between entities. However, these methods often ignore the structural information of the knowledge graph, which makes it impossible to fully explore the associations between entities when dealing with complex relationships, resulting in poor knowledge graph completion performance. Summary of the Invention
[0005] To address the technical problem of existing knowledge graph completion methods failing to fully exploit the relationships between entities when handling complex relationships, resulting in poor knowledge graph completion performance, this invention aims to provide a knowledge graph completion method based on subgraph sampling and a pre-trained language model. The specific technical solution adopted is as follows:
[0006] One embodiment of the present invention provides a knowledge graph completion method based on subgraph sampling and a pre-trained language model, the method comprising the following steps:
[0007] S1: Obtain entity information of several candidate entities in the knowledge graph, and obtain entity information in JSON format after preprocessing the entity information. The entity information includes entity structure information and entity description information.
[0008] S2: Take the entity structure information of the JSON format data as input, and perform second-order subgraph sampling on the head entity and the tail entity respectively to obtain the second-order subgraph structure of the head entity and the tail entity.
[0009] S3: Input the second-order subgraph structures of the head entity and the tail entity into the graph-to-text module for processing to obtain the text sequences corresponding to the second-order subgraph structures of the head entity and the tail entity.
[0010] S4: Concatenate the text sequence and entity description information of the JSON format data corresponding to the head entity and tail entity to obtain the comprehensive text sequence corresponding to the head entity and tail entity;
[0011] S5: Concatenate the comprehensive text sequence and relation of the head entity and input them into two independent BERT models for encoding, respectively, to obtain the embedding vectors corresponding to the head entity relation and the tail entity;
[0012] S6: Take the two embedded vectors as a positive sample pair and input them, along with the negative sample pair obtained by negative sampling, into the contrast learning module to obtain the optimized final vectors corresponding to the head entity relationship and the tail entity.
[0013] S7: Use the two final vectors to complete the missing information in the knowledge graph.
[0014] Furthermore, the implementation process of step S1 includes:
[0015] S11: Obtain entity structure information, entity description information, and a dictionary of correspondences between entity IDs and entity names from the text file. Split the entity structure information into three fields: head_id, relation, and tail_id, according to tabs. Split the entity description information into two fields: entity_id and entity description, according to tabs. The entity structure information is stored in the form of triples.
[0016] S12: Normalize the relation field in each triplet to obtain a relation name that conforms to the text specification. The normalization process is to replace all underscores in the relation field with spaces and remove extra whitespace at the beginning and end.
[0017] S13: After normalization, each triplet example is encapsulated into an object containing head_id, head entity name, normalized relation name, tail_id, and tail entity name. All these objects are written to an output file in JSON format to obtain entity structure information in JSON format. Similarly, entity_id, entity name, and entity description are written to the output file in JSON format to obtain entity description information in JSON format. Further, the implementation process of step S2 includes:
[0018] S21: Randomly sample the first-order neighbors of the head and tail entities;
[0019] S22: Sampling second-order neighbors from first-order neighbors based on degree importance;
[0020] S23: Obtain the second-order subgraph structure of the head entity and the tail entity through the first-order neighbors and the second-order neighbors.
[0021] Furthermore, the implementation process of step S21 includes:
[0022] First, obtain all first-order neighbors of the entity to be determined, including the tail entity and the head entity; then, filter out the tail entity from all the first-order neighbors to obtain the remaining first-order neighbors; finally, randomly select a first preset number of neighbor information from the remaining first-order neighbors to obtain the first-order neighbors used to construct the local subgraph.
[0023] The implementation process of step S22 includes: calculating the degree of the second-order neighbors based on the first-order neighbors used to construct the local subgraph and sorting them to obtain a set of second-order neighbors; selecting a second preset number of second-order neighbors with the highest degree from the set of second-order neighbors to obtain the final second-order neighbors;
[0024] The implementation process of step S23 includes: merging the first-order neighbors used to construct the local subgraph and the final second-order neighbors to obtain the second-order subgraph structure of the entity to be determined, wherein the entity to be determined includes the tail entity or the head entity.
[0025] Furthermore, the implementation process of step S3 includes:
[0026] S31: Input the second-order subgraph structure of the head entity into the graph-to-text module to convert it into a text sequence, and combine the relationship between the head entity and its neighboring entities to output the text representation of the second-order subgraph of the head entity;
[0027] S32: Input the second-order subgraph structure of the tail entity into the graph-to-text module to convert it into a text sequence. Combine the relationship between the tail entity and its neighboring entities to output the text representation of the second-order subgraph of the tail entity.
[0028] Furthermore, the expression for the second-order subgraph text representation of the head entity is:
[0029] T h =concat(e h ,r h,1 ,e t,1 ,r′ h,1 ,e′ h,1 ;…;r h,i ,e h,i ,r′ h,i ,e′ h,i ;…;r h,m ,e h,m In the formula, T h The second-order subgraph text sequence representing the head entity, e h Let i represent the header entity, and r represent the positive integer corresponding to the header entity. h,i r′ represents a first-order relation related to the head entity. h,i e represents a second-order relation related to the head entity. h,i e′ represents a first-order neighbor associated with the head entity. h,i The first-order neighbor is the second-order neighbor associated with the head entity, m is the total number of first-order or second-order neighbors associated with the head entity, and concat is the join function.
[0030] The expression for the second-order subgraph text representation of the tail entity is:
[0031] T t =concat(e t ,r t,1 ,e t,1 ,r′ t,1 ,e′ t,1 ;…;r t,j ,e t,j ,r′ t,j ,e′ t,j ;…;r t,n ,e t,n In the formula, T t The second-order subgraph text representation of the tail entity, e t Represents the tail entity, j represents the positive integer corresponding to the tail entity, r t,j Denotes a first-order relation related to the tail entity, r′ t,j e represents a second-order relation related to the tail entity. t,j e′ represents a first-order neighbor associated with the tail entity. t,j represents the second-order neighbors associated with the tail entity, and n represents the total number of first-order or second-order neighbors associated with the tail entity.
[0032] Furthermore, the expression for the comprehensive text sequence corresponding to the head entity is:
[0033] S h=concat(d(e) h ):T h In the formula, S h This represents the combined text sequence corresponding to the header entity; concat represents the concatenation function; e h Represents the head entity, d(e) h ) represents the descriptive information of the header entity, T h A second-order subgraph text sequence representing the head entity;
[0034] The expression for the comprehensive text sequence corresponding to the tail entity is:
[0035] S t =concat(d(e) t ):T t In the formula, S t The e represents the comprehensive text sequence corresponding to the tail entity. t Represents the tail entity, d(e) t T represents the description information of the tail entity. t The second-order subgraph text representation of the tail entity.
[0036] Furthermore, the implementation process of step S6 includes:
[0037] S61: Take the embedding vectors of the head entity relationship and the tail entity as positive sample pairs, and replace the embedding vector of the tail entity with the embedding vector of the erroneous tail entity by negative sampling to generate negative sample pairs.
[0038] S62: The cosine similarity function is used to process the two embedded vectors of the positive and negative sample pairs to obtain the vector similarity.
[0039] S63: The loss function optimizes the embedding vector by the vector similarity of positive and negative sample pairs, and obtains the optimized final vector of the head entity relationship and the tail entity correspondence.
[0040] Furthermore, the expression for the loss function is:
[0041] In the formula, log represents the logarithmic function, and e represents the natural constant. Let represent the scoring function for candidate triples, y0 represent the function used to adjust the distance between positive and negative samples, and τ represent the temperature coefficient.
[0042] Furthermore, the implementation process of step S7 includes:
[0043] The two optimized final vectors are used for link prediction, and cosine similarity is used to score all candidate entities. The highest score is used as the predicted tail entity.
[0044] The present invention has the following beneficial effects:
[0045] This invention provides a knowledge graph completion method based on subgraph sampling and a pre-trained language model. This method optimizes the representation of entities and relations through contrastive learning, significantly improving performance in knowledge graph completion tasks. Specifically, by jointly using subgraph sampling and the BERT model, the accuracy of triple prediction is enhanced while effectively capturing the structural and semantic information of the knowledge graph. Furthermore, the use of a contrastive learning loss function and boundary adjustment mechanism further improves the model's training efficiency and generalization ability on large-scale datasets. In summary, this invention has strong scalability and application prospects, and is suitable for fields such as intelligent question answering, recommendation systems, and knowledge discovery. Attached Figure Description
[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0047] Figure 1 A flowchart illustrating a knowledge graph completion method based on subgraph sampling and a pre-trained language model, provided as an embodiment of the present invention;
[0048] Figure 2 This is a diagram of the sub-graph sampling process in an embodiment of the present invention;
[0049] Figure 3 This is a framework diagram of a knowledge graph completion method based on subgraph sampling and pre-trained language models, provided as an embodiment of the present invention. Detailed Implementation
[0050] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solution proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0052] The application scenarios of this invention can be:
[0053] Existing text-based knowledge graph completion methods use pre-trained language models to encode textual descriptions of entities and relations to capture semantic relationships between entities. While pre-trained language models have certain advantages in capturing textual semantics, they often neglect the structural information of the knowledge graph, resulting in an inability to fully explore the connections between entities when dealing with complex relationships. Therefore, effectively integrating knowledge graph structural information and textual description information has become a significant challenge for knowledge graph completion.
[0054] One embodiment of the present invention provides a knowledge graph completion method based on subgraph sampling and a pre-trained language model, such as... Figure 1 As shown, the following steps are included:
[0055] S1: Obtain entity information of several candidate entities in the knowledge graph, and obtain entity information in JSON format after preprocessing the entity information.
[0056] Here, entity information includes entity structure information, entity description information, and entity name information.
[0057] As an exemplary implementation, the process of implementing step S1 above includes:
[0058] S11: Obtain entity structure information, entity description information, and a dictionary of entity ID and entity name correspondences from the text file. Split the entity structure information into three fields: head_id, relation, and tail_id, according to tabs. Split the entity description information into two fields: entity_id and entity description, according to tabs.
[0059] Here, entity structure information is stored in the form of triples.
[0060] In this embodiment, entity information from different candidate entity datasets (such as WN18RR and FB15k237) is first loaded. The entity structure information includes the ID of the head entity, the relation, and the ID of the tail entity, which are stored in three text files: train.txt, test.txt, and valid.txt. The entity description information, such as wn18rr_id2ent.txt and fb15k_id2ent.txt, contains each entity ID and its corresponding description information. Then, the script parses the triples (head_id, relation, tail_id) of the structure information and splits them into three fields: head_id, relation, and tail_id. The entity description information is split into two fields: entity_id and entity description.
[0061] S12: Normalize the relation field in each triplet to obtain a relation name that conforms to the text specification.
[0062] Here, the normalization process involves replacing all underscores in the field `relation` with spaces and removing any extra whitespace at the beginning and end.
[0063] In this embodiment, for each relation in the dataset, the script uses the normalize_relations() function to normalize the relation, such as by replacing underscores with spaces to standardize the relation form.
[0064] S13: After completing the normalization process, each triplet example is encapsulated into an object containing head_id, head entity name, normalized relation name, tail_id, and tail entity name. All objects are written to the output file in JSON format to obtain the entity structure information in JSON format. The entity_id, entity name, and entity description are also written to the output file in JSON format to obtain the entity description information in JSON format.
[0065] In this embodiment, the structural information of all candidate entities is extracted and saved as train.json, test.json and valid.json files, and the descriptive information of all candidate entities is extracted and saved as entities.json file, providing a complete entity relationship graph to facilitate subsequent training and analysis.
[0066] Thus, this embodiment has obtained entity information of JSON format data to achieve structured storage of input data.
[0067] S2: Take the entity structure information of the JSON format data as input, and perform second-order subgraph sampling on the head entity and the tail entity respectively to obtain the second-order subgraph structure of the head entity and the tail entity.
[0068] The subgraph sampling process diagram is as follows: Figure 2 As shown.
[0069] As an exemplary implementation, step S2 described above can be achieved through the following sub-steps:
[0070] S21: Randomly sample the first-order neighbors of the head and tail entities.
[0071] Specifically, first, obtain all first-order neighbors of the entity to be determined, including the tail entity and the head entity; to avoid label leakage, remove the tail entity from all first-order neighbors to obtain the remaining first-order neighbors; finally, randomly select a first preset number of neighbor information from the remaining first-order neighbors to obtain the first-order neighbors used to construct the local subgraph.
[0072] In this embodiment, the expression for obtaining first-order neighbors can be:
[0073] N1(e)={e1,e2,e3,...e n};
[0074] N′1(e)=N1(e) / tail id ;
[0075]
[0076] In the formula, N1(e) represents the original first-order neighbor set of the entity to be determined, e1, e2, e3, ..., e n Let N' be the set of n first-order neighbors in the original first-order neighbor set, and let N'1(e) be the set of first-order neighbors after removing the tail entity. id Indicates the tail entity. This indicates that at most K1 first-order neighbors will be randomly selected from N′1(e), where K1 represents the first preset number.
[0077] S22: Sample second-order neighbors from first-order neighbors based on degree importance.
[0078] Based on the first-order subgraph, the selection of second-order neighbors is based on the degree between their neighbors.
[0079] Specifically, based on the first-order neighbors used to construct the local subgraph, the degree of the second-order neighbors is calculated and sorted to obtain a set of second-order neighbors; from the set of second-order neighbors, a second preset number of second-order neighbors with the highest degree are selected to obtain the final set of second-order neighbors.
[0080] The expression for obtaining the final second-order neighbors can be:
[0081] N2(e)={e′1,e′2,e′3....e′ m},degree(e′ i )≥degree(e′ i+1 );
[0082]
[0083] In the formula, N2(e) represents the original second-order neighbor set of the entity to be determined, and degree(e′) i )≥degree(e′ i+1 ) represents e′ i+1 The degree of ) is greater than e′ i The degree, K2 represents the second preset number of second-order neighbors, which consists of the first few second-order neighbors by degree.
[0084] S23: Obtain the second-order subgraph structure of the head entity and the tail entity through first-order neighbors and second-order neighbors.
[0085] Specifically, the first-order neighbors used to construct the local subgraph and the final second-order neighbors are merged to obtain the second-order subgraph structure of the entity to be determined, which includes the tail entity or the head entity.
[0086] In this embodiment, the expression for obtaining the second-order subgraph structure can be:
[0087]
[0088] In the formula, G h G represents the second-order subgraph structure of the head entity. t The second-order subgraph structure representing the tail entity, e h Represents the head entity, e t Indicates the tail entity. This represents the set of first-order neighbors corresponding to the head entity. This represents the set of second-order neighbors corresponding to the head entity. This indicates that the tail entity corresponds to the set of first-order neighbors. This represents the set of second-order neighbors corresponding to the tail entity.
[0089] Thus, this embodiment has obtained the second-order subgraph structure of the head entity and the tail entity.
[0090] S3: Input the second-order subgraph structures of the head entity and the tail entity into the graph-to-text module for processing to obtain the text sequences corresponding to the second-order subgraph structures of the head entity and the tail entity.
[0091] Here, since the BERT pre-trained language model can only learn text sequences, a graph-to-text sequence module was designed to convert the subgraph structure of the knowledge graph into a text sequence for encoding, and learn the subgraph structure information of entities through the pre-trained language model.
[0092] As an exemplary implementation, step S3 above can be achieved through the following sub-steps:
[0093] S31: Input the second-order subgraph structure of the head entity into the graph-to-text module to convert it into a text sequence. Combine the relationship between the head entity and its neighboring entities to output the text representation of the second-order subgraph of the head entity.
[0094] The expression for the second-order subgraph text representation of the head entity is:
[0095] T h =concat(e h ,r h,1 ,e t,1 ,r′ h,1 ,e′h,1 ;…;r h,i ,e h,i ,r′ h,i ,e′ h,i ;…;r h,m ,e h,m In the formula, T h The second-order subgraph text sequence representing the head entity, e h Let i represent the header entity, and r represent the positive integer corresponding to the header entity. h,i r′ represents a first-order relation related to the head entity. h,i e represents a second-order relation related to the head entity. h,i e′ represents a first-order neighbor associated with the head entity. h,i This represents the second-order neighbors associated with the head entity, m represents the total number of first-order or second-order neighbors associated with the head entity, and concat represents the join function.
[0096] S32: Input the second-order subgraph structure of the tail entity into the graph-to-text module to convert it into a text sequence. Combine the relationship between the tail entity and its neighboring entities to output the text representation of the second-order subgraph of the tail entity.
[0097] The expression for the second-order subgraph text representation of the tail entity is:
[0098] T t =concat(e t ,r t,1 ,e t,1 ,r′ t,1 ,e′ t,1 ;…;r t,j ,e t,j ,r′ t,j ,e′ t,j ;…;r t,n ,e t,n In the formula, T t The second-order subgraph text representation of the tail entity, e t Represents the tail entity, j represents the positive integer corresponding to the tail entity, r t,j Denotes a first-order relation related to the tail entity, r′ t,j e represents a second-order relation related to the tail entity. t,j e′ represents a first-order neighbor associated with the tail entity. t,j represents the second-order neighbors associated with the tail entity, and n represents the total number of first-order or second-order neighbors associated with the tail entity.
[0099] Thus, this embodiment has obtained the second-order subgraph text of the head entity and the tail entity.
[0100] S4: Concatenate the text sequences and entity description information corresponding to the head entity and tail entity to obtain the comprehensive text sequence corresponding to the head entity and tail entity.
[0101] As an exemplary implementation, step S4 above can be achieved through the following sub-steps:
[0102] S41: Concatenate the head entity description and the obtained second-order subgraph text sequence of the head entity to generate a comprehensive text sequence of structural and descriptive information. The expression can be:
[0103] S h =concat(d(e) h ):T h In the formula, S h This represents the combined text sequence corresponding to the header entity; concat represents the concatenation function; e h Represents the head entity, d(e) h ) represents the descriptive information of the header entity, T h A second-order subgraph text sequence representing the head entity.
[0104] S42: Combine the tail entity description and the resulting tail entity second-order subgraph text sequence T t By concatenating these elements, a combined text sequence containing structural and descriptive information can be generated. The expression can be:
[0105] S t =concat(d(e) t ):T t In the formula, S t The e represents the comprehensive text sequence corresponding to the tail entity. t Represents the tail entity, d(e) t T represents the description information of the tail entity. t The second-order subgraph text representation of the tail entity.
[0106] Thus, this embodiment obtains the combined text sequence corresponding to the head entity and the tail entity by splicing the sequence and entity description information.
[0107] S5: Concatenate the combined text sequence and relation of the head entity, and input them into two independent BERT models for encoding, along with the combined text sequence of the tail entity, to obtain the embedding vectors corresponding to the head entity relation and the tail entity.
[0108] Here, two independent BERT encoders are used to process the head entity and the tail entity respectively, which makes it easier to encode their semantic information separately, avoids semantic confusion caused by sharing an encoder, and can better capture the independent contextual relationships between entities.
[0109] As an exemplary implementation, step S5 above can be achieved through the following sub-steps:
[0110] S51: The comprehensive text sequence and relation corresponding to the head entity are concatenated via [SEP] and input into the first BERT for encoding, resulting in the embedding vectors of the head entity and relation. The expression can be:
[0111] v hr =BERT(S h ,r); where v hr Embedding vectors representing head entities and relations.
[0112] It should be noted that the text sequence of the head entity and its corresponding relationship is taken as input and processed by the BERT encoder to obtain an embedding vector containing semantic and structural information. This embedding vector can effectively represent the semantic and structural features of the head entity and the relationship.
[0113] S52: Input the tail entity into the second BERT for encoding to obtain the tail entity's embedding vector. The expression can be:
[0114] v t =BERT(S t In the formula, v t This represents the embedding vector of the tail entity.
[0115] It should be noted that the text sequence of the tail entity is taken as input and processed by the BERT encoder to obtain an embedding vector containing semantic and structural information. This embedding vector can effectively represent the semantic and structural features of the tail entity.
[0116] Thus, this embodiment has obtained the embedding vectors corresponding to the head entity relationship and the tail entity.
[0117] S6: Take the two embedding vectors as a positive sample pair and input them, along with the negative sample pair obtained from negative sampling, into the contrastive learning module to obtain the optimized final vectors corresponding to the head entity relationship and the tail entity.
[0118] As an exemplary implementation, step S6 above may include:
[0119] S61: Take the embedding vectors of the head entity relationship and the tail entity as positive sample pairs, and replace the embedding vector of the tail entity with the embedding vector of the erroneous tail entity by negative sampling to generate negative sample pairs.
[0120] In this embodiment, the embedding vectors of the head entity relationship and the tail entity are used as positive sample pairs. A negative sampling strategy is employed to randomly replace the tail entity with an incorrect tail entity, generating mismatched entity pairs as negative sample pairs. These negative sample pairs, along with the positive sample pairs, are used to train the model, helping it distinguish between correct and incorrect tail entities, thereby improving the model's predictive ability and generalization performance.
[0121] S62: The cosine similarity function is used to process the two embedded vectors of the positive and negative sample pairs to obtain the vector similarity.
[0122] As an example, the formula for calculating the vector similarity of positive sample pairs can be:
[0123] In the formula, cos(v hr ,v t ) represents the cosine similarity of positive sample pairs, and ||| represents the magnitude of the vector.
[0124] S63: The loss function optimizes the embedding vector by the vector similarity of positive and negative sample pairs, and obtains the optimized final vector of the head entity relationship and the tail entity correspondence.
[0125] In this embodiment, the loss function optimizes the representation of entities and relationships by comparing the similarity between positive and negative samples, thereby improving the accuracy of triplet completion.
[0126] Specifically, the AdamW optimizer and learning rate scheduler are used to optimize the model parameters, and the gradient is updated through the backpropagation algorithm. After each training session, the contrastive loss is calculated and optimized to enable the model to better learn entities and relationships in the knowledge graph.
[0127] As an exemplary implementation, the expression for the loss function is:
[0128] In the formula, log represents the logarithmic function, and e represents the natural constant. Let represent the scoring function for candidate triples, y0 represent the function used to adjust the distance between positive and negative samples, and τ represent the temperature coefficient.
[0129] In the formula for calculating the loss function, the constant y0 can avoid overfitting and improve the model's score for the correct triplet (h,r,t). The temperature coefficient τ is used to adjust the importance of negative samples. The smaller the temperature coefficient τ, the higher the weight the loss function will give to negative samples, but it is also more likely to overfit to noisy labels.
[0130] Thus, this embodiment has obtained the optimized final vectors corresponding to the head entity relationship and the tail entity.
[0131] S7: Use two final vectors to complete the missing information in the knowledge graph.
[0132] Specifically, the two optimized final vectors are used for link prediction. Cosine similarity is used to score all candidate entities, and the highest-scoring entity is selected as the predicted tail entity. The expression for this is:
[0133] argmax ti cos(v′ hr ,v′ t ),t i ∈ε; where ε represents the candidate entity set, v′ hr v′ represents the optimized final vector corresponding to the head entity relationship. t Let represent the optimized final vector corresponding to the tail entity, argmax represent the function of the independent variable that takes the maximum value, cos represent the cosine similarity function, and t represent the vector. i This represents an entity in the candidate entity set.
[0134] Furthermore, in the link prediction task, the optimized entity embedding vectors are used to predict tail entities. Given the embedding vectors of the head entity and relation, the matching degree is evaluated by calculating their cosine similarity with the embedding vectors of all candidate tail entities. The model selects the tail entity that best matches the head entity and relation, i.e., the tail entity with the highest similarity score, as the prediction result.
[0135] The knowledge graph completion task was completed using the method provided in this embodiment.
[0136] The overall framework diagram for knowledge graph completion based on subgraph sampling and pre-trained language models is as follows: Figure 3 As shown.
[0137] In summary, this invention, through deep semantic representation technology of pre-trained language models and multi-level neighborhood structure feature mining technology—that is, a semantic-structural joint encoding mechanism—achieves in-depth mining and complementarity of the advantages of semantic information and graph structure information. This not only effectively compensates for the representational deficiencies of existing knowledge graph embedding methods in sparse data scenarios but also significantly improves the accuracy of entity link prediction. Furthermore, by introducing adaptive contrastive learning and pre-batch negative sample augmentation techniques, and combining external entity names, types, and textual descriptions, it achieves a significant performance improvement compared to existing technologies. Therefore, this invention brings a new breakthrough to knowledge graph completion technology and provides a more reliable solution for knowledge reasoning and decision support for social and technological development.
[0138] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A knowledge graph completion method based on subgraph sampling and pre-trained language models, characterized in that, Includes the following steps: S1: Obtain entity information of several candidate entities in the knowledge graph, and obtain entity information in JSON format after preprocessing the entity information. The entity information includes entity structure information and entity description information. S2: Take the entity structure information of the JSON format data as input, and perform second-order subgraph sampling on the head entity and the tail entity respectively to obtain the second-order subgraph structure of the head entity and the tail entity. S3: Input the second-order subgraph structures of the head entity and the tail entity into the graph-to-text module for processing to obtain the text sequences corresponding to the second-order subgraph structures of the head entity and the tail entity. S4: Concatenate the text sequence and entity description information of the JSON format data corresponding to the head entity and tail entity to obtain the comprehensive text sequence corresponding to the head entity and tail entity; S5: Concatenate the comprehensive text sequence and relation of the head entity and input them into two independent BERT models for encoding, respectively, to obtain the embedding vectors corresponding to the head entity relation and the tail entity; S6: Take the two embedded vectors as a positive sample pair and input them, along with the negative sample pair obtained by negative sampling, into the contrast learning module to obtain the optimized final vectors corresponding to the head entity relationship and the tail entity. S7: Use the two final vectors to complete the missing information in the knowledge graph.
2. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 1, characterized in that, The implementation process of step S1 includes: S11: Obtain entity structure information, entity description information, and a dictionary of correspondences between entity IDs and entity names from the text file. Split the entity structure information into three fields: head_id, relation, and tail_id, according to tabs. Split the entity description information into two fields: entity_id and entity description, according to tabs. The entity structure information is stored in the form of triples. S12: Normalize the relation field in each triplet to obtain a relation name that conforms to the text specification. The normalization process is to replace all underscores in the relation field with spaces and remove extra whitespace at the beginning and end. S13: After completing the normalization process, each triplet example is encapsulated into an object containing head_id, head entity name, normalized relation name, tail_id, and tail entity name. All such objects are written to the output file in JSON format to obtain entity structure information in JSON format. Similarly, entity_id, entity name, and entity description are written to the output file in JSON format to obtain entity description information in JSON format.
3. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 1, characterized in that, The implementation process of step S2 includes: S21: Randomly sample the first-order neighbors of the head and tail entities; S22: Sampling second-order neighbors from first-order neighbors based on degree importance; S23: Obtain the second-order subgraph structure of the head entity and the tail entity through the first-order neighbors and the second-order neighbors.
4. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 3, characterized in that, The implementation process of step S21 includes: First, obtain all first-order neighbors of the entity to be determined, including the tail entity and the head entity; then, filter out the tail entity from all the first-order neighbors to obtain the remaining first-order neighbors; finally, randomly select a first preset number of neighbor information from the remaining first-order neighbors to obtain the first-order neighbors used to construct the local subgraph. The implementation process of step S22 includes: calculating the degree of the second-order neighbors based on the first-order neighbors used to construct the local subgraph and sorting them to obtain a set of second-order neighbors; selecting a second preset number of second-order neighbors with the highest degree from the set of second-order neighbors to obtain the final second-order neighbors; The implementation process of step S23 includes: merging the first-order neighbors used to construct the local subgraph and the final second-order neighbors to obtain the second-order subgraph structure of the entity to be determined, wherein the entity to be determined includes the tail entity or the head entity.
5. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 1, characterized in that, The implementation process of step S3 includes: S31: Input the second-order subgraph structure of the head entity into the graph-to-text module to convert it into a text sequence, and combine the relationship between the head entity and its neighboring entities to output the text representation of the second-order subgraph of the head entity; S32: Input the second-order subgraph structure of the tail entity into the graph-to-text module to convert it into a text sequence. Combine the relationship between the tail entity and its neighboring entities to output the text representation of the second-order subgraph of the tail entity.
6. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 5, characterized in that, The expression for the second-order subgraph text representation of the head entity is: T h =concat(e h ,r h,1 ,e t,1 ,r′ h,1 ,e′ h,1 ;…;r h,i ,e h,i ,r′ h,i ,e′ h,i ;…;r h,m ,e h,m In the formula, T h The second-order subgraph text sequence representing the head entity, e h Represents the header entity, where i represents the positive integer corresponding to the header entity, and r h,i r′ represents a first-order relation related to the head entity. h,i e represents a second-order relation related to the head entity. h,i e′ represents a first-order neighbor associated with the head entity. h,i This represents the second-order neighbors associated with the head entity, m represents the total number of first-order or second-order neighbors associated with the head entity, and concat represents the join function; The expression for the second-order subgraph text representation of the tail entity is: T t =concat(e t ,r t,1 ,e t,1 ,r′ t,1 ,e′ t,1 ;…;r t,j ,e t,j ,r′ t,j ,e′ t,j ;…;r t,n ,e t,n In the formula, T t The second-order subgraph text representation of the tail entity, e t Represents the tail entity, j represents the positive integer corresponding to the tail entity, r t,j Denotes a first-order relation related to the tail entity, r′ t,j e represents a second-order relation related to the tail entity. t,j e′ represents a first-order neighbor associated with the tail entity. t,j represents the second-order neighbors associated with the tail entity, and n represents the total number of first-order or second-order neighbors associated with the tail entity.
7. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 1, characterized in that, The expression for the comprehensive text sequence corresponding to the header entity is: S h =concat(d(e) h ):T h In the formula, S h This represents the combined text sequence corresponding to the header entity; concat represents the concatenation function; e h Represents the head entity, d(e) h ) represents the descriptive information of the header entity, T h A second-order subgraph text sequence representing the head entity; The expression for the comprehensive text sequence corresponding to the tail entity is: S t =concat(d(e) t ):T t In the formula, S t The e represents the comprehensive text sequence corresponding to the tail entity. t Represents the tail entity, d(e) t T represents the description information of the tail entity. t The second-order subgraph text representation of the tail entity.
8. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 1, characterized in that, The implementation process of step S6 includes: S61: Take the embedding vectors of the head entity relationship and the tail entity as positive sample pairs, and replace the embedding vector of the tail entity with the embedding vector of the erroneous tail entity by negative sampling to generate negative sample pairs. S62: The cosine similarity function is used to process the two embedded vectors of the positive and negative sample pairs to obtain the vector similarity. S63: The loss function optimizes the embedding vector by the vector similarity of positive and negative sample pairs, and obtains the optimized final vector of the head entity relationship and the tail entity correspondence.
9. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 8, characterized in that, The expression for the loss function is: In the formula, log represents the logarithmic function, and e represents the natural constant. Let represent the scoring function for candidate triples, y0 represent the function used to adjust the distance between positive and negative samples, and τ represent the temperature coefficient.
10. The knowledge graph completion method based on subgraph sampling and pre-trained language models according to claim 8, characterized in that, The implementation process of step S7 includes: The two optimized final vectors are used for link prediction, and cosine similarity is used to score all candidate entities. The highest score is used as the predicted tail entity.