A method for knowledge graph completion

Abstract

The present invention provides a method for knowledge graph completion, aiming to improve the accuracy and effectiveness of knowledge graph completion. The method first obtains and expands the triple data of the knowledge graph, and supplements the data set by generating inverse triples. A variety of negative sample generation techniques are combined to improve the model's ability to identify incorrect links, and a dual text encoder is used to perform text embedding learning on entities and relationships in triples. A deformable adaptive attention module is proposed to capture potential features of text features. The triple similarity score is obtained by calculating the cosine similarity score of the text embedding and adding the path reasoning gain score. The information noise contrast estimation loss function with margin is used to optimize the text embedding, which effectively improves the completion effect. Finally, the maximum similarity matching of triples is completed using the trained knowledge graph completion model. The method of the present invention can capture complex semantic associations and enhance the accuracy and robustness of knowledge graph completion.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence knowledge graphs, and in particular, relates to a method for completing a knowledge graph. Background Art

[0002] A knowledge graph is a graphical structure used to store and represent knowledge, which expresses entities and their relationships in the form of nodes and edges. In many application areas, such as search engines, recommendation systems, and intelligent question-answering systems, the application of knowledge graphs has made significant progress. However, despite the excellent performance of knowledge graphs in many aspects, they still face some challenges in practical applications. Existing knowledge graphs are usually constructed by extracting and integrating information from a large number of data sources. However, due to the incompleteness of information, the heterogeneity of data sources, and the limitations of information extraction technology, the constructed knowledge graphs are often incomplete. For example, entities and relationships in the knowledge graph may be missing or inaccurate, which affects its effectiveness in practical applications.

[0003] In order to solve the problem of incompleteness of knowledge graphs, researchers have proposed a variety of knowledge graph completion methods. Existing completion methods mainly include path-based methods, embedding-based methods, and graph neural network-based methods. Path-based methods often make predictions by calculating the number or length of all paths between nodes. Random walk methods capture the relationship between node pairs by randomly walking in the graph. However, these methods may be computationally expensive, and the complexity of the path may cause processing difficulties; embedding-based methods map nodes to low-dimensional space and predict missing links by calculating the similarity of embedded vectors. Node embedding and knowledge graph embedding methods (such as TransE) can capture global structural information and semantic information. However, their performance depends on the quality of the embedding model and the richness of the training data. Graph neural network-based methods use graph neural networks to aggregate information about neighboring nodes and perform link prediction.

[0004] Although many knowledge graph completion methods have been proposed, there are still some problems that need to be solved, such as improving completion accuracy, automating the completion process, and the ability to handle complex relationships. Therefore, a new knowledge graph completion method is urgently needed to further improve the quality and application effect of knowledge graphs. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to propose a method for completing a knowledge graph, solve the problem that the prior art is not fully utilized, and improve the integrity and semantic richness of the knowledge graph through deep learning technology. The method uses a dual-text encoder architecture, a feature gated fusion module, negative sample generation training, a score gain based on path reasoning, and a contrast loss function to complete an incomplete knowledge graph. Specifically, it includes the following steps:

[0006] Step 1: Get entity set data, relationship set data, entity and relationship text description data, and generate inverse triples: t, inverse relationship , h, and add them to the knowledge graph G, and generate negative samples by randomly replacing the tail entity t in the triple;

[0007] Step 2, obtaining the triples and entity set data, relationship set data, entity and relationship text description data described in step 1, using a dual text encoder architecture, the two text encoders use the same Sentence-Bert pre-trained language model for text feature extraction, but the built-in parameters are not shared; the head entity relationship text information in the triple is used as the input of the first text encoder, and the tail entity text information is used as the input of the second text encoder, and the last hidden layer of the two text encoders is extracted as the corresponding text features;

[0008] Step 3: extract the text features in step 2 and generate text embedding through the deformable adaptive attention mechanism;

[0009] Step 4: Using the text embedding obtained in step 3, calculate the cosine similarity score between the head entity relationship embedding and the tail entity embedding, and add the weight gain score based on path reasoning to obtain the triple similarity score;

[0010] Step 5: Based on the triple similarity scores obtained in step 4, an information noise contrast estimation loss function with additional margin is used to optimize the text embedding;

[0011] Step 6: Based on the trained knowledge graph completion model obtained in steps 1 to 5, the cosine similarity score of the text embedding is used to perform triple maximum similarity matching.

[0012] Step 1 includes the following steps:

[0013] In step 1.1, triples (h, r, t) are the basic building blocks of the knowledge graph. Triples are constructed by linking head entities to tail entities through relationships. Entity sets and relationship sets are constructed by extracting head entities and tail entities from the knowledge graph, and extracting link relationships as relationship sets. After deduplication, the final entity sets and relationship sets are obtained. The entity and relationship text description data uses the semantic information provided by WordNet based on text words. Traditional knowledge graph completion usually relies only on structural information, and the semantic information of entities and relationships is insufficient. By introducing the semantic interpretation of WordNet, entities and relationships are mapped to a more specific semantic space, which supplements the content of text descriptions and enhances the understanding of entities and relationships. The rich semantic information enables the model to capture more detailed semantic differences in the representation learning process and reduce its dependence on the surface form of the text. In this way, the subtle differences between different word meanings can be better identified, effectively reducing the problem of semantic fuzzy matching.

[0014] Step 1.2: Convert the positive triple (h, r, t) into the inverse triple (t, r -1 , h), the core of which is r generating r -1 In the present invention, r is converted to r -1 , using the mathematical "non" as the marker of the inverse relationship. Add "non" before the relation text word and swap the head entity with the tail entity to generate an inverse triple. When completing the incomplete knowledge graph G, the entity prediction (h, r, ?) and (?, r, t) are usually used as the target task. Because of the existence of positive and inverse triples, the head entity prediction (?, r, t) can be converted to (t, r -1 ,?), simplified to a task similar to the tail entity prediction task (h, r,?); by combining the forward and reverse triples, the correlation between entity pairs in the knowledge graph is enhanced, effectively improving the model's ability to learn relationships. This expansion strategy can more comprehensively capture the diverse relationships between entities and help improve the accuracy of the knowledge graph completion task.

[0015] Step 1.3, generate negative samples, randomly replace the tail entity t in any triple (h, r, t) to generate negative samples for training the discriminant model. In the present invention, positive and inverse triples can simplify the generation of negative samples for triples, and only the tail entity needs to be replaced to complete the generation of negative samples. The core goal of negative sample training is to help the model learn to distinguish between correct triples (positive samples) and incorrect triples (negative samples). The number of positive samples in the knowledge graph is far less than all possible triple combinations. If negative samples are not used, the model may mistakenly believe that any triple is correct. Through negative sample training, the model will learn which combinations are invalid or wrong, thereby improving the ability to distinguish. Negative sample training forces the model to learn more common semantic relationships and avoid overfitting by adding noise and inaccurate combinations. Comparative learning of positive and negative samples can improve the model's sensitivity to semantic similarity, especially its judgment on subtle differences. The present invention simultaneously uses the following two negative sample generation methods to improve training efficiency:

[0016] Generate negative samples within a batch: Select a batch size and generate negative samples by replacing the tail entity of a triple in the same batch; since the head entity and relationship of a positive triple have a high degree of similarity with the tail entity in the vector space during the embedding learning process, this method can effectively improve the generalization ability of the model. By generating negative samples within a batch, the model can better distinguish between positive and negative samples, thereby optimizing its ability to represent the knowledge graph.

[0017] Generate negative samples in the first batch: Use entities from the previous batch or more to replace the tail entity in the triple to generate more negative samples. Due to the limitation of batch size, the method of generating negative samples within a batch may not provide enough samples for the model. This method generates more negative samples and improves the generalization ability of the model by supplementing the triples with entities from other batches. However, without the dimensional embedding of the head entity relationship and the tail entity in the triple, this may cause a large error between the negative samples and the positive samples, thus affecting the model performance. Therefore, the selection of the number of pre-batch negative samples needs to be cautious.

[0018] Step 2 includes the following steps:

[0019] Step 2.1, concatenate the text words and text descriptions of entities and relationships as the corresponding text information, use the head entity relationship text information as the input of the first text encoder, and the tail entity text information as the input of the second text encoder. The formula is:

[0020]

[0021]

[0022] in, is the head entity relation sentence, is the final entity sentence, It is a head entity word. It is a relational word. It is the final entity word. is a textual description of the head entity, is a textual description of the relationship, is the text description of the tail entity; is a classification tag. is a separator mark;

[0023] The present invention creatively concatenates text words and text descriptions as the input of the encoder. Most previous studies in the knowledge graph direction have not fully utilized the connection between entities and word meanings. Whether it is the early TransE model that lacks rich word meaning information in low-dimensional embedding learning and is difficult to effectively establish the connection between entities, or the knowledge graph completion model based on word meaning overemphasizes the supplementary descriptive information and ignores text words. The present invention chooses to use text words and text descriptions together as the description of text information in the encoder input.

[0024] The dual encoder architecture can handle the dependency of the head entity and the tail entity on the relationship separately. Traditional embedded knowledge graph learning relies on embedding the head entity, relationship, and tail entity in spatial dimensions separately, which causes great errors in the triple matching process. This error not only comes from the fact that there are multiple close associations between an entity pair, but also that a relationship can exist between multiple entity pairs. If the relationship is embedded as an independent entity, the best performance model will not be obtained in dimensional space learning. Therefore, connecting the head entity and the relationship as sentence input greatly avoids the occurrence of the above situation. In the positive and inverse triples, the knowledge graph completion task is simplified to the tail entity prediction task, and under the dual encoder architecture, it is transformed into a semantic matching task.

[0025] In step 2.2, the Sentence-BERT pre-trained language model is used as the text encoder for feature encoding. When the knowledge graph completion task becomes a semantic similarity matching task, the traditional BERT model is not the best choice. The BERT architecture is based on a deeply stacked self-attention layer, and the computational cost is very high, especially when processing hundreds of thousands or even millions of knowledge graph triples. Processing these triples one by one will result in huge computational overhead. In particular, the calculation of semantic similarity requires accurate matching of small differences between entities and relations, and BERT may not be able to effectively distinguish such subtle differences. Therefore, the use of the Sentence-BERT model architecture can overcome these problems. It combines BERT with the Siamese network to generate sentence-level vector representations, which can perform semantic similarity matching more efficiently and accurately, and improve the effect and speed of the knowledge graph completion task. The head entity relationship sentence and the tail entity sentence are used as the input of the first and second text encoders respectively, and the last hidden state of the text encoder is extracted as the text feature of the corresponding text. The formula is:

[0026]

[0027]

[0028] in, It is the functional expression of the first text encoder Sentence-Bert; It is the functional expression of the second text encoder Sentence-Bert; is the input sentence of the first text encoder Head entity relationship text features after the first text encoder; is a sentence consisting of the text word of the tail entity and its text description, yes Tail entity text features after passing through the second text encoder.

[0029] In step 3, the deformable adaptive attention mechanism includes a deformable multi-head attention mechanism and an adaptive gated fusion mechanism; step 3 specifically includes the following steps:

[0030] In step 3.1, a deformable multi-head attention mechanism is introduced to perform offset sampling on text features. The purpose of introducing the deformable multi-head attention mechanism is that the head entity and relationship as fixed sentences need to establish an association with the tail entity. However, different tail entities have different attention features for the head entity relationship sentences, so obtaining potential features is also an important feature extraction. The text features are offset sampled by query features to generate offset samples and offset weights, and these offset samples are bilinearly interpolated with the text features to obtain multi-head sampling features. On this basis, the multi-head sampling features are weighted and the weighted sum of the sampling features is generated. After linear transformation processing, the deformable multi-head attention features are finally obtained. This process effectively improves the model's adaptive modeling ability for features, allowing the model to more flexibly capture the complex relationship between features.

[0031] In step 3.2, the text features are concatenated with the deformable multi-head attention features, and the gating weights are calculated through the activation function. The concatenated features are gated, and the gating weights are adaptively fused with the text features and the deformable multi-head attention features to generate a fused embedding vector, and finally the feature embedding for knowledge graph completion is obtained.

[0032] Step 3.1 includes the following steps:

[0033] Step 3.1.1, calculate the sampling weight and sampling offset by querying the features, the formula is:

[0034]

[0035]

[0036] in, , weight is a function expression of linear transformation, and the output dimension of offset is , the output dimension of weight is ; is a dimensional representation, M is the total number of head attentions, K is the total number of samples in each head attention; X is the query feature; and They represent the sampling weight and sampling offset of the kth sampling point in the mth head attention respectively;

[0037] Step 3.1.2, normalize the offset weight, the formula is:

[0038]

[0039] in, is the sum of K sample weights in the mth head attention, is the sampling weight of the kth sampling point in the mth head attention after normalization;

[0040] Step 3.1.3, offset the sampling of text features, and use bilinear interpolation to calculate multi-head sampling features. The formula is:

[0041]

[0042] in, is the multi-head sampling feature of the k-th sampling point of the m-th head attention; is the function representation of bilinear interpolation; H is the text feature;

[0043] Step 3.1.4, use the following formula for weighted summation:

[0044]

[0045] in, is the weighted sum of the sampled features of the mth head attention;

[0046] Step 3.1.5, through linear transformation processing, the weighted sum of the sampled features is output as a deformable multi-head attention feature, the formula is:

[0047]

[0048] Where D is the deformable multi-head attention feature, is a learnable weight matrix.

[0049] Step 3.2 includes the following steps:

[0050] Step 3.2.1, concatenate the text features with the deformable multi-head attention features, the formula is:

[0051]

[0052] in, is the splicing feature, It is a functional expression of text concatenation;

[0053] Step 3.2.2, the concatenated features are used to calculate the gate weight through the activation function sigmoid, the formula is:

[0054]

[0055] Among them, GATE is the gating weight, is the activation function;

[0056] Step 3.2.3, adaptively fuse the gated weights with the text features and deformable multi-head attention features. The formula is:

[0057]

[0058] Where E is the output embedding vector.

[0059] Step 4 includes the following steps:

[0060] Step 4.1, measure the degree of association by calculating the cosine similarity between the embedded vectors, the formula is:

[0061]

[0062] in, is the cosine similarity function, is the head entity relation embedding, is the tail entity embedding, || || is the calculation norm; Represents vector dot product operation;

[0063] Step 4.2, the present invention proposes a weighted gain score based on path reasoning. In a given knowledge graph, the spatial limitations of the knowledge graph are utilized to capture the spatial distance between entity pairs to generate a weighted gain score:

[0064] = exp[-ɤ·

[0065] in, is the corresponding weight gain score between the head entity h and the tail entity t, ɤ is a hyperparameter, Indicates that the tail entity t is within the L-hop neighborhood of the head entity h; exp is a natural exponential function;

[0066] In step 4.3, the triple similarity score can be calculated through the cosine similarity score and the weight gain score.

[0067]

[0068] in, is the triple similarity score.

[0069] Step 5 includes the following steps,

[0070] The information noise contrast estimation loss function with additional margin is used to optimize the embedding cosine similarity score of positive triples and improve the prediction accuracy of the model. The formula is:

[0071]

[0072] in, is the model loss of cosine similarity embedding, N is the number of negative samples, is the tail entity in the nth negative sample, is an additional margin used to improve the embedding cosine similarity score of positive triples; is a hyperparameter used to adjust the relative importance of negative samples and avoid overfitting learning; e is a natural constant.

[0073] The information noise contrast estimation loss function guides the model to learn more discriminative feature embeddings by maximizing the score difference between positive and negative samples. Combined with the additional margin in the contrast loss, the robustness of the model can be further improved to prevent score confusion. Finally, the network parameters are optimized through gradient descent, making the completion results of the knowledge graph more accurate and efficient.

[0074] In step 5, the present invention optimizes the semantic relevance of triples by capturing the spatial structural information of the knowledge graph network, thereby improving the ability to handle subtle semantic differences. When the head entity and relationship are fixed, the tail entity embedding may not be able to effectively distinguish the most relevant triples due to the high overlap of text content. For triples with shorter path lengths, their relevance is usually higher, while entities with longer paths should show more obvious separation in spatial embedding. A scoring gain mechanism based on path reasoning is introduced to give higher weights to entity pairs with shorter path distances. This path gain scoring method can effectively capture the implicit semantic associations in the graph, allowing the model to use path information for deeper reasoning and enhance the completion effect of the knowledge graph.

[0075] Step 6 includes: using triple maximum similarity matching: matching any tail entity when the head entity relationship is certain, the triple with the maximum score similarity is the potential knowledge graph triple:

[0076]

[0077] in, The function represents finding the maximum value within the target range. is an entity set; It is the i-th tail entity, which is used for similarity matching with the head relation entity.

[0078] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the described method.

[0079] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is run on a computer, the steps of the method described are executed.

[0080] The present invention has the following beneficial effects: First, a method for knowledge graph completion of the present invention effectively integrates entity relationship text words and their text descriptions, supplements and expands the text information in detail, further compensates for the deficiencies in the understanding of word meanings in traditional methods, and thus improves the accuracy of knowledge graph completion.

[0081] Second, a method for knowledge graph completion of the present invention can achieve the semantic similarity matching task of head entity relationship and tail entity through a dual text encoder architecture and using the Sentence-BERT pre-trained model. By using the head entity and relationship as the input of one encoder and the tail entity as the input of another encoder, this method can more effectively capture the semantic association between entities and relationships, greatly improving the performance of the model in the knowledge graph completion task.

[0082] Third, a knowledge graph completion method of the present invention introduces a feature gated fusion module, which can adaptively adjust the weights and positions of each attention head according to different input features. It captures local and global feature correlations between different positions, and combines the gating mechanism to selectively enhance important features, making the final feature embedding more expressive.

[0083] Fourth, a method for knowledge graph completion of the present invention proposes a score gain mechanism for path reasoning, which utilizes multi-hop reasoning information on the path, so that the model can have a deeper understanding of the complex relationship between entities and relationships, and weights the completion results in combination with the path information, thereby significantly improving the reasoning accuracy and model performance in the knowledge graph completion task. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 It is a flow chart of a method for completing a knowledge graph according to an embodiment of the present invention.

[0085] Figure 2 It is a complete network model diagram of a method for completing a knowledge graph in an embodiment of the present invention.

[0086] Figure 3 It is a deformable adaptive attention mechanism module of a knowledge graph completion method in an embodiment of the present invention. DETAILED DESCRIPTION

[0087] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0088] This embodiment discloses a method for completing a knowledge graph, which uses a negative sample training method, a dual-text encoder architecture, a feature gated fusion module, and a path reasoning score gain to effectively complete missing knowledge graph data. To verify the effectiveness and reliability of this embodiment, the knowledge graph public data WN18RR was selected for training and testing.

[0089] See also Figure 1 and Figure 2 , the method of this embodiment includes the following steps:

[0090] Step 1: Obtain entity set data, relationship set data, entity and relationship text description data. Generate inverse triples (t, r) from the triples (h, r, t) of the given knowledge graph G. -1 , h) and add it to the knowledge graph G. Negative samples are generated by randomly replacing the tail entity t in the triple.

[0091] The public dataset WN18RR selected by the present invention is a knowledge graph dataset, which is a modified version of the original WordNet18 dataset. It corrects the problem that positive triples and inverse triples have the same relationship in WordNet18. Since the simple reverse relationship is removed, the task difficulty of the WN18RR dataset is higher, making it more fair and reliable when evaluating the knowledge graph embedding model. According to the statistics of the WN18RR dataset, there are 40,943 entities, 11 relationships, 40,943 training set triples, 3,034 validation set triples, 3,134 test set triples, and a total of 93,003 triples.

[0092] Obtain entity set data, relationship set data, entity and relationship text description data. Entity sets and relationship sets are both refinements of triples in the WN18RR dataset. The head entity and tail entity in the triple are extracted into entity sets, and the relationship is extracted into relationship sets. After deduplication, the final entity set and relationship set are obtained.

[0093] The entity relationship text description dataset is built based on the WordNet vocabulary database, which contains a large number of words and their semantic relationships. Each word in the text description dataset usually corresponds to a specific definition, providing word meaning information input to the text encoder. A large amount of text information can make the predicted triples have a strong connection.

[0094] The core of generating inverse triples from positive triples and adding them to the knowledge graph is the generation of inverse relations. Usually, there are two ways to generate inverse triples: the first one is to manually add an antonym relation r to the dataset based on the semantic information of relation r. -1; The second type, adding "non" before the relation r marks the current relation as an inverse relation word. Although the current dataset WN18RR has a small number of relations and can add datasets manually, the model's generalization ability is weak when relying on external datasets, so the second type of generation method is selected. Add "inverse" before the relation word to mark it as an inverse word, and swap the positions of the head entity and the tail entity, and add them to the complete network.

[0095] Select a sample batch, randomly replace the tail entity in the triple, generate negative samples, and filter the generated samples to exclude positive samples. Because the dual text encoder architecture fixes the head entity and the relationship together, generating negative sample training only requires replacing the tail entity in the positive triple to generate negative samples. There are two main ways:

[0096] The first method is to generate negative samples within a batch. In the same batch, the tail entity in the positive triple is replaced with the entity. This intra-batch replacement method ensures that the entities in the negative samples have a high semantic similarity with the positive samples, thereby improving the generalization ability of the model.

[0097] The second method is to generate negative samples in the first batch. The tail entity in the positive triplet is replaced with the entity in the first two batches. The number of negative samples generated in this way can be flexibly controlled to ensure that the model is exposed to more negative samples during the training process, thereby improving the recognition ability of negative samples. However, too many negative samples generated in the first batch can easily cause the model to learn misleading features, resulting in reduced generalization ability.

[0098] Step 2, obtain the triples and data sets described in step 1, adopt a dual text encoder architecture, use the Sentence-Bert pre-trained language model, take the head entity h and relation r in the triple (h, r, t) as the input of the first text encoder, and the tail entity t as the input of the second text encoder, and extract the last hidden layer of the text encoder as the text feature.

[0099] In step 2, the Sentence-Bert pre-trained language model is used to replace the traditional BERT model to encode text information. The head entity h and relation r in the triple and their text description are used as the input of the first encoder, and the tail entity t and text description are used as the input of the second encoder to obtain the corresponding text features. The steps are mainly divided into the following two steps:

[0100] Step 2.1: Take the triples of the given knowledge graph G and its text description as input to the encoder.

[0101]

[0102]

[0103] in, is the head entity relation sentence, is the final entity sentence, It is a head entity word. It is a relational word. It is the final entity word. is a textual description of the head entity, is a textual description of the relationship, is the text description of the tail entity; is a classification tag. Is a separator mark.

[0104] In step 2.2: Use Sentence-Bert as the text encoder to encode the head entity, relation, tail entity and description in the triple. Extract the features of the last hidden layer to form the text feature representation H.

[0105]

[0106] in, It is the functional expression of the first text encoder Sentence-Bert; It is the functional expression of the second text encoder Sentence-Bert; is the input sentence of the first text encoder Head entity relationship text features after the first text encoder. is a sentence consisting of the text word of the tail entity and its text description, yes Tail entity text features after passing through the second text encoder.

[0107] like Figure 3 As shown, the specific functions of the deformable adaptive attention mechanism module are as described in step 3:

[0108] Step 3: extract the text features in step S2, introduce the deformable multi-head attention mechanism, capture the potential features of the text, obtain the deformable multi-head attention features, and perform adaptive gating fusion with the text features to generate text embedding. The steps are as follows:

[0109] Step 3.1, based on the extracted text features, use the deformable multi-head attention mechanism to enhance the feature modeling capability. This mechanism can flexibly adjust the weight and sampling position of each attention head by offsetting the sampling position, thereby capturing more complex feature associations. The input text feature H is offset sampled by the query feature to obtain the multi-head offset feature. In each attention head m, the position offset of the kth sampling point is adjusted by the deformable mechanism , and assign sampling weights The sampled multi-head offset features are transformed linearly to obtain the deformable multi-head attention output . Its formula is as follows:

[0110]

[0111] Among them, M is the total number of head attentions of multi-head attention, and K is the total number of samples of each head attention; and They represent the sampling weight and sampling offset of the kth sampling point in the mth head attention, is a learnable weight matrix; is the function representation of the bilinear interpolation method. D is the deformable multi-head attention feature;

[0112] The detailed steps include:

[0113] Step 3.1.1, calculate the sampling weight and sampling offset by querying the features, the formula is:

[0114]

[0115]

[0116] in, , weight is a function expression of linear transformation, but the output dimension of offset is , the output dimension of weight is ; is a dimensional representation, M is the total number of head attentions, K is the total number of samples in each head attention; X is the query feature; and They represent the sampling weight and sampling offset of the kth sampling point in the mth head attention respectively;

[0117] Step 3.1.2, normalize the offset weight, the formula is:

[0118]

[0119] in, is the sum of K sample weights in the mth head attention, is the sampling weight of the kth sampling point in the mth head attention after normalization;

[0120] Step 3.1.3, offset the sampling of text features, and use bilinear interpolation to calculate multi-head sampling features. The formula is:

[0121]

[0122] in, is the multi-head sampling feature of the k-th sampling point of the m-th head attention; is the function representation of bilinear interpolation; H is the text feature;

[0123] Step 3.1.4, use the following formula for weighted summation:

[0124]

[0125] in, is the weighted sum of the sampled features of the mth head attention; K is the total number of samples in each head attention;

[0126] Step 3.1.5, through linear transformation processing, the weighted sum of the sampled features is output as a deformable multi-head attention feature, the formula is:

[0127]

[0128] Where D is the deformable multi-head attention feature, is a learnable weight matrix;

[0129] Step 3.2, concatenate the text feature H encoded by Sentence-BERT with the deformable multi-head attention output D, and adaptively fuse them through the gating mechanism to generate the final embedding vector. The two features are concatenated and passed through the linear layer and the Sigmoid activation function to calculate the gate weight GATE, which controls the aggregation process of the text feature and the attention feature D.

[0130] Step 3.2.1: Concatenate text features with deformable multi-head attention features.

[0131]

[0132] in, is the splicing feature, is the function expression of text concatenation, is the text feature, and D is the deformable multi-head attention feature.

[0133] Step 3.2.2: The concatenated features are passed through the activation function sigmoid to calculate the gating weight.

[0134]

[0135] Among them, GATE is the gating weight, is the activation function.

[0136] Step 3.2.3: Adaptively fuse the gated weights with the text features and deformable multi-head attention features.

[0137]

[0138] Where E is the output embedding vector.

[0139] Step 4, calculate the cosine similarity score between the head entity relationship embedding and the tail entity embedding, and add a weight gain score based on the path distance between the head entity h and the tail entity t during the calculation process.

[0140] In step 4.1, the head entity relationship embedding is calculated by cosine similarity With tail entity embedding The correlation between them is as follows:

[0141]

[0142] in, is the cosine similarity function, is the head entity relation embedding, is the tail entity embedding, || || is the calculation norm, Represents a vector dot product operation.

[0143] In step 4.2, in the knowledge graph completion task, the weighted gain score can be used to enhance the reasoning relationship between entities. Capturing the spatial limitations between entities enables better reasoning and prediction in the knowledge graph. The smaller the spatial distance, the stronger the association between entities, indicating that their positions in the knowledge graph are closer. In order to measure the impact of this spatial distance on the entity pair relationship, a formula is used to convert the spatial distance into a weighted gain score, as follows:

[0144] = exp[-ɤ·

[0145] in, is the corresponding weight gain score between the head entity h and the tail entity t, ɤ is a hyperparameter, The tail entity t is located within the L-hop neighborhood of the head entity h; exp is the natural exponential function.

[0146] In step 4.3, the triple similarity score is calculated by the cosine similarity score and the weight gain score.

[0147]

[0148] in, is the triple similarity score.

[0149] In step 5, the information noise contrast estimation loss function with additional margin is used to maximize the similarity of positive samples and minimize the similarity of negative samples to learn the representation of triples. Improve the similarity of positive triplets, thereby further widening the similarity gap between positive and negative samples during the optimization process and improving the prediction accuracy of the model. The formula is as follows:

[0150]

[0151] in, is the model loss of cosine similarity embedding, N is the number of negative samples, is the tail entity in the nth negative sample, is the additional margin used to improve the positive triple score; is a hyperparameter used to adjust the relative importance of negative samples and avoid overfitting learning; e is a natural constant.

[0152] Step 6: In the tail entity prediction task, by considering the spatial structure information in the knowledge graph, we can effectively improve the ability to handle subtle differences, further optimize the accuracy of prediction, and select the tail entity embedding that best matches the semantic association. In the given set of tail entity candidates, select the tail entity t with the highest cosine similarity through the argmax operation:

[0153]

[0154] in, The function represents finding the maximum value within the target range. Is an entity set. It is the i-th tail entity, which is used for similarity matching with the head relation entity.

[0155] The knowledge graph completion model obtained after training in the above steps uses MRR (mean reciprocal ranking), H@1, H@3, and H@10 (H@A indicates whether the correct answer is in the first A candidate results returned by the model) as common evaluation indicators for measuring model performance in the knowledge graph completion task, and the performance of this method is evaluated. The specific table is shown in Table 1. The results show that a knowledge graph completion method of this embodiment has strong capabilities and effects in improving the knowledge graph.

[0156] Table 1

[0157]

[0158] Among them, TransE: maps entities and relations to the same vector space, aiming to capture the relationship between triples. DistMult: uses bilinear mapping to model the interaction of entities and relations. Prediction is achieved by calculating the inner product between the head entity and the tail entity and the relation in the embedding space. RotatE: treats relations as rotation operations in complex vectors to enhance representation capabilities. TuckER: uses tensor decomposition to process knowledge graph embedding and handles multiple relationship types. TuckER: generates predictions of triples by embedding entities and relations. KG-BERT: uses BERT's contextual representation capabilities to process triple information in knowledge graphs. MTL-KGC: a multi-task learning framework that simultaneously solves knowledge graph completion and other related tasks and exploits shared knowledge between tasks. StAR: captures structural information in knowledge graph embedding and enhances the representation capabilities of embeddings through an adaptive aggregation mechanism. SimKGC: uses a contrastive learning method to improve the embedding effect of knowledge graphs by maximizing the similarity between similar entities and minimizing the similarity between dissimilar entities. C-LMKE: uses text information to enhance the representation capabilities of entities. LP-BERT: Considers the relationship between nodes to improve the effect of knowledge graph reasoning. KGLM: Combines the characteristics of knowledge graph and language model to improve the performance on specific tasks. LERP: Improves the ability to model complex relationships by introducing linear embedding and relationship smoothing.

[0159] The present invention provides a method for completing a knowledge graph. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A method for completing a knowledge graph, characterized in that: The steps include: Step 1: Get entity set data, relationship set data, entity and relationship text description data, and generate inverse triples: t, inverse relationship , h, and add them to the knowledge graph G, and generate negative samples by randomly replacing the tail entity t in the triple; Step 2, obtaining the triples and entity set data, relationship set data, entity and relationship text description data described in step 1, using a dual text encoder architecture, the two text encoders use the same Sentence-Bert pre-trained language model for text feature extraction, but the built-in parameters are not shared; the head entity relationship text information in the triple is used as the input of the first text encoder, and the tail entity text information is used as the input of the second text encoder, and the last hidden layer of the two text encoders is extracted as the corresponding text features; Step 3: extract the text features in step 2 and generate text embedding through the deformable adaptive attention mechanism; Step 4: Using the text embedding obtained in step 3, calculate the cosine similarity score between the head entity relationship embedding and the tail entity embedding, and add the weight gain score based on path reasoning to obtain the triple similarity score; Step 5: Based on the triple similarity scores obtained in step 4, an information noise contrast estimation loss function with additional margin is used to optimize the text embedding; Step 6: Based on the trained knowledge graph completion model obtained in steps 1 to 5, the cosine similarity score of the text embedding is used to perform triple maximum similarity matching.

2. The method according to claim 1, characterized in that Step 1 includes the following steps: In step 1.1, the triple (h, r, t) is the basic component unit of the knowledge graph. The triple is linked to the tail entity through the head entity through the relationship to form the basic data; the entity set and relationship set are constructed by extracting the head entity and the tail entity from the knowledge graph, and extracting the link relationship as a relationship set. After deduplication, the final entity set and relationship set are obtained; the entity and relationship text description data uses the semantic information provided by WordNet based on the text words; Step 1.2: Convert the positive triple (h, r, t) into the inverse triple (t, r -1 ,h), add non before the relation text word and swap the position of the head entity with the tail entity to generate an inverse triple; Step 1.3, generate negative samples, randomly replace the tail entity t in any triple (h, r, t) to generate negative samples for training the discriminant model, and use the following two negative sample generation methods: Generate negative samples within a batch: Select a batch size and generate negative samples by replacing the tail entity of a triple in the same batch; Generate negative samples from the previous batch: Use entities from the previous one or more batches to replace the tail entity in the triplet to generate more negative samples.

3. The method according to claim 2, characterized in that Step 2 includes the following steps: Step 2.1, concatenate the text words and text descriptions of entities and relationships as the corresponding text information, use the head entity relationship text information as the input of the first text encoder, and the tail entity text information as the input of the second text encoder. The formula is: ， ， in, is the head entity relation sentence, is the final entity sentence, It is a head entity word. It is a relational word. It is the final entity word. is a textual description of the head entity, is a textual description of the relationship, is the text description of the tail entity; is a classification tag. is a separator mark; Step 2.2: Use the Sentence-BERT pre-trained language model as the text encoder for feature encoding, and extract the last hidden state of the text encoder as the text feature of the corresponding text. The formula is: ， ， in, It is the functional expression of the first text encoder Sentence-Bert; It is the functional expression of the second text encoder Sentence-Bert; is the input sentence of the first text encoder Head entity relationship text features after the first text encoder; is a sentence consisting of the text word of the tail entity and its text description, yes Tail entity text features after passing through the second text encoder.

4. The method according to claim 3, characterized in that In step 3, the deformable adaptive attention mechanism includes a deformable multi-head attention mechanism and an adaptive gated fusion mechanism; step 3 specifically includes the following steps: In step 3.1, a deformable multi-head attention mechanism is introduced to offset the sampling of text features; In step 3.2, the text features are concatenated with the deformable multi-head attention features, and the gating weights are calculated through the activation function. The concatenated features are gated, and the gating weights are adaptively fused with the text features and the deformable multi-head attention features to generate a fused embedding vector, and finally the feature embedding for knowledge graph completion is obtained.

5. The method according to claim 4, characterized in that Step 3.1 includes the following steps: Step 3.1.1, calculate the sampling weight and sampling offset by querying the features, the formula is: ， ， in, , weight is a function expression of linear transformation, and the output dimension of offset is , the output dimension of weight is ; is a dimensional representation, M is the total number of head attentions, K is the total number of samples in each head attention; X is the query feature; and They represent the sampling weight and sampling offset of the kth sampling point in the mth head attention respectively; Step 3.1.2, normalize the offset weight, the formula is: ， in, is the sum of K sample weights in the mth head attention, is the sampling weight of the kth sampling point in the mth head attention after normalization; Step 3.1.3, offset the sampling of text features, and use bilinear interpolation to calculate multi-head sampling features. The formula is: ， in, is the multi-head sampling feature of the k-th sampling point of the m-th head attention; is the function representation of bilinear interpolation; H is the text feature; Step 3.1.4, use the following formula for weighted summation: ， in, is the weighted sum of the sampled features of the mth head attention; Step 3.1.5, through linear transformation processing, the weighted sum of the sampled features is output as a deformable multi-head attention feature, the formula is: ， Where D is the deformable multi-head attention feature, is a learnable weight matrix.

6. The method according to claim 5, characterized in that Step 3.2 includes the following steps: Step 3.2.1, concatenate the text features with the deformable multi-head attention features, the formula is: ， Among them, AF is the splicing feature, and Com is the function expression of text splicing; Step 3.2.2, the concatenated features are used to calculate the gate weight through the activation function sigmoid, the formula is: ， Among them, GATE is the gating weight, is the activation function; Step 3.2.3, adaptively fuse the gated weights with the text features and deformable multi-head attention features. The formula is: ， Where E is the output embedding vector.

7. The method according to claim 6, characterized in that Step 4 includes the following steps: Step 4.1, measure the degree of association by calculating the cosine similarity between the embedded vectors, the formula is: ， in, is the cosine similarity function, is the head entity relation embedding, is the tail entity embedding, || || is the calculation norm; Represents vector dot product operation; Step 4.2, in a given knowledge graph, exploit the spatial limitations of the knowledge graph and generate a weighted gain score by capturing the spatial distance between entity pairs: = exp[-ɤ· , in, is the corresponding weight gain score between the head entity h and the tail entity t, ɤ is a hyperparameter, Indicates that the tail entity t is within the L-hop neighborhood of the head entity h; exp is a natural exponential function; Step 4.3, calculate the triple similarity score through the cosine similarity score and the weight gain score: ， in, is the triple similarity score.

8. The method according to claim 7, characterized in that Step 5 includes: The information noise contrast estimation loss function with additional margin is used to optimize the embedding cosine similarity score of positive triples and improve the prediction accuracy of the model. The formula is: ， in, is the model loss of cosine similarity embedding, N is the number of negative samples, is the tail entity in the nth negative sample, is the additional margin; is a hyperparameter; e is a natural constant.

9. The method according to claim 8, characterized in that Step 6 also includes: Use triple maximum similarity matching: When the head entity relationship is certain, match any tail entity, and the one with the maximum score similarity is the potential knowledge graph triple: ， in, The function represents finding the maximum value within the target range. is an entity set; It is the i-th tail entity, which is used for similarity matching with the head relation entity.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

Images