Induction relation prediction method for efficient negative sampling and retrieval fusion context path

Abstract

The invention relates to an efficient negative sampling and retrieval fused context path inductive relationship prediction method, and belongs to the field of knowledge graph link prediction. The method comprises the following steps: carrying out semantic perception negative sampling; extracting and filtering a path; extracting and filtering a context; performing detailed information retrieval based on a large language model; sentence generation; performing embedding generation; and generating a prediction result and training. According to the method, a more effective negative triple is generated by introducing a semantic perception negative sampling technology, the limitation of traditional random negative sampling is overcome, paths and multi-hop contexts are extracted and filtered from the query triple, long and short entities and relation description are combined into sentences, the multi-hop contexts of a head entity and a tail entity are innovatively sampled, and the sentence extraction efficiency is improved. And irrelevant contents are filtered based on similarity, so that the problem of path scarcity or missing is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph link prediction, and specifically relates to an inductive relationship prediction method of an efficient negative sampling and retrieval fusion context path. Background Art

[0002] Knowledge Graphs (KGs) are directed graph structures with entities as nodes and relationships as edges, and are widely used in many fields. However, with the exponential growth of the scale of real-world data, current knowledge graphs have gradually exposed their limitations. For example, in the face of the continuous addition of new entities, existing methods are usually limited to the transductive framework and can only predict the entities and relationships already in the training data, which requires frequent retraining of the entire graph, which is inefficient and difficult to adapt to dynamically changing scenarios. To this end, researchers proposed inductive link prediction, which can better cope with the dynamic and open knowledge graph environment by using known relationship structures to predict the relationships of unseen entities.

[0003] Our proposed related method KRST first extracts and filters unreliable paths, and converts paths and triples into sentences using entity and relation descriptions. These sentences are embedded by sentence transformers, and relation prediction is performed using cosine similarity scores. However, the KRST method still has the following shortcomings: (1) Random negative sampling may generate negative samples that lack practical meaning, increase training noise, and limit the generalization ability and prediction accuracy of the model; (2) Existing text descriptions are difficult to fully cover the diversity of entities and relations, resulting in insufficient performance of the model in understanding and predicting complex relations; (3) Relying only on closed paths to connect query head entities and tail entities is susceptible to path missing or sparsity problems, which adversely affects model performance. Therefore, it is of great theoretical value and practical significance to study an inductive relation prediction method that integrates efficient negative sampling and retrieval of contextual paths. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an inductive relationship prediction method of an efficient negative sampling and retrieval fusion context path.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions: An inductive relation prediction method for efficiently negative sampling and retrieval fusion context path includes the following steps: S1. Semantic-aware negative sampling: Negative sampling is performed through semantic awareness to generate negative sampling triplets that are the least similar to the query triplet; S2. Path extraction and filtering: Use the breadth-first search algorithm to extract the path information of the query triple and the negative sampling triple generated in step S1, and filter the paths according to the relationship confidence to obtain the path set of the query triple and its negative sampling triple; S3. Context extraction and filtering: Through a recursive multi-hop context search algorithm, the context triples of the query triple and the negative sampling triple generated in step S1 are explored respectively, and filtered according to cosine similarity to obtain the context triples of the query triple and its negative sampling triple; S4. Detailed information retrieval based on the large language model: Input the short description of the entity into the large language model to generate the corresponding long description of the entity; S5. Sentence generation: Combining the short description of the entity, the relationship description, and the long description of the entity generated in step S4, the query triple and its negative sampling triple, the path set of the query triple and its negative sampling triple extracted in step S2, and the context triple of the query triple and its negative sampling triple generated in step S3 are all converted into natural language sentences to obtain the query triple and its negative sampling triple sentence, the path sentence, and the context sentence; S6. Embedding generation: Input the natural language sentence generated in step S5 into the Sentence Transformer model to generate the query triple and its negative sampling triple sentence embedding vector, the path sentence embedding vector and the context sentence embedding vector; S7. Generate prediction results and train: Obtain the final result of the inductive relationship prediction by calculating the cosine similarity scores between the path sentence embedding vector and the context sentence embedding vector generated in step S6 and the query triple and its negative sampling triple sentence embedding vector; Optimize the model using the loss function.

[0006] Furthermore, step S1 specifically includes: S11. Each entity in the knowledge graph The corresponding short description Input BERT model to generate entity embedding ; S12. In head entity mode, from the entity set of the knowledge graph Select the tail entity in the query of Negative sampling tail entity , and based on the similarity score Sort in ascending order, and query the tail entity The most dissimilar entities are sorted first, from the lowest similarity to the lowest End entity Generate negative sampling triples in head entity mode ; Similarly, in the tail entity mode, generate the negative sampling triplet set in the tail entity mode ; The formula is as follows: , , in, Represents the set of negative sampling triplets in head entity mode, The negatively sampled tail entity representing the set of negatively sampled triplets in head entity mode, It means taking the first k values. Represents a collection of entities, Represents the query tail entity Entity embedding, express Negative sampling tail entity Entity embedding, Indicates the number of entities. represents the set of negative sampling triplets in tail entity mode, The negative sampling head entity representing the set of negative sampling triplets in tail entity mode, Represents a query header entity Entity embedding, express Negative sampling head entity Entity embedding.

[0007] Furthermore, in step S2, a breadth-first search algorithm is used to extract the path information of the query triple and the negative sampling triple generated in step S1, and the path is filtered according to the relationship confidence. The path extraction and filtering process is as follows: S21. Path Extraction: In the head entity mode, trace step by step through relations and entities alternately to the tail entity , search for query triples in the knowledge graph using a breadth-first search algorithm De novo entities To the end entity Path ,path Expressed as , from the path Extracting De Novo Entities To the end entity The relationship path , relationship path Expressed as ; Similarly, in the tail entity mode, get the tail entity To the end entity The relationship path ; S22. Path filtering: Use the relationship path confidence function to calculate the confidence of the relationship path, the relationship path confidence in the head entity mode The formula is as follows: , in, Represents a relationship path The number of triplets of Represents a relationship path The number of triplets in the head entity To the end entity The relationship path The relationship paths whose relationship path confidences in the head entity mode are greater than or equal to the preset relationship path confidence threshold are screened out to obtain the valid paths in the head entity mode; the preset relationship path confidence threshold is set to 5×10 -3 ; Similarly, obtain the valid path in tail entity mode.

[0008] Similarly, according to the query triple In the path extraction and filtering process, the negative sampling triplet generated in step S1 is subjected to path extraction and filtering to obtain the corresponding valid path.

[0009] Furthermore, in step S3, a recursive multi-hop context search algorithm is used to obtain context triples of the query triple and the negative sampling triple generated in step s1, and filter them according to cosine similarity; the context triple search process of the query triple is as follows: S31. Context extraction: recursively apply a multi-hop context search algorithm to explore query triples The neighborhood context of the head entity and the tail entity. The specific steps are as follows: According to the query triple , initialize the head entity node collection and tail entity node set ; Head entity node collection To include the header entity A collection of , tail entity node set To include the tail entity A collection of ; Create an empty header entity context triple set at the same time and an empty set of tail entity context triples , used to store the neighborhood context triples of the head entity and the tail entity; In the single-hop context triple construction, starting from the query head entity and the query tail entity, the nodes and relationships directly connected to them are extracted respectively; for the head entity node set , extract the predecessor node connected to it from the knowledge graph and relationship , forming a triple ; At the same time, extract the successor node connected to it and relationship , generating triples ; Add these triples to the head entity context triple set , and extract the predecessor node and successor nodes Add to the new head entity node collection Similarly, for the tail entity node set , extract the predecessor node and successor node of each node and their relationship, generate triples and add them to the tail entity context triple set , and update the new tail entity node set at the same time ; Through this process, the single-hop stage completes the expansion of the direct neighborhood relationship between the query head entity and the tail entity, forming an initial set of context triples.

[0010] In the process of constructing multi-hop context triples, the head entity node set generated in the single-hop stage is and tail entity node set , assign it to the current head entity node set and tail entity node set , and and Set to an empty set, in In this hop, for the current head entity node set and tail entity node set , extract the predecessor node and successor node and relationship, generate the corresponding triples and add them to the head entity context triple set respectively and tail entity context triples At the same time, the new node is added to the updated head entity node set and tail entity node set ; By recursively expanding the neighborhood relationship through multiple jumps, a multi-hop context triple set containing deep semantic information is finally constructed and . ; The final head entity context collection The context triples of the and tail entities Merge into a new set of context triples ; If the query triple Already exists in , then remove it; S32. Context filtering: Use the pre-trained BERT model to filter new context triples. Each context triple in and query triples Encode the entities and relations in the , generate the embedding vectors for each entity and relation, and store them in the dictionary. The formula is as follows: , , in, , , Represents context triples The embedding vectors of the head entity, relation, and tail entity in , , , Represents a query triple The embedding vectors of the head entity, relation, and tail entity in the calculation context triple and query triples The embedding vector of is expressed as follows: , in, represents the context triple embedding vector, Represents the query triple embedding vector; calculates the context triple embedding vector and query triplet embedding vector The cosine similarity of is expressed as follows: , When the cosine similarity is greater than or equal to the preset threshold of 0.9, the context triplet and the similarity score corresponding to the context triplet embedding vector are added to the similarity context triplet set. In; according to the similarity score Sort the context triples in descending order and select the first triples, as the final set of context triples ; Similarly, according to step S31 and step S32, the final context triplet set of the negative sampling triples generated in the head entity mode and the tail entity mode of step S1 is obtained.

[0011] Furthermore, in step S4, the short description of each entity in the knowledge graph is transformed into Generate long description .

[0012] Further, in step S5, the query triple and its negative sampling triple, the path set of the query triple and its negative sampling triple extracted in step S2, and the context triple of the query triple and its negative sampling triple generated in step S3 are all converted into natural language sentences through the entity short description, the relationship description and the long description generated in step S4; the query triple, the path set of the query triple and the context triple of the query triple are converted into natural language sentences, including four scenarios, and the specific methods are as follows: S51. Query triple sentence generation: The sentence starts with a [CLS] token, followed by a short description of the query triple head entity and long description ; Next is the query triple relationship description ; Finally, the short description of the end entity of the query triple and long description ; The head entity and its short and long descriptions, the relationship and its description, the tail entity and its short and long descriptions are separated by the [SEP] tag; similarly, negative sampling triplets are obtained; sentence generation; S52. Path sentence generation: In head entity mode from query head entity to query tail entity, a sentence starts with a [CLS] token, followed by a short description of the query triple head entity. and long description ; Then the first relationship and its relationship description in the valid path in the obtained header entity mode ; followed by a short description of the intermediate entity corresponding to the first relationship and long description , repeat for each subsequent relationship and entity until the short description of the tail entity is reached and long description ; The head entity and its short description and long description, the first relationship and relationship description, the first intermediate entity and its short description and long description, the second relationship and relationship description, the second intermediate entity and its short description and long description, ..., the nth relationship and relationship description, the tail entity and its short description and long description, each part is separated by the [SEP]} tag; similarly, the path sentence in the tail entity mode from the tail entity to the head entity is obtained; S53. Context sentence generation: The context information of the query triple extracted in step S3 is represented in the form of triples, which are specifically divided into the following five types: a single-hop triple consisting of the predecessor node of the query head entity, a single-hop triple consisting of the successor node of the query head entity, a single-hop triple consisting of the predecessor node of the query tail entity, a single-hop triple consisting of the successor node of the query tail entity, and a triple generated by multi-hop expansion that does not contain the query head entity and the tail entity; a single-hop triple consisting of the predecessor node of the query head entity The sentence generation format is: the sequence starts with [CLS]}, followed by a short description of the head entity and long description , relationships and relationship descriptions , and a short description of the tail entity and long description , each part is separated by the [SEP]} tag; similarly, the sentence generation is obtained by obtaining the single-hop triples composed of the successor nodes of the query head entity, the single-hop triples composed of the predecessor nodes of the query tail entity, and the single-hop triples composed of the successor nodes of the query tail entity; query single-hop triples Single-hop triplet of the successor node of the middle-tail entity , that is, querying the head entity and query tail entity Unrelated multi-hop triples, whose sentence generation format is: the sequence starts with [CLS]}, followed by a short description of the head entity and long description , relationships and relationship descriptions , and a short description of the successor entity and long description , each part is separated by the [SEP]} mark; Similarly, we obtain the context sentence generation of the negative sampling triples generated in the head entity mode and the tail entity mode; S54. Context-free sentence generation: When the path and context of the query triple do not exist, the sentence only contains [CLS] and [SEP] tokens; Furthermore, in step S7, the cosine similarity is used to measure the similarity between the query triplet and its negative sampling triplet sentence embedding vector obtained in step S6 and the path sentence embedding vector, and the formula is as follows: in, is to embed the query triple and its negative sampling triple sentence into a vector, Represents the path sentence embedding vector; similarly, obtain the similarity between the query triple and its negative sampling triple sentence embedding vector and the context sentence embedding vector; take the highest similarity score as the query triple The final score is expressed as follows: , in, Representing and querying triples The embedding vector of the sentence generated by the path or context triplet, Represents a query triple The path or index of the context triplet, represents the final score of the query triple,L represents the number of all path or context triplets, Indicates that the maximum value is selected from all similarity scores; similarly, the final scores of the negative sampling triplets generated under the head entity mode and the tail entity mode are obtained, and the average of the final scores of the negative sampling triplets generated under the head entity mode and the tail entity mode is taken as the final result of the inductive relationship prediction; the training process is optimized by the cosine embedding loss function, and the formula is expressed as follows: in, Represents the minimum interval between positive and negative samples, Represents the label of the sample, where 1 represents a positive sample and -1 represents a negative sample.

[0013] The advantages of the present invention are: The method of the present invention introduces semantic-aware negative sampling to generate more effective negative triples, which solves the limitations of random negative sampling. Subsequently, the path and multi-hop context are extracted and filtered from the query triple, and the long and short entities and relationship descriptions are combined and converted into sentences. These sentences are embedded through a sentence converter and aggregated through cosine similarity to form triple scores. We innovatively sample the multi-hop context of the head entity and the tail entity, and filter irrelevant content based on similarity, solving the problem of scarce or missing paths. Experimental results show that the method of the present invention outperforms other models on all datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0015] Figure 1 Flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Example 1 In this embodiment, Figure 1 As shown, the present invention provides an inductive relationship prediction method for efficiently negative sampling and retrieval fusion context path, and the specific steps include: S1. Semantic-aware negative sampling: Negative sampling is performed through semantic awareness to generate negative sampling triplets that are the least similar to the query triplet; Specifically, S11. transforms each entity in the knowledge graph into The corresponding short description Input BERT model to generate entity embedding ; S12. In head entity mode, from the entity set of the knowledge graph Select the tail entity in the query of Negative sampling tail entity , and based on the similarity score Sort in ascending order, and query the tail entity The most dissimilar entities are sorted first, from the lowest similarity to the lowest End entity Generate negative sampling triples in head entity mode ; Similarly, in the tail entity mode, generate the negative sampling triplet set in the tail entity mode ; The formula is as follows: , , in, Represents the set of negative sampling triplets in head entity mode, The negatively sampled tail entity representing the set of negatively sampled triplets in head entity mode, It means taking the first k values. Represents a collection of entities, Represents the query tail entity Entity embedding, express Negative sampling tail entity Entity embedding, Indicates the number of entities. represents the set of negative sampling triplets in tail entity mode, The negative sampling head entity representing the set of negative sampling triplets in tail entity mode, Represents a query header entity Entity embedding, express Negative sampling head entity Entity embedding.

[0018] S2. Path extraction and filtering: Use the breadth-first search algorithm to extract the path information of the query triple and the negative sampling triple generated in step S1, and filter the paths according to the relationship confidence to obtain the path set of the query triple and its negative sampling triple; Specifically, the breadth-first search algorithm is used to extract the path information of the query triple and the negative sampling triple generated in step S1, and the path is filtered according to the relationship confidence. The path extraction and filtering process is as follows: S21. Path extraction: In head entity mode, alternately extracting from head entities through relations and entities Tracking to the end entity step by step , search for query triples in the knowledge graph using a breadth-first search algorithm De novo entities To the end entity Path ,path Expressed as , from the path Extracting De Novo Entities To the end entity The relationship path , relationship path Expressed as ; Similarly, in the tail entity mode, get the tail entity To the end entity The relationship path ; S22. Path filtering: Use the relationship path confidence function to calculate the confidence of the relationship path, the relationship path confidence in the head entity mode The formula is as follows: , in, Represents a relationship path The number of triplets of Represents a relationship path The number of triplets in the head entity To the end entity The relationship path The relationship paths whose relationship path confidences in the head entity mode are greater than or equal to the preset relationship path confidence threshold are screened out to obtain the valid paths in the head entity mode; the preset relationship path confidence threshold is set to 5×10 -3 ; Similarly, obtain the valid path in tail entity mode.

[0019] Similarly, according to the query triple In the path extraction and filtering process, the negative sampling triplet generated in step S1 is subjected to path extraction and filtering to obtain the corresponding valid path.

[0020] S3. Context extraction and filtering: Through a recursive multi-hop context search algorithm, the context triples of the query triple and the negative sampling triple generated in step S1 are explored respectively, and filtered according to cosine similarity to obtain the context triples of the query triple and its negative sampling triple; Specifically, through the recursive multi-hop context search algorithm, the context triples of the query triple and the negative sampling triple generated in step s1 are obtained and filtered according to the cosine similarity; the context triple search process of the query triple is as follows: S31. Context extraction: recursively apply a multi-hop context search algorithm to explore query triples The neighborhood context of the head entity and the tail entity. The specific steps are as follows: According to the query triple , initialize the head entity node collection and tail entity node set ; Head entity node collection To include the header entity A collection of , tail entity node set To include the tail entity A collection of ; Create an empty header entity context triple set at the same time and an empty set of tail entity context triples , used to store the neighborhood context triples of the head entity and the tail entity; In the single-hop context triple construction, starting from the query head entity and the query tail entity, the nodes and relationships directly connected to them are extracted respectively; for the head entity node set , extract the predecessor node connected to it from the knowledge graph and relationship , forming a triple ; At the same time, extract the successor node connected to it and relationship , generating triples ; Add these triples to the head entity context triple set , and extract the predecessor node and successor nodes Add to the new head entity node collection Similarly, for the tail entity node set , extract the predecessor node and successor node of each node and their relationship, generate triples and add them to the tail entity context triple set , and update the new tail entity node set at the same time ; Through this process, the single-hop stage completes the expansion of the direct neighborhood relationship between the query head entity and the tail entity, forming an initial set of context triples.

[0021] In the process of constructing multi-hop context triples, the head entity node set generated in the single-hop stage is and tail entity node set , assign it to the current head entity node set and tail entity node set , and and Set to an empty set, in In this hop, for the current head entity node set and tail entity node set , extract the predecessor node and successor node and relationship, generate the corresponding triples and add them to the head entity context triple set respectively and tail entity context triples At the same time, the new nodes are added to the updated head entity node set and tail entity node set ; By recursively expanding the neighborhood relationship through multiple jumps, a multi-hop context triple set containing deep semantic information is finally constructed and . ; The final head entity context collection The context triples of the and tail entities Merge into a new set of context triples ; If the query triple Already exists in , then remove it; S32. Context filtering: Use the pre-trained BERT model to filter new context triples. Each context triple in and query triples Encode the entities and relations in the , generate the embedding vectors for each entity and relation, and store them in the dictionary. The formula is as follows: , , in, , , Represents context triples The embedding vectors of the head entity, relation, and tail entity in , , , Represents a query triple The embedding vectors of the head entity, relation, and tail entity in the calculation context triple and query triples The embedding vector of is expressed as follows: , in, represents the context triple embedding vector, Represents the query triple embedding vector; calculates the context triple embedding vector and query triplet embedding vector The cosine similarity of is expressed as follows: , When the cosine similarity is greater than or equal to the preset threshold of 0.9, the context triplet and the similarity score corresponding to the context triplet embedding vector are added to the similarity context triplet set. In; according to the similarity score Sort the context triples in descending order and select the first triples, as the final set of context triples ; Similarly, according to step S31 and step S32, the final context triplet set of the negative sampling triples generated in the head entity mode and the tail entity mode of step S1 is obtained.

[0022] S4. Detailed information retrieval based on large language model: Input the short description of the entity into the large language model to generate the corresponding long description of the entity to enrich the entity information; Specifically, the short description of each entity in the knowledge graph is transformed into Generate long description .

[0023] S5. Sentence generation: Combine the short description of the entity, the relationship description, and the long description of the entity generated in step S4, convert the query triple and its negative sampling triple, the path set of the query triple and its negative sampling triple extracted in step S2, and the context triple of the query triple and its negative sampling triple generated in step S3 into natural language sentences, and obtain the query triple and its negative sampling triple sentences, path sentences, and context sentences, which are used to construct the positive samples and negative sampling samples required for training; Specifically, through the entity short description, the relationship description and the long description generated in step S4, the query triple and its negative sampling triple, the path set of the query triple and its negative sampling triple extracted in step S2, and the context triple of the query triple and its negative sampling triple generated in step S3 are all converted into natural language sentences; the query triple, the path set of the query triple and the context triple of the query triple are converted into natural language sentences, including four scenarios, and the specific methods are as follows: S51. Query triple sentence generation: The sentence starts with a [CLS] token, followed by a short description of the query triple head entity and long description ; Next is the query triple relationship description ; Finally, the short description of the end entity of the query triple and long description ; The head entity and its short and long descriptions, the relationship and its description, the tail entity and its short and long descriptions are separated by the [SEP] tag; similarly, negative sampling triplets are obtained; sentence generation; For example, for the query triple , the short description of “Schleswig-Holstein” is “A state in northern Germany.”, and the long description is “Schleswig-Holsteinis the northernmost state of Germany, bordered by Denmark to the north andknown for its coastline and cultural heritage.”; the description of “capital of” is “Indicates the capital city of a state.”; the short description of the tail entity “Kiel” is “The capital ofSchleswig-Holstein.”, and the long description is “Kiel is a port city in northern Germany, knownfor its maritime culture, universities, and sailing events.”, the generated sentence is “[CLS] Schleswig-Holstein: A state in northern Germany. Schleswig-Holstein isthe northernmost state of Germany, known for its coastline and culture. [SEP]Capital of: Indicates the capital city of a state. [SEP] Kiel: The capital ofSchleswig-Holstein. Kiel is a port city in northern Germany, known for itsmaritime culture, universities, and sailing events.” This format is concise and informative.

[0024] S52. Path sentence generation: In head entity mode from query head entity to query tail entity, a sentence starts with a [CLS] token, followed by a short description of the query triple head entity. and long description ; Then the first relationship and its relationship description in the valid path in the obtained header entity mode ; followed by a short description of the intermediate entity corresponding to the first relationship and long description , repeat for each subsequent relationship and entity until the short description of the tail entity is reached and long description ; The head entity and its short description and long description, the first relationship and relationship description, the first intermediate entity and its short description and long description, the second relationship and relationship description, the second intermediate entity and its short description and long description, ..., the nth relationship and relationship description, the tail entity and its short description and long description, each part is separated by the [SEP]} tag; similarly, the path sentence in the tail entity mode from the tail entity to the head entity is obtained; For example, for the query triple The short description of the head entity "Schleswig-Holstein" is "A state in northern Germany.", and the long description is "Schleswig-Holstein is the northernmost state of Germany, bordered by Denmark to the north and known for its coastline and cultural heritage."; the description of the first relationship "is_part_of" is "Indicates that an entity is part of another entity."; the short description of the intermediate entity "Germany" is "A country in central Europe.", and the long description is "Germany is a federal republic in central Europe, known for its rich history and economic influence."; the description of the second relationship "has_capital" is "Indicates the capital city of a state."; the short description of the tail entity "Kiel" is "The capital of Schleswig-Holstein.", and the long description is "Kiel is a port city in northern Germany, known for its maritime culture, universities, and sailing events." The generated path sentence is "[CLS] Schleswig-Holstein: A federal state in northernGermany. Schleswig-Holstein is the northernmost state of Germany, bordered by Denmark to the north and known for its coastline and cultural heritage. [SEP]is_part_of: Indicates that an entity is part of another entity. [SEP]Germany: A country in central Europe.This generation method ensures the semantic integrity and expression fluency of entities and relationships in the path.

[0025] S53. Context sentence generation: The context information of the query triple extracted in step S3 is represented in the form of triples, which are specifically divided into the following five types, including a single-hop triple consisting of the predecessor node of the query head entity, a single-hop triple consisting of the successor node of the query head entity, a single-hop triple consisting of the predecessor node of the query tail entity, a single-hop triple consisting of the successor node of the query tail entity, and a triple generated by multi-hop expansion that does not contain the query head entity and the tail entity; these context triples comprehensively supplement the semantics and neighborhood relationship information of the query triples, and provide a basis for context sentence generation.

[0026] A single-hop triple consisting of the predecessor nodes of the query head entity The sentence generation format is: the sequence starts with [CLS]}, followed by a short description of the head entity and long description , relationships and relationship descriptions , and a short description of the tail entity and long description , each part is separated by the [SEP]} tag; query single-hop triples Single-hop triplet of the successor node of the middle-tail entity , that is, querying the head entity and query tail entity Unrelated multi-hop triples, whose sentence generation format is: the sequence starts with [CLS]}, followed by a short description of the head entity and long description , relationships and relationship descriptions , and a short description of the successor entity and long description , each part is separated by the [SEP]} mark; More specifically, for the query triple For the single-hop triple following the head entity : The head entity is "Schleswig-Holstein", the short description is "A state in northern Germany, the long description is "Schleswig-Holsteinis the northernmost state of Germany, bordered by Denmark to the north andknown for its coastline and cultural heritage."; the relationship is "is_part_of", the description is Indicates that an entity is part of another entity; the tail entity is "Germany", the short description is "A country in central Europe", the long description is "Germany is a federal republic incentral Europe, known for its rich history and economic influence". The sentence can be generated: Schleswig-Holstein, A state in northern Germany, chleswig-Holstein is thenorthernmost state of Germany, bordered by Denmark to the north and known forits coastline and cultural heritage, is_part_of, Indicates that an entity ispart of another entity, Germany, A country in central Europe, Germany is afederal republic in central Europe, known for its rich history and economicinfluence.

[0027] S54. Context-free sentence generation: When the path and context of the query triple do not exist, the sentence only contains [CLS] and [SEP] tokens; S6. Embedding generation: Input the natural language sentence generated in step S5 into the Sentence Transformer model to generate the query triple and its negative sampling triple sentence embedding vector, the path sentence embedding vector and the context sentence embedding vector; S7. Generate prediction results and train: Obtain the final result of the inductive relationship prediction by calculating the cosine similarity scores between the path sentence embedding vector and the context sentence embedding vector generated in step S6 and the query triple and its negative sampling triple sentence embedding vector; Optimize the model using the loss function.

[0028] Specifically, the cosine similarity is used to measure the similarity between the query triplet and its negative sampling triplet sentence embedding vector obtained in step S6 and the path sentence embedding vector, and the formula is as follows: in, is to embed the query triple and its negative sampling triple sentence into a vector, Represents the path sentence embedding vector; similarly, obtain the similarity between the query triple and its negative sampling triple sentence embedding vector and the context sentence embedding vector; take the highest similarity score as the query triple The final score is expressed as follows: , in, Representing and querying triples The embedding vector of the sentence generated by the path or context triplet, Represents a query triple The path or index of the context triplet, represents the final score of the query triple, L represents the number of all path or context triplets, Indicates that the maximum value is selected from all similarity scores; similarly, the final scores of the negative sampling triplets generated under the head entity mode and the tail entity mode are obtained, and the average of the final scores of the negative sampling triplets generated under the head entity mode and the tail entity mode is taken as the final result of the inductive relationship prediction; the training process is optimized by the cosine embedding loss function, and the formula is expressed as follows: in, Represents the minimum interval between positive and negative samples, Represents the label of the sample, where 1 represents a positive sample and -1 represents a negative sample.

[0029] Example 2 We used the datasets provided by Grail and the versions processed by BERTRL to evaluate the performance of our proposed model in transductive and inductive prediction. Each dataset in Grail (WN18RR, FB15k-237, NELL-995) is divided into four inductive subsets (v1, v2, v3, v4). BERTRL has adjusted these subsets and renamed WN18RRv1, FB15k-237 v1 and NELL-995 v2 to WN18RR_full, FB15k-237_full and NELL-995_full respectively. The statistics of the datasets are shown in Table 1. Among them, #entities, #relations and #triples represent the number of entities, the number of relations and the number of triplets respectively.

[0030] Table 1 Statistics of three datasets and their variants To ensure consistency with previous methods, this study uses the same evaluation metrics as BERTRL: mean reciprocal ranking (MRR) and Hit@1. These metrics are evaluated between each test triple and 50 negative candidate entities. MRR represents the mean of the average reciprocal ranking of positive triples, and Hit@1 represents the proportion of positive triples ranked first among candidate entities. It is worth noting that higher MRR and Hit@1 values ​​indicate that the model performs better in link prediction tasks.

[0031] Our model is based on the sentence transformer implementation in PyTorch, and the default model is sentence-transformers / all-mpnet-base-v2 from the Hugging Face repository. Training was performed on a 24GB NVIDIA GeForceRTX 3090 GPU, and the random seed was set to 42 to ensure the reproducibility of the results. In the training and validation phases, each positive triplet was paired with 5 negative samples; in the test phase, each positive triplet was paired with 50 negative samples. We set the number of paths loaded for each triplet to 3, the search depth to 5, and the support threshold to 5×10-3. In addition, the number of contexts loaded for each triplet was set to 3.

[0032] We compare our model with the following baseline models: MINERVA, RuleN, GraIL, KG-BERT, BERTRL, KRST, and APST. Specifically, MINERVA uses reinforcement learning for path reasoning; RuleN is a refined evaluation method based on rules and embeddings; GraIL uses graph neural networks to reason about local subgraph structures; KG-BERT is the first model to predict missing triples using a pre-trained language model; BERTRL incorporates reasoning paths and triples as input into a pre-trained language model; KRST uses a sentence transformer for interpretable inductive relation prediction; and APST further improves performance by anchoring paths to reduce the reliance on closed paths based on KRST.

[0033] We compare the MRR and Hit@1 results of the randomly sampled CPNR_R and CPNR_S models of the proposed method with the baseline models on the WN18RR_full, FB15K-237_full, and NELL-995_full datasets, and evaluate them in direct (see Table 2) and inductive (see Table 3) settings. The specific performance is as follows: (1) CPNR_S performs well on all datasets, while CPNR_R leads in 2 / 3 datasets. Compared with APST, in the direct setting, CPNR_S improves MRR and Hit@1 by 9.31% and 16.21% on WN18RR_full, 28.68% and 43.08% on FB15K-237_full, and 18.60% and 31.95% on NELL-995_full. In the inductive setting, CPNR_S improves MRR and Hit@1 by 7.16% and 18.40% on WN18RR_full, 30.10% and 54.43% on FB15K-237_full, and 20.29% and 34.09% on NELL-995_full. In comparison, CPNR_R improves MRR and Hit@1 by 0.55% and 0.60% on WN18RR_full, and by 0.25% on NELL-995_full. Under the inductive setting, CPNR_R improves MRR by 1.32% and 2.63% on WN18RR_full, and by 2.73% and 2.71% on NELL-995_full.

[0034] (2) APST performs poorly on the FB15K-237_full dataset in both transfer and induction settings, highlighting the advantages of the anchor path strategy in capturing the complex relationships of FB15K-237 and significantly improving prediction accuracy. Although CPNR_R is slightly inferior to APST on some datasets, it has obvious advantages over KRST. For example, in the FB15K-237 dataset in direct push settings, CPNR_R's MRR exceeds KRST by 6.67% and Hit@1 exceeds 5.95%. In the induction setting, CPNR_R's MRR exceeds KRST by 3.91% and Hit@1 exceeds 2.83%.

[0035] (3) The semantic negative sampling method enhances model training by generating more challenging negative samples, which has significant advantages over random sampling. Semantic negative sampling significantly improves CPNR_S compared to CPNR_R in MRR and Hit@1. In the direct setting, WN18RR_full improves by 8.71% and 15.52%, FB15K-237_full improves by 29.69% and 46.68%, and NELL-995_full improves by 18.31% and 31.95%. In the inductive setting, WN18RR_full improves by 5.76% and 15.37%, FB15K-237_full improves by 33.60% and 60.94%, and NELL-995_full improves by 17.09% and 30.54%. These results highlight the significant advantage of semantic negative sampling in improving model performance.

[0036] The link prediction results of the method of the present invention (CPNR_S) and the prior art on the WN18RR_full, FB15k-237_full and NELL-995_full direct push datasets are shown in Table 2. The highest score is marked in bold, and the second highest score is marked in underline.

[0037] Table 2 Link prediction results on the direct push dataset The link prediction results of the method of the present invention (CPNR_S) and the prior art on the WN18RR_full, FB15k-237_full and NELL-995_full summary data sets are shown in Table 3. Among them, the highest score is marked in bold, and the second highest score is marked in underline.

[0038] Table 3 Link prediction results on the inductive dataset We show the MRR and Hit@1 ablation results of the proposed CPNR_R and CPNR_S models on the WN18RR_full, FB15k-237_full and NELL-995_full datasets in the inductive setting. This study aims to explore the specific impact of CPNR_R and CPNR_S by isolating different components. These components include: 1) detailed description retrieval (w / o retrieval) of large language models (LLMs); 2) path extraction and filtering (w / o path); 3) context sampling and filtering (w / o context). As shown in Tables 4 and 5, CPNR_R and CPNR_S outperform their variants in most of the performance of MRR and Hit@1, as shown in the following: (1) Overall, CPNR_R outperforms other variants in 1 / 3 of the cases, while CPNR_S performs well in 4 / 4 of the cases, showing its significant superiority and robustness in the inductive setting. It is worth noting that on some datasets, removing the retrieval module and the context module improves the model performance, while removing the path module leads to a consistent decline in performance, further verifying the key role of the path in model performance.

[0039] (2) w / o retrieval: This module enriches contextual information, enhances semantic representation, reduces ambiguity, and supports complex relational reasoning by supplementing short descriptions with detailed long descriptions in KGs. After removing the retrieval module, the average performance of CPNR_S decreases, which further verifies the effectiveness of this module. However, after removing this module, CPNR_R performs best in the inductive setting of WN18RR_full. This may be because the WN18RR dataset has a simple structure and clear relationships, and the introduction of long descriptions may introduce noise, thereby reducing model performance. In addition, after removing this module, the performance of CPNR_S decreases significantly, which further confirms the importance of the retrieval module.

[0040] (3) w / o path: The path module enriches the semantic information, enhances the reasoning ability of the model, and significantly improves the prediction accuracy by showing how two entities are connected through multiple intermediate relations and entities. After removing the path module, the performance of CPNR_R and CPNR_S both dropped significantly, further verifying the core role of the path in the model.

[0041] (4) w / o context: The context module focuses on the local associations of the query entity and its neighborhood, which helps capture core features and reduce noise. After removing the context module, CPNR_R achieves the highest MRR and Hit@1 in the FB15K-237_full inductive setting, indicating that inappropriate context may weaken performance. At the same time, the performance of CPNR_S drops significantly, further demonstrating the importance of context information to model performance.

[0042] Table 4 Ablation experiment results of CPNR_R model.

[0043] Table 5 Ablation experiment results of CPNR_R model.

[0044] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An efficient negative sampling and retrieval fusion context path inductive relationship prediction method, characterized in that: The following steps are involved: S1. Semantic-aware negative sampling: Negative sampling is performed through semantic awareness to generate negative sampling triplets that are the least similar to the query triplet; S2. Path extraction and filtering: Use the breadth-first search algorithm to extract the path information of the query triple and the negative sampling triple generated in step S1, and filter the paths according to the relationship confidence to obtain the path set of the query triple and its negative sampling triple; S3. Context extraction and filtering: Through a recursive multi-hop context search algorithm, the context triples of the query triple and the negative sampling triple generated in step S1 are explored respectively, and filtered according to cosine similarity to obtain the context triples of the query triple and its negative sampling triple; S4. Detailed information retrieval based on the large language model: input the short description of the entity into the large language model to generate the corresponding long description of the entity; S5. Sentence generation: Combining the short description of the entity, the relationship description, and the long description of the entity generated in step S4, the query triple and its negative sampling triple, the path set of the query triple and its negative sampling triple extracted in step S2, and the context triple of the query triple and its negative sampling triple generated in step S3 are all converted into natural language sentences to obtain the query triple and its negative sampling triple sentence, the path sentence, and the context sentence; S6. Embedding generation: Input the natural language sentence generated in step S5 into the Sentence Transformer model to generate the query triple and its negative sampling triple sentence embedding vector, the path sentence embedding vector and the context sentence embedding vector; S7. Generate prediction results and train: Obtain the final result of the inductive relationship prediction by calculating the cosine similarity scores between the path sentence embedding vector and the context sentence embedding vector generated in step S6 and the query triple and its negative sampling triple sentence embedding vector; Optimize the model using the loss function.

2. The method for predicting inductive relations by integrating context paths with efficient negative sampling and retrieval according to claim 1, characterized in that: Step S1 specifically includes: S11. Each entity in the knowledge graph The corresponding short description Input BERT model to generate entity embedding ; S12. In head entity mode, from the entity set of the knowledge graph Select the tail entity in the query of Negative sampling tail entity , and based on the similarity score Sort in ascending order, and query the tail entity The most dissimilar entities are sorted first, from the lowest similarity to the lowest End entity Generate negative sampling triples in head entity mode ; Similarly, in the tail entity mode, generate the negative sampling triplet set in the tail entity mode ; The formula is as follows: ， ， in, Represents the set of negative sampling triplets in head entity mode, The negatively sampled tail entity representing the set of negatively sampled triplets in head entity mode, It means taking the first k values. Represents a collection of entities, Represents the query tail entity Entity embedding, express Negative sampling tail entity Entity embedding, Indicates the number of entities. represents the set of negative sampling triplets in tail entity mode, The negative sampling head entity representing the set of negative sampling triplets in tail entity mode, Represents a query header entity Entity embedding, express Negative sampling head entity Entity embedding.

3. The method for predicting inductive relations by integrating context paths with efficient negative sampling and retrieval according to claim 2 is characterized in that: Step S2 specifically includes: Use the breadth-first search algorithm to extract the path information of the query triple and the negative sampling triple generated in step S1, and filter the path according to the relationship confidence. The path extraction and filtering process is as follows: S21. Path extraction: In head entity mode, alternately extracting from head entities through relations and entities Tracking to the end entity step by step , search for query triples in the knowledge graph using a breadth-first search algorithm De novo entities To the end entity Path ,path Expressed as , from the path Extracting De Novo Entities To the end entity The relationship path , relationship path Expressed as ; Similarly, in the tail entity mode, get the tail entity To the end entity The relationship path ; S22. Path filtering: Use the relationship path confidence function to calculate the confidence of the relationship path, the relationship path confidence in the head entity mode The formula is as follows: ， in, Represents a relationship path The number of triplets of Represents a relationship path The number of triplets in the head entity To the end entity The relationship path The relationship paths whose relationship path confidence in the head entity mode is greater than or equal to the preset relationship path confidence threshold are screened out to obtain the valid paths in the head entity mode; the preset relationship path confidence threshold is set to 5×10 -3 ; Similarly, obtain the valid path in the tail entity mode; Similarly, according to the query triple In the path extraction and filtering process, the negative sampling triplet generated in step S1 is subjected to path extraction and filtering to obtain the corresponding valid path.

4. The method for predicting inductive relations by integrating efficient negative sampling and retrieval context paths according to claim 3 is characterized in that: Step S3 specifically includes: Through the recursive multi-hop context search algorithm, the context triples of the query triples and the negative sampling triples generated in step s1 are obtained and filtered according to the cosine similarity; the context triple search process of the query triples is as follows: S31. Context extraction: recursively apply a multi-hop context search algorithm to explore query triples The neighborhood context of the head entity and the tail entity. The specific steps are as follows: According to the query triple , initialize the head entity node collection and tail entity node set ; Head entity node collection To include the header entity A collection of , tail entity node set To include the tail entity A collection of ; Create an empty header entity context triple set at the same time and an empty set of tail entity context triples , used to store the neighborhood context triples of the head entity and the tail entity; In the single-hop context triple construction, starting from the query head entity and the query tail entity, the nodes and relationships directly connected to them are extracted respectively; for the head entity node set , extract the predecessor node connected to it from the knowledge graph and relationship , forming a triple ; At the same time, extract the successor node connected to it and relationship , generating triples ; Add these triples to the head entity context triple set , and extract the predecessor node and successor nodes Add to the new head entity node collection Similarly, for the tail entity node set , extract the predecessor node and successor node of each node and their relationship, generate triples and add them to the tail entity context triple set , and update the new tail entity node set at the same time ;Through this process, the single-hop stage completes the expansion of the direct neighborhood relationship between the query head entity and the tail entity, forming the initial set of context triples; In the process of constructing multi-hop context triples, the head entity node set generated in the single-hop stage is and tail entity node set , assign it to the current head entity node set and tail entity node set , and and Set to an empty set, in In this hop, for the current head entity node set and tail entity node set , extract the predecessor node and successor node and relationship, generate the corresponding triples and add them to the head entity context triple set respectively and tail entity context triples At the same time, the new node is added to the updated head entity node set and tail entity node set ; By recursively expanding the neighborhood relationship through multiple jumps, a multi-hop context triple set containing deep semantic information is finally constructed and ; The final header entity context collection The context triples of the and tail entities Merge into a new set of context triples ; If the query triple Already exists in , then remove it; S32. Context filtering: Use the pre-trained BERT model to filter new context triples. Each context triple in and query triples Encode the entities and relations in the , generate the embedding vectors for each entity and relation, and store them in the dictionary. The formula is as follows: ， ， in, , , Represents context triples The embedding vectors of the head entity, relation, and tail entity in , , , Represents a query triple The embedding vectors of the head entity, relation, and tail entity in the calculation context triple and query triples The embedding vector of is expressed as follows: ， in, represents the context triple embedding vector, Represents the query triple embedding vector; calculates the context triple embedding vector and query triplet embedding vector The cosine similarity of is expressed as follows: ， When the cosine similarity is greater than or equal to the preset threshold of 0.9, the context triplet and the similarity score corresponding to the context triplet embedding vector are added to the similarity context triplet set. In; according to the similarity score Sort the context triples in descending order and select the first triples, as the final set of context triples ; Similarly, according to step S31 and step S32, the final context triplet set of the negative sampling triples generated in the head entity mode and the tail entity mode of step S1 is obtained.

5. The method for predicting inductive relations by integrating efficient negative sampling and retrieval context paths according to claim 4, characterized in that: In step S4, the short description of each entity in the knowledge graph is transformed into Generate long description .

6. The method for predicting inductive relations by integrating context paths with efficient negative sampling and retrieval according to claim 5, characterized in that: Step S5 specifically includes: Through the entity short description, relationship description and the long description generated in step S4, the query triple and its negative sampling triple, the path set of the query triple and its negative sampling triple extracted in step S2, and the context triple of the query triple and its negative sampling triple generated in step S3 are all converted into natural language sentences; the query triple, the path set of the query triple and the context triple of the query triple are converted into natural language sentences, including four scenarios, and the specific methods are as follows: S51. Query triple sentence generation: The sentence starts with a [CLS] token, followed by a short description of the query triple head entity and long description ; Next is the query triple relationship description ; Finally, the short description of the end entity of the query triple and long description ; The head entity and its short and long descriptions, the relationship and its description, the tail entity and its short and long descriptions are separated by the [SEP] tag; similarly, negative sampling triplets are obtained; sentence generation; S52. Path sentence generation: In head entity mode from query head entity to query tail entity, a sentence starts with a [CLS] token, followed by a short description of the query triple head entity. and long description ; Then the first relationship and its relationship description in the valid path in the obtained header entity mode ; followed by a short description of the intermediate entity corresponding to the first relationship and long description , repeat for each subsequent relationship and entity until the short description of the tail entity is reached and long description ; The head entity and its short description and long description, the first relationship and relationship description, the first intermediate entity and its short description and long description, the second relationship and relationship description, the second intermediate entity and its short description and long description, ..., the nth relationship and relationship description, the tail entity and its short description and long description, each part is separated by the [SEP]} tag; similarly, the path sentence in the tail entity mode from the tail entity to the head entity is obtained; S53. Context sentence generation: The context information of the query triple extracted in step S3 is represented in the form of triples, which are specifically divided into the following five types: a single-hop triple consisting of the predecessor node of the query head entity, a single-hop triple consisting of the successor node of the query head entity, a single-hop triple consisting of the predecessor node of the query tail entity, a single-hop triple consisting of the successor node of the query tail entity, and a triple generated by multi-hop expansion that does not contain the query head entity and the tail entity; a single-hop triple consisting of the predecessor node of the query head entity The sentence generation format is: the sequence starts with [CLS]}, followed by a short description of the head entity and long description , relationships and relationship descriptions , and a short description of the tail entity and long description , each part is separated by the [SEP]} tag; similarly, the sentence generation of the single-hop triple consisting of the successor node of the query head entity, the single-hop triple consisting of the predecessor node of the query tail entity, and the single-hop triple consisting of the successor node of the query tail entity is obtained; query single-hop triples Single-hop triplet of the successor node of the middle-tail entity , that is, querying the head entity and query tail entity Unrelated multi-hop triples, whose sentence generation format is: the sequence starts with [CLS]}, followed by a short description of the head entity and long description , relationships and relationship descriptions , and a short description of the successor entity and long description , each part is separated by the [SEP]} mark; Similarly, we obtain the context sentence generation of the negative sampling triples generated in the head entity mode and the tail entity mode; S54. Context-free sentence generation: When the path and context of the query triple do not exist, the sentence only contains [CLS] and [SEP] tokens.

7. The method for predicting inductive relations by integrating efficient negative sampling and retrieval context paths according to claim 6, characterized in that: Step S7 specifically includes: The cosine similarity is used to measure the similarity between the query triplet obtained in step S6 and its negative sampling triplet sentence embedding vector and the path sentence embedding vector. The formula is as follows: in, is to embed the query triple and its negative sampling triple sentence into a vector, Represents the path sentence embedding vector; similarly, obtain the similarity between the query triplet and its negative sampling triplet sentence embedding vector and the context sentence embedding vector; take the highest similarity score as the query triplet The final score is expressed as follows: ， in, Representing and querying triples The embedding vector of the sentence generated by the path or context triplet, Represents a query triple The path or index of the context triple, represents the final score of the query triple, L represents the number of all path or context triplets, Indicates that the maximum value is selected from all similarity scores; similarly, the final scores of the negative sampling triplets generated under the head entity mode and the tail entity mode are obtained, and the average of the final scores of the negative sampling triplets generated under the head entity mode and the tail entity mode is taken as the final result of the inductive relationship prediction; the training process is carried out through the cosine embedding loss function For optimization, the formula is as follows: in, Represents the minimum interval between positive and negative samples, Represents the label of the sample, where 1 represents a positive sample and -1 represents a negative sample.

Images