Knowledge graph completion method and system based on large language model and comparative learning

Abstract

The invention provides a knowledge graph completion method and system based on a large language model and comparative learning, and belongs to the field of knowledge graph completion, and the method comprises the steps: constructing a prompt template for a to-be-processed knowledge graph, and generating a dynamic entity semantic description based on the prompt template through employing the large language model; encoding the entity semantic description generated by the large language model by using a BERT model to obtain an entity initial representation generated by a semantic context; and updating the initial entity representation for the initial entity representation aggregated neighbor information by adopting an entity representation prediction model to obtain a target entity representation complemented knowledge graph. According to the method, limitation of static corpora is overcome through a large language model, entity representation rich in semantics is generated, supervised contrast learning is introduced to enhance training signals, the problem of insufficient positive samples is relieved, the representation precision and complementation performance of low-frequency entities in the sparse graph are improved, and finally the efficiency and accuracy of knowledge graph complementation are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph completion, and in particular relates to a knowledge graph completion method and system based on large language models and contrastive learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Knowledge graphs are frameworks that organize knowledge in a graph structure, with triples (head entity-relation-tail entity) as their basic unit. They can efficiently store and represent complex relationships in the real world. The construction and maintenance of knowledge graphs face many challenges, primarily due to data incompleteness. Because of the diversity of knowledge sources and the cost limitations of manual annotation, existing knowledge graphs commonly suffer from missing triples, directly impacting their performance in downstream tasks. To address this issue, Knowledge Graph Completion (KGC) has become a key research focus. Its goal is to infer missing entities and relations and generate new, valid triples from existing graph structures and semantic information.

[0004] The evolution of knowledge graph completion technology from rule-based reasoning to machine learning currently falls into three main categories. The first category is rule-based reasoning, which mines implicit relationships through logical rules. While highly interpretable, its rule design relies on expert knowledge, making it difficult to adapt to large-scale and diverse graphs. The second category is embedding representation-based methods, which map entities and relationships to a vector space and predict missing triples using geometric or algebraic operations. These methods are computationally efficient but have limited ability to model complex relationships. The third category is deep learning-based methods, which utilize the non-linear representation capabilities of neural networks, combined with graph structures and external corpora, to generate more semantically deep embedding representations, significantly improving completion performance. However, these methods still suffer from insufficient representation and prediction bias when dealing with highly sparse graphs or low-frequency entities.

[0005] Currently, traditional knowledge graph completion methods based on graph neural networks mainly rely on the frequent occurrence of entities and relations in knowledge graph triples to generate high-quality embedding representations. However, in practical applications, knowledge graphs often exhibit significant sparsity, and low-frequency entities struggle to obtain accurate embedding representations due to a lack of neighbor information, leading to a significant decline in relation prediction performance. For example, in datasets containing a large number of long-tail entities, traditional models lack sufficient training signals, making it difficult to effectively capture the semantic features of these entities, thus affecting the accuracy of the completion task. Furthermore, while existing entity description-based completion methods attempt to compensate for the lack of triple information by introducing external text resources, they are limited by the quality and completeness of the descriptive information. On the one hand, the coverage of internet corpora is limited, and descriptive information for many niche or obscure entities is missing or too brief, making it difficult to provide comprehensive semantic support; on the other hand, the captured descriptive information often contains noise, redundancy, or even contradictory content, directly affecting the accuracy of the embedding representation. This dual deficiency makes traditional methods prone to significant bias when processing sparse graphs or predicting rare relations, limiting the applicability of knowledge graph completion technology in practical applications.

[0006] In addition, existing completion methods based on graph convolutional networks, graph convolutional models, or Transformer structures utilize neighbor aggregation and semantic enhancement techniques, but their performance is limited by noise interference in highly sparse graphs and the instability of external corpus quality, resulting in certain accuracy bottlenecks in complex relationship prediction. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention proposes a knowledge graph completion method and system based on a large language model and contrastive learning. On one hand, it employs SemGCL-KGC to dynamically generate multi-perspective entity description information using a semantic context generator and a large language model. This information is then encoded into semantically rich vector representations by a pre-trained language model, improving the accuracy and completeness of low-frequency entity representations in sparse graphs and overcoming the limitations of traditional static corpora, thus generating semantically rich entity representations. On the other hand, its knowledge contrastive learning optimizer uses supervised contrastive learning to enhance training signals through contrastive loss, bringing similar entities closer together and distancing different entity embeddings, effectively alleviating the problem of insufficient positive samples caused by sparsity and improving entity discrimination and completion performance.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, this invention discloses a knowledge graph completion method based on a large language model and contrastive learning, comprising: The knowledge graph to be processed is constructed with a prompt template, and a dynamic entity semantic description is generated based on the prompt template using a large language model; The BERT model is used to encode the semantic description of entities generated by the large language model, so as to obtain the initial representation of entities generated by the semantic context; An entity representation prediction model is used to aggregate neighbor information to update the initial entity representation, thereby obtaining a target entity representation to complete the knowledge graph. The entity representation prediction model is a graph convolutional neural network with an attention mechanism, and the graph convolutional neural network uses contrastive learning to optimize the embedding distribution of the target entity representation.

[0009] Secondly, this invention discloses a knowledge graph completion system based on a large language model and contrastive learning, comprising: The semantic description module is configured to: construct a prompt template for the knowledge graph to be processed, and generate dynamic entity semantic descriptions based on the prompt template using a large language model; The semantic encoding module is configured to: use the BERT model to encode the semantic description of entities generated by the large language model to obtain the initial representation of entities generated by the semantic context; The graph completion module is configured to: use an entity representation prediction model to aggregate neighbor information to update the initial entity representation, thereby obtaining a target entity representation to complete the knowledge graph; wherein, the entity representation prediction model is a graph convolutional neural network with an attention mechanism, and the graph convolutional neural network uses contrastive learning to optimize the embedding distribution of the target entity representation.

[0010] Thirdly, the present invention discloses an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when run by the processor, complete the steps of the knowledge graph completion method based on large language models and contrastive learning described above.

[0011] Fourthly, the present invention discloses a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the knowledge graph completion method based on a large language model and contrastive learning described above.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention utilizes a large language model to dynamically generate multi-perspective, task-related entity descriptions, and encodes them into semantically rich vector representations using a pre-trained language model BERT. This effectively compensates for the deficiencies of insufficient coverage and noise interference in static corpora, and improves the representation quality of low-frequency entities and the semantic consistency of triples.

[0013] This invention combines graph-structured aggregation networks with graph neural networks and attention mechanisms to dynamically integrate neighbor information, thereby enhancing the embedding and representation capabilities of low-frequency entities in sparse graphs.

[0014] During the training phase, this invention employs a knowledge contrastive learning optimizer that uses supervised contrastive learning and ConvR knowledge projection to bring the embedded representations of the same entities closer together and push the embedding representations of different entities further apart, thereby alleviating the problem of insufficient positive samples and significantly improving entity discrimination and completion performance.

[0015] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 This is a schematic diagram of the knowledge graph completion method based on large language models and contrastive learning described in Embodiment 1 of the present invention. Detailed Implementation

[0018] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0020] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0021] Terminology Explanation: Representation: Representing entities or relationships using vectors or other data structures so that computers can understand and process them. Embedding refers to the process of mapping objects (such as words, text, entities, etc.) from a high-dimensional space to a low-dimensional dense vector space. This representation method can help machine learning models better understand and process information in knowledge graphs.

[0022] Neighbors: In a knowledge graph, for a given entity, other entities that are directly connected to that entity through relationships.

[0023] Example 1 In one or more embodiments, a knowledge graph completion method based on a large language model and contrastive learning is disclosed, such as Figure 1 As shown, it includes the following steps: Step S1: Construct a prompt template for the knowledge graph to be processed, and generate dynamic entity semantic descriptions based on the prompt template using a large language model; Step S1-1: Construct a prompt template , including head entity ,relation Tail-end entity The requirement for multi-perspective descriptions. For example, a prompt template: Entity Background: "Describes basic information about the <header entity>, including its definition, category, and main attributes." Relational semantics: "Explaining the meaning of <relation> and its role in <domain>." Triple connections: "Explain the association between the <head entity> and the <tail entity> through the <relation>, providing specific background and examples." Domain-specific: "In a domain, what are the typical application scenarios for head entities and tail entities through relationships?" Step S1-2: Input the prompts into the Large Language Model (LLM) to generate a natural language description, i.e., an entity semantic description, that includes entity context, relational semantics, and triple associations. Use the Large Language Model (LLM) to output the entity context. (1) Where, It is a natural language description that includes entity context, relational semantics, and triple relationships. For large language models.

[0024] Based on the characteristics of entities or relationships (such as entity frequency, relationship type, and domain context), the prompt template is dynamically selected or adjusted. For low-frequency entities, the level of detail in the background description is increased. For complex relationships, more contextual information is added. For specific domains (such as healthcare), domain terminology or external knowledge bases are introduced, based on entity frequency. (Number of occurrences in the graph), adjust the level of detail in the prompts: (2) in, It is the frequency threshold. This is a detailed prompt. This is a simplified prompt.

[0025] This embodiment uses the Big Prophet model to generate dynamic, task-related entity semantic descriptions, making up for the deficiencies of insufficient coverage and incomplete information in static corpora.

[0026] Step S2: Use the BERT model to encode the semantic description of the entities generated by the large language model to obtain the initial representation of the entities generated by the semantic context; Describing the semantics of entities The input is encoded into a pre-trained language model, and semantic vectors are extracted to obtain the initial representation of the entity. (3) Where, It is the initial entity representation generated through semantic context, and [CLS] is the tag vector. This is obtained by applying the [CLS] tag vector to the input text. This vector not only captures the semantic context of the entity, but also enhances the semantic consistency of the triple through contextual information about relations and tail entities.

[0027] Using entity description text as input, the pre-trained language model BERT is used for encoding to extract and generate semantic information representation vectors of entities to support subsequent triple prediction.

[0028] Step S3: Use the entity representation prediction model to aggregate neighbor information to update the initial entity representation, and obtain the target entity representation to complete the knowledge graph; The entity representation prediction model is a graph convolutional neural network with an attention mechanism, and the graph convolutional neural network uses contrastive learning to optimize the embedding distribution of the target entity representation.

[0029] In order to make full use of the structural information of the knowledge graph, this embodiment uses a graph neural network to aggregate neighbor information, and enhances the representation of low-frequency entities by aggregating neighbor information through a graph neural network and attention mechanism.

[0030] Specifically, graph convolutional neural networks containing attention mechanisms include computation of target entities. and its neighboring entities and connection relationship The relative importance of each node is used to obtain an attention score. The attention mechanism extracts the weights of different nodes when aggregating neighbor nodes in a graph neural network. The attention score is normalized to generate attention weights. Based on the attention weights, messages are passed from neighbor entities to the target entity to obtain the aggregated message. The target entity representation is updated based on the aggregated message, thereby achieving knowledge graph completion.

[0031] Specifically, for the target entity and its neighboring entities and connection relationship Calculate attention scores to measure triples The relative importance of attention scores. Defined as: (4) Where, and for The learnable parameters of the layer For the first Hidden representations of head entities, relations, and tail entities in a layer. This is the activation function layer.

[0032] Then, use Normalize the attention score for each triple: (5) Where, For attention weights, For attention scores of specific head and tail entities, the matrix Indicates and Adjacent sets of entities It represents the set of relationships between connected entities and their neighboring entities.

[0033] Based on attention weights, messages are passed from neighboring entities to the target entity, and the message aggregation formula is as follows: (6) Where, This indicates that the data is passed from the neighboring entity to the target entity. The news Represents the feedforward aggregation matrix. Represents context triples Attention weights This represents the activation function. This represents the head and tail entities. Using message aggregation functions to aggregate neighbor nodes can fully capture the network structure of the knowledge graph.

[0034] Then, update the target entity representation: (7) Where, For the target entity representation, This represents the activation function. For messages gathered from neighbors, It represents the entity of the higher level.

[0035] In this embodiment, supervised contrastive learning is used to optimize the embedding distribution and enhance entity discrimination during the training phase of the entity representation prediction model.

[0036] Contrastive learning is used to bring the representations of the same entities closer together and push the representations of different entities further apart. Negative sample feedback is introduced, and knowledge projection is combined to further optimize the semantic consistency of triples.

[0037] First, design the contrast loss, calculated as follows: (8) Where, Indicates a specific entity The associated set of embedded vectors This represents a batch of standardized entity embedding vectors. For the knowledge projection module, The degree hyperparameter is used to control the smoothness of the softmax function. It is a tail entity. As an adjustable parameter, it controls the interval between positive and negative samples, thereby optimizing the discriminative learning performance. By designing the contrastive loss function, the model effectively increases the participation of negative samples, thus improving the utilization efficiency of sparse positive triple information and enhancing the discriminative ability of entity representation. Furthermore, the knowledge projection module is implemented using ConvR technology.

[0038] The ConvR module uses head entities... Tail entity and relationships Convolution processing is performed, and the convolution kernel is dynamically selected according to different relation types to extract features that are highly related to the relation, thereby further improving the semantic accuracy and consistency of the triple representation: (9) In the formula, Each Transform a vector into a vector of dimension 1 and Matrix; The operation converts the result into a vector; It is a weight matrix that projects vectors onto the entity feature space. Finally, it is combined with the embedding vector of the tail entity. The fraction is calculated by performing an inner product.

[0039] Secondly, the formula for the cross-entropy loss function is: (10) In the formula, This represents the training triple set. For the size of the set, when Represents a positive triple, when At that time, it was a negative triplet. It is a tail entity.

[0040] Finally, by summing the comparison loss and the cross-entropy loss, the final objective function is obtained: (11) By jointly optimizing these two objectives, we can capture the similarity between embeddings of the same entity and distinguish them from other entities, thereby improving the performance of link prediction.

[0041] The knowledge contrastive learning optimizer aims to optimize the distribution of entity embeddings through supervised contrastive learning, thereby enhancing the discriminative power of entities. Especially in sparse graphs where positive samples are insufficient, it can bring similar entity representations closer together and push different entity representations further apart, thus improving the quality of entity embeddings.

[0042] Example 2 In one or more embodiments, a knowledge graph completion system based on a large language model and contrastive learning is disclosed, specifically including: The semantic description module is configured to: construct a prompt template for the knowledge graph to be processed, and generate dynamic entity semantic descriptions based on the prompt template using a large language model; The semantic encoding module is configured to: use the BERT model to encode the semantic description of entities generated by the large language model to obtain the initial representation of entities generated by the semantic context; The graph completion module is configured to: use an entity representation prediction model to aggregate neighbor information to update the initial entity representation, thereby obtaining a target entity representation to complete the knowledge graph; wherein, the entity representation prediction model is a graph convolutional neural network with an attention mechanism, and the graph convolutional neural network uses contrastive learning to optimize the embedding distribution of the target entity representation.

[0043] Example 3 This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps of the knowledge graph completion method based on large language models and contrastive learning described above.

[0044] Example 4 This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the knowledge graph completion method based on a large language model and contrastive learning described above.

[0045] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0048] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0049] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A knowledge graph completion method based on large language models and contrastive learning, characterized in that, include: The knowledge graph to be processed is constructed with a prompt template, and a dynamic entity semantic description is generated based on the prompt template using a large language model; The BERT model is used to encode the semantic description of entities generated by the large language model, so as to obtain the initial representation of entities generated by the semantic context; An entity representation prediction model is used to aggregate neighbor information to update the initial entity representation, thereby obtaining a target entity representation to complete the knowledge graph. The entity representation prediction model is a graph convolutional neural network with an attention mechanism, and the graph convolutional neural network uses contrastive learning to optimize the embedding distribution of the target entity representation.

2. The knowledge graph completion method based on large language models and contrastive learning as described in claim 1, characterized in that, The prompt template is a multi-perspective description requirement that includes head entities, relationships, and tail entities.

3. The knowledge graph completion method based on large language models and contrastive learning as described in claim 1, characterized in that, The specific steps of generating dynamic entity semantic descriptions based on the prompt template using a large language model are as follows: In the formula, For entity semantic description, For the head entity, For the relationship, For tail entities, This is a prompt template. For large language models.

4. The knowledge graph completion method based on large language models and contrastive learning as described in claim 1, characterized in that, The entity is initially represented as follows: In the formula, This is the initial representation of the entity, where [CLS] is the tag vector. This is a semantic description of the entity.

5. The knowledge graph completion method based on large language models and contrastive learning as described in claim 1, characterized in that, The graph convolutional neural network containing the attention mechanism calculates the relative importance of the target entity, its neighboring entities, and the connection relationships to obtain an attention score. The attention score is normalized to generate attention weights. Based on the attention weights, messages are passed from neighboring entities to the target entity to obtain the aggregated message. The target entity representation is updated based on the aggregated message.

6. The knowledge graph completion method based on large language models and contrastive learning as described in claim 5, characterized in that, The relative importance of the target entity, its neighboring entities, and connection relationships is calculated to obtain an attention score: In the formula, and for The learnable parameters of the layer For the first Hidden representations of head entities, relations, and tail entities in a layer. For activation function layers; The target entity for updating the aggregated message is represented as follows: Based on attention weights, messages are passed from neighboring entities to the target entity, and the message aggregation formula is as follows: In the formula, This indicates that the data is passed from the neighboring entity to the target entity. The news, Represents the feedforward aggregation matrix. Represents context triples Attention weights This represents the activation function. This represents the head entity and the tail entity.

7. The knowledge graph completion method based on large language models and contrastive learning as described in claim 1, characterized in that, The entity representation prediction model uses contrastive learning to optimize the embedding distribution during the training phase, specifically using a hybrid loss function that sums contrastive loss and cross-entropy loss. The contrastive loss function is: In the formula, Indicates a specific entity The associated set of embedded vectors This represents a batch of standardized entity embedding vectors. For the knowledge projection module, For degree hyperparameters, It is a tail entity; The cross-entropy loss function is: Where, This represents the training triple set. For the size of the set, when Represents a positive triple, when At that time, it was a negative triplet. It is a tail entity.

8. A knowledge graph completion system based on large language models and contrastive learning, characterized in that: include: The semantic description module is configured to: construct a prompt template for the knowledge graph to be processed, and generate dynamic entity semantic descriptions based on the prompt template using a large language model; The semantic encoding module is configured to: use the BERT model to encode the semantic description of entities generated by the large language model to obtain the initial representation of entities generated by the semantic context; The graph completion module is configured to: use an entity representation prediction model to aggregate neighbor information to update the initial entity representation, thereby obtaining a target entity representation to complete the knowledge graph; wherein, the entity representation prediction model is a graph convolutional neural network with an attention mechanism, and the graph convolutional neural network uses contrastive learning to optimize the embedding distribution of the target entity representation.

9. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the knowledge graph completion method based on a large language model and contrastive learning as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the knowledge graph completion method based on a large language model and contrastive learning as described in any one of claims 1-7.

Images