Knowledge graph completion method and system based on decoupling structure embedding

Abstract

The invention discloses a knowledge graph completion method based on decoupling structure embedding, and the method comprises the steps: inputting the structure information of a knowledge graph into a pre-training language model in a prompt manner through the capability of the pre-training language model and a prompt engineering method, and fusing a text score and a result score during the output, the strong text processing capability of the pre-training language model is prevented from covering the structure information provided in the prompt, and the quality of the knowledge graph in the search engine is improved; meanwhile, a relation perception attention mechanism is added in a decoupling graph learning device, decoupling is carried out according to different meanings represented by entities in different triads, the structure information of the knowledge graph can be transmitted to the text information more accurately, the text information can be utilized by the structure information more selectively, and the knowledge graph learning efficiency is improved. The balance during fusion of the structural information and the text information is achieved, and the accuracy of the knowledge graph in the search engine is improved.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph technology, and specifically relates to a knowledge graph completion method and system based on decoupled structure embedding. Background Art

[0002] In today's big data era, the search engine field is facing unprecedented challenges and opportunities. With the surge in data volume, traditional search engines can no longer meet the needs of efficient and accurate retrieval, and a large number of search engines have begun to use knowledge graphs as knowledge bases. The knowledge graph completion method based on decoupled structure embedding and pre-trained language model is an innovative technology that emerged in this context.

[0003] As a structured semantic knowledge base, knowledge graph plays an important role in the search engine field. However, existing knowledge graphs are often incomplete, that is, many possible facts are missing in the knowledge graph, resulting in the inability to quickly and accurately obtain the correct information during retrieval. In order to solve this problem, an effective knowledge graph completion method is needed to infer these missing facts through existing facts and external resources to improve the efficiency and accuracy of retrieval.

[0004] At present, there are some problems and challenges in the knowledge graph completion method: (1) How to effectively integrate the structured information of the knowledge graph and the unstructured information of the text. Existing methods often have difficulty in processing these two types of information at the same time, resulting in the inability to fully utilize all effective information. (2) In recent years, pre-trained language models, such as BERT, have achieved great success in the field of natural language processing. However, pre-trained language models are designed for text processing tasks and cannot be directly applied to knowledge graph completion tasks that require processing both graph structure information and text information. (3) Existing methods for applying pre-trained language models to the field of knowledge graph completion usually require fine-tuning the model, which is not only computationally expensive, but also may cause the model to overfit, limiting the generalization ability of the model.

[0005] In summary, knowledge graph completion is a technology with great value and challenges. It can mine more knowledge that is originally missing in the knowledge base for the knowledge graph. However, under the current technical level, there are still some problems, and a new method is needed to solve the problems of information fusion, application of pre-trained language models and high model calculation cost. Summary of the invention

[0006] In order to solve the problems that the structured information of the knowledge graph and the unstructured information of the text are difficult to integrate, the pre-trained language model cannot be directly applied to the knowledge graph completion task and the model calculation cost is high, the purpose of the present invention is to provide a knowledge graph completion method and system based on decoupled structure embedding.

[0007] In a first aspect, the present invention provides a knowledge graph completion method based on decoupled structure embedding, which comprises the following steps: Step 1: Obtain the knowledge graph dataset; Step 2: Construct a knowledge graph completion model; the knowledge graph completion model includes a feature extraction module, a structure embedding module, a text embedding module, a decoupled graph learner G1 and a pre-trained language model PLM1; the feature extraction module extracts the hidden layer embedding and inputs it into the structure embedding module and the text embedding module respectively; the decoupled graph learner G1 is used to obtain the structure embedding ; Pre-trained language model PLM1 is used to obtain text embedding ; The structure embedding module embeds and hidden layer embedding to obtain the structural side score ; The text embedding module embeds And hidden layer embedding to obtain text side score ; Score on the text side and structural side scores The probability score obtained after weighted fusion , as the output of the knowledge graph completion model; Step 3: Use the knowledge graph dataset constructed in step 1 to train the knowledge graph completion model; Step 4: Use the trained knowledge graph completion model to complete the missing components in the knowledge graph to be completed.

[0008] Preferably, in the step 1, the knowledge graph dataset includes an entity set, a relationship set and a triple set; the entity set includes multiple entities; each entity includes the name of the entity, the unique identifier of the entity and the text description of the entity; the relationship set includes multiple relationships, each relationship includes the name of the relationship, the unique identifier of the relationship and the text description of the relationship; the triple set includes multiple triples, each triple includes the unique identifiers of two entities and the unique identifiers of the corresponding relationships between the two entities.

[0009] Preferably, in the step 2, the feature extraction module includes a decoupled graph learner G2, an adapter, and a pre-trained language model PLM2 connected in sequence; the decoupled graph learner G2 and the adapter are used to obtain the corresponding decoupled structure embedding according to the unique identifier in a single triple ; The pre-trained language model PLM2 is embedded according to the decoupled structure of the input And task hard prompt , get the hidden layer embeddings that are input into the structure mapping layer and the text mapping layer respectively.

[0010] As a preferred embodiment, the task hard prompt X prompt = [ w CLS , w ℎ , w SEP , w r , w SEP , w MASK , w SEP ] ;in, The text description corresponding to the unique identifier of the known entity in the triple in the entity set; The text description corresponding to the unique identifier of the relation in the triple in the relation set; and Two special tags for the pre-trained language model PLM2; is a placeholder.

[0011] Preferably, the decoupled graph learner G2 has the same structure as the decoupled graph learner G1, and both include an entity embedding layer, a relationship embedding layer, a dimension expansion layer, an attention layer, and an output layer; the entity embedding layer and the dimension expansion layer connected in sequence process the unique identifier of the entity to obtain the decoupled entity vector; the relationship embedding layer is used to process the unique identifier of the relationship to obtain the relationship space matrix and the relationship vector ; The attention layer and output layer connected in sequence are used to process the outputs of the relation embedding layer and the dimension expansion layer to obtain the output of the decoupled graph learner.

[0012] Preferably, the adapter comprises an input matrix And the output matrix ; By inputting the matrix And the output matrix The output and relationship vector of the decoupled graph learner G2 are Processing to obtain decoupled known entity structure embedding and decoupled relational structure embedding ; Embedded in a decoupled structure As the output of the adapter.

[0013] Preferably, in the step 2, the structure embedding module includes a structure mapping layer, a knowledge graph embedding model and an activation function; the structure mapping layer obtains the structure representation according to the hidden layer embedding and inputs the knowledge graph embedding model and the activation function respectively; the knowledge graph embedding model is used to represent the structure and embed the structure. Processing is performed, and the output result is weightedly fused with the output of the activation function to obtain the structural side score ; The text embedding module adopts a text mapping layer; the output of the text mapping layer is fused with the text embedding Get the text side score .

[0014] In the second aspect, the present invention provides a knowledge graph completion system based on decoupled structural embedding, which is used to execute the above-mentioned knowledge graph completion method; the knowledge graph completion system includes a feature extraction module, a structure embedding module, a text embedding module and an output module; the feature extraction module is used to extract features in the knowledge graph dataset and input them into the structure embedding module and the text embedding module respectively; the structure embedding module is used to obtain the structure side score in the knowledge graph dataset; the text embedding module is used to obtain the text side score in the knowledge graph dataset; the output module is used to obtain the completion result after weighted fusion of the structure side score and the text side score.

[0015] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores the computer program; and the processor executes the above-mentioned knowledge graph completion method.

[0016] In a fourth aspect, the present invention provides a readable storage medium storing a computer program; when the computer program is executed by a processor, it is used to implement the above-mentioned knowledge graph completion method.

[0017] The present invention has the following beneficial effects: 1. The present invention utilizes the capabilities of the pre-trained language model and the method of prompt engineering to input the structural information of the knowledge graph into the pre-trained language model through prompts, and merges the text score and the structure score during output, thereby avoiding the situation where the powerful text processing capability of the pre-trained language model covers the structural information provided in the prompt. This is in line with the application of the pre-trained language model in the task of knowledge graph completion and improves the quality of the knowledge graph in the search engine.

[0018] 2. The present invention adds a relationship-aware attention mechanism to the decoupled graph learner, and decouples entities according to the different meanings they represent in different triples, so that the structural information of the knowledge graph can be more accurately transmitted to the text information, while allowing the structural information to more selectively utilize the text information, thereby achieving a balance when the structural information and text information are integrated, thereby increasing the accuracy of the knowledge graph in the search engine.

[0019] 3. When introducing the pre-trained language model, the present invention does not need to fine-tune it, which greatly reduces the time complexity while retaining the powerful generalization ability of the pre-trained language model, making the completion process of the knowledge graph in the search engine more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the overall flow chart of the present invention.

[0021] Figure 2 It is a structural schematic diagram of the knowledge graph completion model of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings.

[0023] Example 1 like Figure 1 As shown in FIG, a knowledge graph completion method based on decoupled structure embedding includes the following steps: Step 1: Get the dataset Get a knowledge graph dataset, which includes entity sets, relationship sets, and triple sets; the entity set includes N entities, each entity includes the name of the entity, the unique identifier of the entity, and the text description of the entity; the relationship set includes several relationships, each relationship includes the name of the relationship, the unique identifier of the relationship, and the text description of the relationship; the triple set includes several triples, each triple includes the unique identifiers of two entities and the unique identifiers of the corresponding relationships between the two entities. The two entities in the triple are respectively regarded as known entities and unknown entities, and the unknown entity is predicted.

[0024] Step 2: Build a knowledge graph completion model like Figure 2 As shown, the knowledge graph completion model includes a feature extraction module, a structure embedding module, a text embedding module, a decoupled graph learner G1 and a pre-trained language model PLM1.

[0025] 2-1. The feature extraction module includes a decoupled graph learner G2, an adapter, and a pre-trained language model PLM2 which are connected in sequence.

[0026] 2-1-1. The decoupled graph learner G2 includes an entity embedding layer, a relationship embedding layer, a dimension expansion layer, an attention layer, and an output layer. The entity embedding layer is used to convert the unique identifier of the entity in the triple into an entity vector with a dimension of d; the relationship embedding layer is used to convert the unique identifier r of the relationship in the triple into the relationship space matrix W r and relationship vector ; The dimension expansion layer is used to convert the entity vector into K decoupled entity vectors with dimension d through linear transformation; the attention layer applies the relationship-aware attention mechanism to obtain the decoupled entity vector in the relationship space matrix W according to the output of the relationship embedding layer and the dimension expansion layer r The attention score in , attention score The expression is:

[0027] in, is the decoupled entity vector obtained by the unique identifier i of the known entity, ; is the decoupled entity vector obtained by the unique identifier j of the unknown entity, ; is the set of all triples containing the unique identifier i of a known entity in the triple; r' and j' are the sets Unique identifiers for relations and unknown entities in the ; ; N is the number of entities in the entity set; softmax(·) is the normalized exponential function; T is the transposed symbol; is the Hadamard product; exp(·) is the exponential function.

[0028] The output layer is used to output the decoupled entity vector The final expression , and finally expressed The expression is: Among them, σ is a nonlinear activation function.

[0029] 2-1-2. The adapter includes a The input matrix and a dimension of The output matrix ;in, is the hidden dimension of the adapter; L is the number of layers of the pre-trained language model BERT; H is the hidden dimension of the pre-trained language model BERT; n is the length of the prompt that needs to be generated for each element. The adapter is used to output the final representation according to the decoupled graph learner G2 and relationship vector Get decoupled structural embedding ;in, To decouple known entity structure embedding, p i =[ p i 1 , p i 2 ,..., p i K ] ; Decoupled relational structure embedding. Decoupled known entity structure embedding The expression is:

[0030] Decoupled Relationship Hint Embedding The expression is: 2-1-3. The input of the pre-trained language model PLM2 is the decoupled structure embedding And task hard prompt X prompt = [ w CLS , w ℎ , w SEP , w r , w SEP , w MASK , w SEP ] , the hidden layer embedding of the output decoupling structure and the hidden layer embedding of the task hard prompt ;in, The text description corresponding to the unique identifier of the known entity in the triple in the entity set; The text description corresponding to the unique identifier of the relation in the triple in the relation set; and Two special tags for the pre-trained language model BERT; is a placeholder. The input of the first layer of the pre-trained language model PLM2 is the hidden layer embedding of the previous layer of decoupled structure and the hidden layer embedding of the task hard prompt , output the hidden layer embedding of the decoupled structure of the current layer and the hidden layer embedding of the task hard prompt ;in, ; Decoupled structure embedding And task hard prompt As and .

[0031] In this embodiment, the pre-trained language model adopts the BERT model (Bidirectional Transformer model).

[0032] 2-2. The structure of the decoupled graph learner G1 is the same as that of the decoupled graph learner G2; the input of the decoupled graph learner G1 is the unique identifier of all entities and relations in the triple, and the output is the structural embedding of all entities The structure of the pre-trained language model PLM1 is the same as that of the pre-trained language model PLM2; the input of the pre-trained language model PLM1 is the name and text description of all entities in the entity set, and the output is the text embedding of the entity .

[0033] 2-3. The structure embedding module includes the structure mapping layer, the knowledge graph embedding model and the softmax activation function. p2s The input is the hidden layer embedding , the output is the structural representation of the known entities in the triple The structural representation of the relationship p2s The softmax activation function is used for the structure mapping layer W p2s Output structure representation and r p2s Process and obtain the entity The structure of the elements Structural representation of relationship p2s The correlation score between , relevance score The expression is:

[0034] .

[0035] The input of the knowledge graph embedding model is the structural representation of a single entity and relationship and the structure represents r p2s And the structural embeddings of all entities , the output is the fusion score ; is the fusion score for each element; fusion score The expression is: Among them, γ is the preset parameter; ||·||2 is the symbol of the two-norm.

[0036] Based on relevance score Get the structure side score The expression is: 2-4. Text embedding module uses text mapping layer , whose input is the hidden layer embedding Placeholders in , text map layer The output of the joint text embedding Get the text side score , text side score The expression is: in, .

[0037] 2-5. Score the text side and structural side scores Weighted fusion to obtain the entities in the entity set as the likelihood scores of the unknown entities in the triplet , select the likelihood score The maximum value in As the output of the knowledge graph completion model, the likelihood score The expression is: Among them, α is the learnable weight.

[0038] Step 3: Training the model The knowledge graph completion model is trained using the knowledge graph dataset. The stochastic gradient descent optimization method is adopted. The unknown entities in the entity set are used as target triples and input into the knowledge graph completion model. The possibility scores of all entities as unknown entities of the target triples are obtained, and the cross entropy loss is calculated using the possibility scores. The parameters of the decoupled graph learner, adapter, text mapping layer, structure mapping layer, knowledge graph embedding model and learnable weight α are updated according to the cross entropy loss until the cross entropy loss stabilizes or the number of iterations reaches the maximum value. The iteration is stopped to obtain the final model.

[0039] Step 4: Complete the knowledge graph dataset.

[0040] All entities and all relationships are arranged and combined, and input into the knowledge graph completion model as known entities and relationships to obtain the maximum likelihood score. If the maximum likelihood score is greater than 50% of the preset threshold, the corresponding entities, relationships, and final prediction results are stored as new triples in the knowledge graph dataset.

[0041] Example 2 A knowledge graph completion system based on decoupled structure embedding, which is used to execute the knowledge graph completion method in Example 1; the knowledge graph completion system includes a feature extraction module, a structure embedding module, a text embedding module and an output module; the feature extraction module is used to extract features in the knowledge graph data set and input them into the structure embedding module and the text embedding module respectively; the structure embedding module is used to obtain the structure side score in the knowledge graph data set; the text embedding module is used to obtain the text side score in the knowledge graph data set; the output module is used to obtain the completion result after weighted fusion of the structure side score and the text side score.

[0042] It should be noted that the above is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, there will be changes in the specific implementation methods based on the ideas provided by the present invention, and these changes should also be regarded as the scope of protection of the present invention.

Claims

1. A knowledge graph completion method based on decoupled structure embedding, characterized by: The following steps are involved: Step 1: Obtain the knowledge graph dataset; Step 2: Construct a knowledge graph completion model; the knowledge graph completion model includes a feature extraction module, a structure embedding module, a text embedding module, a decoupled graph learner G1 and a pre-trained language model PLM1; the feature extraction module extracts the hidden layer embedding and inputs it into the structure embedding module and the text embedding module respectively; the decoupled graph learner G1 is used to obtain the structure embedding ; Pre-trained language model PLM1 is used to obtain text embedding ; The structure embedding module embeds and hidden layer embedding to obtain the structural side score ; The text embedding module embeds And hidden layer embedding to obtain text side score ; Score on the text side and structural side scores The probability score obtained after weighted fusion , as the output of the knowledge graph completion model; Step 3: Use the knowledge graph dataset constructed in step 1 to train the knowledge graph completion model; Step 4: Use the trained knowledge graph completion model to complete the missing components in the knowledge graph to be completed.

2. A knowledge graph completion method based on decoupled structure embedding according to claim 1, characterized in that: In the step 1, the knowledge graph dataset includes an entity set, a relationship set and a triple set; the entity set includes multiple entities; each entity includes the name of the entity, the unique identifier of the entity and the text description of the entity; the relationship set includes multiple relationships, each relationship includes the name of the relationship, the unique identifier of the relationship and the text description of the relationship; the triple set includes multiple triples, each triple includes the unique identifiers of two entities and the unique identifiers of the corresponding relationships between the two entities.

3. A knowledge graph completion method based on decoupled structure embedding according to claim 2, characterized in that: In the step 2, the feature extraction module includes a decoupled graph learner G2, an adapter, and a pre-trained language model PLM2 connected in sequence; the decoupled graph learner G2 and the adapter are used to obtain the corresponding decoupled structure embedding according to the unique identifier in a single triple ; The pre-trained language model PLM2 is embedded according to the decoupled structure of the input And task hard prompt , get the hidden layer embeddings that are input into the structure mapping layer and the text mapping layer respectively.

4. A knowledge graph completion method based on decoupled structure embedding according to claim 3, characterized in that: The task hard prompt ;in, The text description corresponding to the unique identifier of the known entity in the triple in the entity set; The text description corresponding to the unique identifier of the relation in the triple in the relation set; and Two special tags for the pre-trained language model PLM2; is a placeholder.

5. According to claim 3, a knowledge graph completion method based on decoupled structure embedding is characterized by: The decoupled graph learner G2 has the same structure as the decoupled graph learner G1, both of which include an entity embedding layer, a relationship embedding layer, a dimension expansion layer, an attention layer, and an output layer; the entity embedding layer and the dimension expansion layer connected in sequence process the unique identifier of the entity to obtain the decoupled entity vector; the relationship embedding layer is used to process the unique identifier of the relationship to obtain the relationship space matrix and the relationship vector ; The attention layer and output layer connected in sequence are used to process the outputs of the relation embedding layer and the dimension expansion layer to obtain the output of the decoupled graph learner.

6. A knowledge graph completion method based on decoupled structure embedding according to claim 5, characterized in that: The adapter includes an input matrix And the output matrix ; By inputting the matrix And the output matrix The output and relationship vector of the decoupled graph learner G2 are Processing to obtain decoupled known entity structure embedding and decoupled relational structure embedding ; Embedded in a decoupled structure As the output of the adapter.

7. The knowledge graph completion method based on decoupled structure embedding according to claim 1 is characterized by: In the step 2, the structure embedding module includes a structure mapping layer, a knowledge graph embedding model and an activation function; the structure mapping layer obtains the structure representation according to the hidden layer embedding and inputs the knowledge graph embedding model and the activation function respectively; the knowledge graph embedding model is used to represent the structure and embed the structure. Processing is performed, and the output result is weightedly fused with the output of the activation function to obtain the structural side score ; The text embedding module adopts a text mapping layer; the output of the text mapping layer is fused with the text embedding Get the text side score .

8. A knowledge graph completion system based on decoupled structure embedding, characterized by: Used to execute a knowledge graph completion method based on decoupled structure embedding as described in any one of claims 1 to 7; the knowledge graph completion system includes a feature extraction module, a structure embedding module, a text embedding module and an output module; The feature extraction module is used to extract features from the knowledge graph dataset and input them into the structure embedding module and the text embedding module respectively; the structure embedding module is used to obtain the structure side score in the knowledge graph dataset; the text embedding module is used to obtain the text side score in the knowledge graph dataset; The output module is used to obtain the completion result by weighted fusion of the structure-side score and the text-side score.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The memory stores computer programs; the processor executes a knowledge graph completion method based on decoupled structure embedding as described in any one of claims 1-7.

10. A readable storage medium storing a computer program; characterized in that: When the computer program is executed by a processor, it is used to implement a knowledge graph completion method based on decoupled structure embedding as described in any one of claims 1 to 7.

Images