A few-shot knowledge graph completion model and method

Abstract

The application belongs to the technical field of deep learning and knowledge graph completion, and relates to a few-shot knowledge graph completion model and method, wherein the completion model is divided into three modules: (1) LLM-PCA semantic encoder: responsible for extracting the text semantic features of entities and relations in the knowledge graph triplets, and reducing the dimension of the extracted semantic feature vectors to obtain effective low-dimensional semantic representation of the knowledge graph elements; (2) attention mapping network: responsible for fusing the initial semantic encoding vectors of the triplets and mapping the text semantic representation vectors of the triplets under the same relation to a unified semantic space, combining the structural information with the text semantics; (3) authenticity scoring module: responsible for judging the possibility of the real existence of the to-be-queried triplets. The method can not only effectively handle complex relations and sparse data, but also realize high-quality knowledge graph completion under the condition of a small amount of training samples, and significantly improves the accuracy and generalization ability of the completion.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of knowledge graph completion, and particularly relates to a knowledge graph completion model and method based on semantic information, which are mainly used for solving the problem of how to use the few-sample knowledge in the knowledge graph to complete the knowledge graph reasoning. BACKGROUND

[0002] As a structured knowledge representation method, knowledge graph is widely used in artificial intelligence and natural language processing. However, due to the complexity and dynamics of knowledge acquisition, large-scale knowledge graphs often have incomplete information, and need to be completed by knowledge graph reasoning. Traditional knowledge graph completion methods mainly focus on the structural information of the knowledge graph, but often ignore the effective use of text semantic information, and have limitations in dealing with complex relationships and sparse data. In addition, existing methods usually require a large number of training samples and perform poorly in the face of new relationships or few-sample scenarios. Therefore, it is necessary to design a knowledge graph completion method that can effectively utilize semantic information and adapt to few-sample learning, in order to improve the accuracy and generalization ability of completion. SUMMARY

[0003] The application provides a few-sample knowledge graph completion model and method to solve the problems in the prior art.

[0004] To achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0005] A few-sample knowledge graph completion model, comprising an LLM-PCA semantic encoder, an attention mapping network and a truth score module.

[0006] The LLM-PCA semantic encoder comprises a large model semantic information extraction module and a principal component analysis module, and is used to extract the text semantic features of entities and relationships in the knowledge graph triple. The semantic feature vectors of entity and relationship text are extracted by a large language model, and the extracted semantic feature vectors are subjected to principal component analysis PCA dimension reduction to obtain effective low-dimensional semantic representation of knowledge graph elements.

[0007] The attention mapping network comprises a self-attention mechanism layer and a multi-layer mapping network layer. The self-attention mechanism layer adaptively learns the interaction and mapping relationship between the text semantics of entities and relationship types, and the multi-layer mapping network layer and the contrast learning method are used to map the text semantic representation vectors of the triples under the same relationship to a unified semantic space, combining the structural information with the text semantics.

[0008] The truth score module is used to calculate the similarity between the query triple and the support set, and to score the truth of the query triple, thereby completing the few-sample knowledge graph completion.

[0009] Further, a few-shot knowledge graph completion method using the few-shot knowledge graph completion model is provided, comprising:

[0010] 1) The processing flow of the LLM-PCA semantic encoder is as follows:

[0011] The knowledge graph entity and relationship text are preprocessed to remove the spaces between words;

[0012] The pre-trained large-scale language model is used to encode the text description of the triple entity and relationship;

[0013] The principal component analysis dimensionality reduction is performed on the encoded feature vector to obtain a compact semantic representation vector;

[0014] 2) The processing flow of the attention mapping network is as follows:

[0015] The semantic vector of the triple is input into the multi-head self-attention network layer to dynamically assign the importance weight of each component;

[0016] The semantic vector aggregated by the attention is mapped to the semantic space through the multi-layer perceptron network;

[0017] The negative sample triple is constructed, and the similarity between the real triple and the negative sample triple is calculated;

[0018] The contrast loss function is designed to make the semantic similarity of the real triple under the same relationship high and the semantic similarity of the negative sample triple low.

[0019] 3) The processing flow of the authenticity scoring module is as follows:

[0020] The known triple under the same relationship is taken as the support set, and the support set semantic vector representation of the relationship is obtained by averaging the semantic vector representations of each triple;

[0021] The to-be-queried triple is input into the above model to obtain the semantic vector representation of the query triple;

[0022] The similarity semantic vector representation of the query triple and the support set is calculated, the authenticity of the query triple is scored, and the few-shot knowledge graph completion is realized.

[0023] To optimize the above technical solution, the specific measures taken further include:

[0024] Further, the specific steps of the LLM-PCA semantic encoder are as follows:

[0025] S1: Preprocess the entity h and relationship r name, remove the spaces between words, and ensure that the multi-word entity and relationship name is processed as a single unit;

[0026] S2: input the preprocessed text into the tokenizer of the large language model for word segmentation;

[0027] S3: encode the segmented sequence using the word embedding layer and multi-layer Transformer encoder of the large language model to obtain the i-th layer hidden layer vector

[0028] where e i represents the i-th word element, and d represents the hidden layer dimension of the large model;

[0029] S4: extract the hidden state vectors of the first and last layers of the Transformer encoder, and average pool all word elements to obtain the semantic vector representation of the text

[0030] where L represents the number of hidden layers of the large model;

[0031] S5: perform principal component analysis on the semantic vector to reduce the dimension to obtain the final compact semantic representation

[0032] where k represents the dimension size of the semantic vector after dimension reduction.

[0033] Further, the specific steps of the attention mapping network are as follows:

[0034] S1: arrange the semantic vectors of the triplets into a matrix

[0035] S2: fuse information through a multi-head self-attention network layer to obtain an output vector y = att(X);

[0036] S3: input y into a multi-layer perceptron network to obtain the final triplet semantic representation vector:

[0037] z = MLP(y) = W n (... (W2(W1y + b1) + b2)...) + b n

[0038] where MLP represents a multi-layer perceptron, W i represents the i-th layer matrix, and b i represents the i-th layer bias vector.

[0039] Further, the specific steps of the contrastive learning are as follows:

[0040] S1: for each real triplet (h, r, t), construct a negative sample triplet (h, r, t') by replacing the tail entity;

[0041] S2: input the positive and negative sample triplets into the above model to obtain the final semantic vectors;

[0042] S3: Calculate the cosine similarity between the real triple and the negative sample triple;

[0043] S4: Design the contrast loss function:

[0044]

[0045] wherein z pos represents the semantic representation vector of the positive sample, z neg represents the semantic representation vector of the negative sample, cosine(·, ·) represents the cosine similarity function, and mean(·) represents the average operation;

[0046] S5: Use the Adam optimizer to optimize the parameters on the training set.

[0047] Further, the specific steps of the authenticity scoring module are as follows:

[0048] S1: Take the known triples under the same relationship as the support set S r , and obtain the semantic vector representation of the support set of the relationship by averaging the semantic vector representations of each triple

[0049] S2: Input the triple to be queried into the above model to obtain the semantic vector representation z q of the query triple;

[0050] S3: Calculate the similarity semantic vector representation of the query triple and the support set, and score the authenticity of the query triple:

[0051]

[0052] The beneficial effects of the present application are: a new knowledge graph completion method is constructed to improve the integrity and accuracy of the knowledge graph. It divides the completion process into two stages of semantic encoding and relationship reasoning. In the semantic encoding stage, the LLM-PCA semantic encoder effectively extracts the semantic information of the text, saving a lot of manual feature engineering work, thereby improving the generalization ability of the model. In the relationship reasoning stage, the attention mapping network and the scoring module can accurately capture the complex relationship between entities and have good adaptability to new relationships and few sample scenarios. This method not only effectively handles complex relationships and sparse data, but also realizes high-quality knowledge graph completion under the condition of a small amount of training samples, significantly improving the accuracy and generalization ability of the completion. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is the framework diagram of the knowledge graph completion model of the present application; DETAILED DESCRIPTION

[0054] The application will be described in further detail below with reference to the drawings.

[0055] As Figure 1 The main idea of the few-shot knowledge graph completion model is shown in the figure: the knowledge graph completion system is divided into three modules: LLM-PCA semantic encoder, attention mapping network and authenticity scoring module. The three modules each process the corresponding task, while being coupled with each other to achieve the purpose of knowledge graph completion.

[0056] The LLM-PCA semantic encoder is mainly responsible for extracting the text semantic features of entities and relationships in the knowledge graph triplets. The semantic feature vectors of entity and relationship texts are extracted through a large language model, and the extracted semantic feature vectors are subjected to principal component analysis PCA dimension reduction to obtain effective low-dimensional semantic representations of knowledge graph elements;

[0057] The attention mapping network is responsible for fusing and mapping the initial semantic encoding vectors of the triplets to a common semantic space. Through a self-attention mechanism, the network adaptively learns the interaction and mapping relationship between the text semantics of entities and relationship types, and through a multi-layer mapping network layer and a contrastive learning method, the text semantic representation vectors of triplets under the same relationship are mapped to a unified semantic space, combining structural information with text semantics.

[0058] The authenticity scoring module is responsible for judging the possibility of the existence of the query triplets. The cosine similarity between the semantic vector of the query triplet and the average semantic representation vector of the support set triplets is calculated to score the authenticity of the query triplets, achieving few-shot knowledge graph completion.

[0059] The specific steps of the LLM-PCA semantic encoder are as follows: a pre-trained large language model can extract semantic information from text to form a semantic representation vector. Based on this, the entity and relationship texts of the knowledge graph are preprocessed to remove spaces between words, and a pre-trained large-scale language model is used to encode the text descriptions of the triplet entities and relationships. Then, the encoded feature vectors are subjected to principal component analysis dimension reduction to obtain compact semantic representation vectors;

[0060] Collect the text description information of entities and relationships in the knowledge graph triplets, such as the head entity name h, the relationship name r, and the tail entity name t.

[0061] Input the collected text information into a large language model to extract semantic features. For multi-word entity names, remove spaces between words during preprocessing to ensure that they are processed as a single unit.

[0062] Use the tokenizer of the large language model to decompose the text into word pieces.

[0063] Use the pre-trained word embedding layer parameters to convert the word pieces into word vectors ei .

[0064] The word vectors are input into the multi-layer Transformer encoder of the large language model in the order of the text. Let the output of the i-th layer Transformer encoder on the i-th word embedding e i be

[0065] The semantic vector representation of the entire text is obtained by aggregating all word representations through a pooling operation:

[0066]

[0067] where L is the number of layers of the Transformer encoder, and n is the number of words. This pooling method combines the model's understanding of text semantics from shallow to deep, obtaining a comprehensive and concise text semantic representation.

[0068] The principal component analysis (PCA) method is used to reduce the dimension of the semantic feature vector, obtaining a low-dimensional semantic representation:

[0069]

[0070] where k is the dimension after dimension reduction, which is much smaller than the original dimension d.

[0071] The specific steps of the attention mapping network are as follows: the attention mapping network first learns the interaction and mapping relationship between the text semantics of entities and relationship types through self-attention mechanism, and then maps the text semantic representation vectors of triples under the same relationship to a unified semantic space through multi-layer mapping network layers and contrast learning method, combining structure information with text semantics.

[0072] The semantic vectors of the head entity h * , the relationship r * and the tail entity t * of the knowledge graph triple are concatenated into a matrix X:

[0073]

[0074] Information fusion is performed on X using multi-head self-attention mechanism:

[0075] y = att(X)

[0076] where att represents the multi-head self-attention network layer.

[0077] The vector after attention fusion is further processed using a multi-layer perceptron (MLP):

[0078] z = MLP(y) = W n (... (W2(W1y + b1) + b2)...) + bn

[0079] where W i and b i represent the weight matrix and bias vector of the i-th layer MLP respectively.

[0080] In the training phase, for each real triple (h, r, t), a negative sample triple (h, r, t') is constructed by replacing the tail entity;

[0081] The positive and negative sample triples are respectively input into the above model to obtain the final semantic vector;

[0082] The contrast loss function and the L2 regularization loss function are designed:

[0083] The contrast loss function is:

[0084]

[0085] where Z pos represents the semantic representation vector of the positive sample, Z neg represents the semantic representation vector of the negative sample, cosine(·, ·) represents the cosine similarity function, and mean(·) represents the average operation.

[0086] The L2 regularization loss function is:

[0087]

[0088] where Θ represents the model parameter set, p represents a single parameter, and λ l2 is a hyperparameter that controls the strength of L2 regularization.

[0089] The total loss function is calculated as:

[0090] L = L con + L l2

[0091] The Adam optimizer is used to optimize the parameters on the training set.

[0092] The specific steps of the authenticity scoring module are: first, the existing few triples under the same relationship are taken as the support set to obtain their respective semantic representation vectors, then the prototype vector of the relationship is obtained by averaging the evaluation, the cosine similarity between the semantic vector of the query triple and the prototype representation vector of the relationship is calculated to score the authenticity of the query triple, and the few-shot knowledge graph completion is realized.

[0093] The known triples under the same relationship are taken as the support set S r , and the semantic vector representation of the support set of the relationship is obtained by averaging the semantic vector representations of each triple

[0094] Input the query triple to be queried into the above model to obtain a semantic vector representation z of the query triple q ;

[0095] Calculate the similarity of the semantic vector representation of the query triple and the support set, and score the authenticity of the query triple:

[0096]

[0097] Based on the score of the query triple, it is judged whether the triple exists or not, and the triple that exists is taken as the result of the knowledge graph completion.

[0098] It should be noted that the terms such as "up", "down", "left", "right", "front", "back" and the like cited in the invention are only for the convenience of clear description, and are not used to limit the scope of the implementation of the invention. The change or adjustment of the relative relationship is also considered as the implementation of the invention without substantial change of the technical content.

[0099] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the scope of the present application is within the protection scope of the present application. It should be noted that for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application are considered as the protection scope of the present application.

Claims

1. A few-shot knowledge graph completion model, characterized in that, The LLM-PCA semantic encoder comprises a large model semantic information extraction module and a principal component analysis module, and is used for extracting text semantic features of entities and relationships in a knowledge graph triple; a semantic feature vector of the entity and relationship text is extracted through a large language model, and principal component analysis PCA dimension reduction is performed on the extracted semantic feature vector to obtain an effective low-dimensional semantic representation of a knowledge graph element; The attention mapping network comprises a self-attention mechanism layer and a multi-layer mapping network layer, learns the interaction and mapping relationship between the text semantics of the entity and the relationship type adaptively through the self-attention mechanism, and maps the text semantic representation vectors of the triples under the same relationship to a unified semantic space through the multi-layer mapping network layer and a contrast learning method, and combines the structural information with the text semantics; The authenticity scoring module takes the known triples under the same relationship as a support set, is used for calculating the similarity of the query triple and the support set, scoring the authenticity of the query triple, and realizing the few-shot knowledge graph completion. It comprises:

2. A few-shot knowledge graph completion method employing the few-shot knowledge graph completion model according to claim 1, characterized in that, 1) The processing procedure of the LLM-PCA semantic encoder is as follows: The entity and relationship text of the knowledge graph is preprocessed to remove the spaces between words; The text description of the triple entity and relationship is encoded by using a pre-trained large-scale language model; The dimension of the encoded feature vector is reduced by principal component analysis to obtain a compact semantic representation vector; 2) The processing procedure of the attention mapping network is as follows: The semantic vector of the triple is input into a multi-head self-attention network layer to dynamically allocate the importance weight of each component; The semantic vector aggregated by attention is mapped to a semantic space through a multi-layer perception network; A negative sample triple is constructed, and the similarity between the real triple and the negative sample triple is calculated; A contrast loss function is designed to make the semantic similarity of the real triple under the same relationship high and the semantic similarity of the negative sample triple low; 3) The processing procedure of the authenticity scoring module is as follows: The known triples under the same relationship are taken as a support set, and the semantic vector representation of the support set of the relationship is obtained by averaging the semantic vector representations of the triples; The query triple is input into the above model to obtain the semantic vector representation of the query triple; The similarity of the semantic vector representation of the query triple and the support set is calculated, the authenticity of the query triple is scored, and the few-shot knowledge graph completion is realized. The specific steps of the LLM-PCA semantic encoder are as follows:

3. The few-shot knowledge graph completion method of claim 2, wherein, S1: Preprocess the head entity h, the relationship r and the tail entity t, remove the spaces between words, and ensure that the multi-word entity and relationship name are processed as a single unit; S2: Input the preprocessed text into the tokenizer of the large-scale language model for tokenization; S4: Extract the hidden state vectors of the first layer and the last layer of the Transformer encoder, and average pool all the word units to obtain the semantic vector representation of the text: S3: encode the segmented sequence using the word embedding layer and multi-layer Transformer encoder of the large-scale language model to obtain the i-th layer hidden layer vector where e i denotes the i-th wordpiece, and d denotes the dimension of the hidden layer of the model; Wherein L represents the number of hidden layers of the large model, n represents the number of word units of the input sequence, and d represents the dimension of the hidden layer; S5: Perform principal component analysis PCA dimension reduction on the semantic vector to obtain the final compact semantic representation: Wherein k is the dimension after dimension reduction. ​ 4. The few-shot knowledge graph completion method of claim 3, wherein, The process of the attention mapping network is as follows: S1: arrange the semantic vectors of the triples into a matrix where h * , r * and t * are compact representations of the head entity h, the relation r and the tail entity t, respectively; S2: information fusion is performed on X through a multi-head self-attention network layer to obtain an output vector y = att(X), wherein att represents a multi-head self-attention fusion operation; S3: y is input into a multi-layer perception network to obtain a final triple semantic representation vector z: z = MLP(y) = W n (...(W2(W1y + b1) + b2)...) + b n where W i and b i denote the weight matrix and bias vector of the i-th layer MLP, respectively.

5. The few-shot knowledge graph completion method of claim 4, wherein, The process of the contrastive learning is as follows: S1: for each real triple (h, r, t), a negative sample triple (h, r, t') is constructed by replacing the tail entity; S2: the positive and negative sample triples are respectively input into the above model to obtain final semantic vectors; S3: the cosine similarity between the real triple and the negative sample triple is calculated; S4: a contrastive loss function is designed: where z pos represents the semantic representation vector of the positive sample, Z neg represents the semantic representation vector of the negative sample, cosine represents the cosine similarity function, and mean represents the average operation; S5: an Adam optimizer is used to optimize parameters on the training set.

6. The few-shot knowledge graph completion method of claim 2, wherein, The process of the authenticity scoring module is as follows: S1: take known triples under the same relation as the support set S r get the support set semantic vector representation of the relation by averaging the semantic vector representations of each triple S2: input the to-be-queried triple into the above model to obtain a semantic vector representation z of the query triple q ; S3: the similarity semantic vector representation of the query triple and the support set is calculated, and the authenticity score of the query triple is calculated: S4: whether the triple exists is determined according to the authenticity score of the query triple, and the triple that exists is added to the existing knowledge graph, so that the purpose of knowledge graph completion is achieved.

Images