Few-sample knowledge graph completion model and method

Abstract

The invention belongs to the technical field of deep learning and knowledge graph completion, and relates to a few-sample knowledge graph completion model and a few-sample knowledge graph completion method, the completion model is divided into three modules: (1) an LLM-PCA semantic encoder is responsible for extracting text semantic features of entities and relationships in knowledge graph triads, and performing dimension reduction on extracted semantic feature vectors to obtain an LLM-PCA matrix; effective knowledge graph element low-dimensional semantic representation is obtained; (2) an attention mapping network which is responsible for fusing triad initial semantic coding vectors, mapping triad text semantic representation vectors under the same relation to a unified semantic space, and combining structure information with text semantics; and (3) an authenticity scoring module which is responsible for judging the possibility of true existence of the triad to be queried. According to the method, not only can complex relations and sparse data be effectively processed, but also high-quality knowledge graph completion can be realized under the condition of a small number of training samples, and the accuracy and generalization ability of completion are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graph completion, and specifically relates to a knowledge graph completion model and method based on semantic information, which is mainly used to solve the problem of how to use a small number of sample knowledge in a knowledge graph to perform knowledge graph reasoning and completion. Background Art

[0002] As a structured knowledge representation method, knowledge graphs have been widely used in fields such as artificial intelligence and natural language processing. However, due to the complexity and dynamism of knowledge acquisition, large-scale knowledge graphs usually have incomplete information and require knowledge graph reasoning and completion. Traditional knowledge graph completion methods mainly focus on the structural information of knowledge graphs, but often ignore the effective use of text semantic information and have limitations in dealing with complex relations and sparse data. In addition, existing methods usually require a large number of training samples and perform poorly when faced with new relations or few-sample scenarios. Therefore, it is necessary to design a knowledge graph completion method that can effectively utilize semantic information and adapt to few-sample learning to improve the accuracy and generalization ability of completion. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides a few-sample knowledge graph completion model and method.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A few-shot knowledge graph completion model, including: LLM-PCA semantic encoder, attention mapping network and authenticity scoring module;

[0006] The LLM-PCA semantic encoder includes a large model semantic information extraction module and a principal component analysis module, which are used to extract the text semantic features of entities and relationships in the knowledge graph triples. The semantic feature vectors of entity and relationship texts are extracted through a large language model, and the principal component analysis PCA dimensionality reduction is performed on the extracted semantic feature vectors to obtain an effective low-dimensional semantic representation of knowledge graph elements;

[0007] The attention mapping network includes a multi-head self-attention network layer and a multi-layer perceptron network layer. It adaptively learns the interaction and mapping relationship between the text semantics of entity and relationship types through the self-attention mechanism, and maps the triple text semantic representation vectors under the same relationship to a unified semantic space through the multi-layer perceptron network layer and the contrastive learning method, thereby combining structural information with text semantics.

[0008] The authenticity scoring module is used to calculate the similarity between the query triple and the support set, perform authenticity scoring on the query triple, and achieve few-sample knowledge graph completion.

[0009] In addition, a few-shot knowledge graph completion method using the above few-shot knowledge graph completion model is proposed, including:

[0010] 1) The processing flow of the LLM-PCA semantic encoder is as follows:

[0011] Preprocess the knowledge graph entities and relationship texts to remove spaces between words;

[0012] Encode textual descriptions of triple entities and relations using a pre-trained large-scale language model;

[0013] Perform principal component analysis on the encoded feature vector to reduce its dimension and obtain a compact semantic representation vector;

[0014] 2) The processing flow of the attention mapping network is as follows:

[0015] The semantic vector of the triplet is input into the multi-head self-attention network layer to dynamically assign the importance weight of each component;

[0016] The semantic vector after attention aggregation is mapped to the semantic space through a multi-layer perceptron network;

[0017] Construct negative sample triples and calculate the similarity between the true triples and the negative sample triples;

[0018] The contrast loss function is designed so that the semantic similarity of true triplets under the same relation is high and the semantic similarity with negative sample triplets is low.

[0019] 3) The processing flow of the authenticity scoring module is as follows:

[0020] The known triples under the same relation are taken as the support set, and the support set semantic vector representation of the relation is obtained by averaging the semantic vector representations of each triple;

[0021] Input the query triple into the above model to obtain the semantic vector representation of the query triple;

[0022] The similarity semantic vector representation between the query triple and the support set is calculated, and the authenticity score of the query triple is performed to achieve few-sample knowledge graph completion.

[0023] To optimize the above technical solutions, the specific measures taken also include:

[0024] Furthermore, the specific steps of the LLM-PCA semantic encoder are as follows:

[0025] S1: Preprocess entity h and relation r names, remove spaces between words, and ensure that multi-word entity and relation names are processed as a single unit;

[0026] S2: Input the preprocessed text into the word segmenter of the large-scale language model for word segmentation;

[0027] S3: Use the word embedding layer of the large-scale language model and the multi-layer Transformer encoder to encode the sequence after word segmentation to obtain the i-th hidden layer vector

[0028] Where ei represents the i-th word, and d represents the hidden layer dimension of the large model;

[0029] S4: Extract the hidden state vectors of the first and last layers of the Transformer encoder, perform average pooling on all word units, and obtain the semantic vector representation of the text

[0030] Where L represents the number of hidden layers of the large model;

[0031] S5: Perform principal component analysis on the semantic vector to reduce its dimension and obtain the final compact semantic representation

[0032] Where k represents the dimension size of the semantic vector after dimensionality reduction.

[0033] Furthermore, the specific steps of the attention mapping network are as follows:

[0034] S1: Arrange the semantic vectors of triples into a matrix

[0035] S2: Information fusion of x is performed through the multi-head self-attention network layer to obtain the output vector y = att(x);

[0036] S3: Input y into the multi-layer perceptron network to obtain the final triple semantic representation vector:

[0037] z=MLP(y)=W n (…(W2(W1y+b1)+b2)…)+b n

[0038] Where MLP represents a multi-layer perceptron network, W i represents the i-th layer matrix, b i Represents the bias vector of the i-th layer.

[0039] Furthermore, the specific steps of contrastive learning are as follows:

[0040] S1: For each true triple (h, r, t), construct a negative sample triple (h, r, t′) by replacing the tail entity;

[0041] S2: Input the positive and negative sample triplets into the above model to obtain the final semantic vector;

[0042] S3: Calculate the cosine similarity between the true triples and the negative sample triples;

[0043] S4: Design contrast loss function:

[0044]

[0045] where z pos Represents the semantic representation vector of the positive sample, z neg represents the semantic representation vector of negative samples, cosine(·,·) represents the cosine similarity function, and mean(·) represents the average operation;

[0046] S5: Use the Adam optimizer to perform parameter tuning on the training set.

[0047] Furthermore, the specific steps of the authenticity scoring module are as follows:

[0048] S1: Take the known triples under the same relation as the support set S r , by averaging the semantic vector representations of each triplet, we get the semantic vector representation of the support set of the relationship

[0049] S2: Input the query triple into the above model to obtain the semantic vector representation z of the query triple q ;

[0050] S3: Calculate the similarity semantic vector representation between the query triple and the support set, and perform authenticity scoring on the query triple:

[0051]

[0052] The beneficial effects of the present invention are as follows: a new knowledge graph completion method is constructed to improve the integrity and accuracy of the knowledge graph. It divides the completion process into two stages: semantic encoding and relational reasoning. In the semantic encoding stage, the LLM-PCA semantic encoder is used to effectively extract text semantic information, saving a lot of manual feature engineering work, thereby improving the generalization ability of the model. In the relational reasoning stage, the attention mapping network and the scoring module can accurately capture the complex relationship between entities, and have good adaptability to new relationships and few sample scenarios. This method can not only effectively handle complex relationships and sparse data, but also achieve high-quality knowledge graph completion with a small number of training samples, significantly improving the accuracy and generalization ability of completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a framework diagram of the knowledge graph completion model of the present invention; DETAILED DESCRIPTION

[0054] The present invention will now be described in further detail with reference to the accompanying drawings.

[0055] like Figure 1 The main idea of ​​the few-shot knowledge graph completion model shown is to divide the knowledge graph completion system into three modules: LLM-PCA semantic encoder, attention mapping network and authenticity scoring module. These three modules handle corresponding tasks respectively and are coupled with each other to achieve the purpose of knowledge graph completion.

[0056] The LLM-PCA semantic encoder is mainly responsible for extracting the textual semantic features of entities and relations in the knowledge graph triples. It extracts the semantic feature vectors of entity and relationship texts through a large language model, and performs principal component analysis (PCA) dimensionality reduction on the extracted semantic feature vectors to obtain an effective low-dimensional semantic representation of knowledge graph elements.

[0057] The attention mapping network is responsible for fusing the initial semantic encoding vectors of the triples and mapping them into a common semantic space. It adaptively learns the interaction and mapping relationship between the text semantics of entity and relationship types through the self-attention mechanism, and maps the triple text semantic representation vectors under the same relationship to a unified semantic space through a multi-layer perceptron network layer and contrastive learning method, combining structural information with text semantics.

[0058] The authenticity scoring module is responsible for judging the possibility of the real existence of the queried triple. It scores the authenticity of the query triple by calculating the cosine similarity between the semantic vector of the query triple and the average semantic representation vector of the support set triple, thereby achieving few-sample knowledge graph completion.

[0059] The specific steps of the LLM-PCA semantic encoder are: the pre-trained large language model can extract the semantic information of the text to form a semantic representation vector. Based on this, the knowledge graph entity and relationship text are pre-processed to remove the spaces between words, and the pre-trained large-scale language model is used to encode the text description of the triple entity and relationship. Then, the encoded feature vector is subjected to principal component analysis and dimensionality reduction to obtain a compact semantic representation vector;

[0060] Collect text description information of entities and relations in knowledge graph triples, such as head entity name h, relation name r, and tail entity name t.

[0061] The collected text information is fed into a large language model for semantic feature extraction. For multi-word entity names, spaces between words are removed during preprocessing to ensure that they are processed as a single unit.

[0062] A tokenizer that uses a large language model to break text into tokens.

[0063] Use the pre-trained word embedding layer parameters to convert word units into word vectors e i.

[0064] The word vectors are input into the multi-layer Transformer encoder of the large language model in text order. Assume that the i-th layer Transformer encoder embeds the i-th word element e i The output is

[0065] All word-unit representations are aggregated through pooling operations to obtain the semantic vector representation of the entire text:

[0066] Where L is the number of layers of the Transformer encoder and n is the number of tokens. This pooling method combines the model's understanding of text semantics from shallow to deep layers to obtain a comprehensive and concise text semantic representation.

[0067] The principal component analysis (PCA) method is used to reduce the dimension of the semantic feature vector to obtain a low-dimensional semantic representation:

[0068] Where k is the dimension after dimensionality reduction, which is much smaller than the original dimension d.

[0069] The specific steps of the attention mapping network are: the attention mapping network first adaptively learns the interaction and mapping relationship between the text semantics of entity and relationship types through the self-attention mechanism, and then maps the triple text semantic representation vector under the same relationship to a unified semantic space through a multi-layer perceptron network layer and contrastive learning method, combining structural information with text semantics.

[0070] Concatenate the semantic vectors of the head entity h*, relation r*, and tail entity t* of the knowledge graph triple into a matrix X:

[0071] Use the self-attention mechanism to fuse information of X: y=att(X)

[0072] Where att represents the multi-head self-attention network layer.

[0073] The attention fused vector is further processed using a multi-layer perceptron (MLP): z=MLP(y)=W n (…(W2(W1y+b1)+b2)…)+b n

[0074] Where W i and b i They represent the weight matrix and bias vector of the i-th layer MLP respectively.

[0075] In the training phase, for each true triple (h, r, t), a negative sample triple (h, r, t′) is constructed by replacing the tail entity;

[0076] Input the positive and negative sample triplets into the above model to obtain the final semantic vector;

[0077] The present invention designs a contrast loss function and an L2 regularization loss function:

[0078] Contrastive loss function:

[0079] where z pos Represents the semantic representation vector of the positive sample, z neg represents the semantic representation vector of negative samples, cosine(·,·) represents the cosine similarity function, and mean(·) represents the average operation;

[0080] L2 regularization loss function:

[0081] where Θ represents the model parameter set, p represents a single parameter, and λ 12 is a hyperparameter that controls the strength of L2 regularization.

[0082] Calculate the total loss function:

[0083] The Adam optimizer is used to tune parameters on the training set.

[0084] The specific steps of the authenticity scoring module are: first, a small number of existing triples under the same relationship are used as support sets to calculate their respective semantic representation vectors, and then the prototype vector of the relationship is obtained by averaging. By calculating the cosine similarity between the semantic vector of the query triple and the prototype representation vector of the relationship, the authenticity of the query triple is scored to achieve few-sample knowledge graph completion.

[0085] Take the known triples under the same relation as the support set S r , by averaging the semantic vector representations of each triplet, we get the semantic vector representation of the support set of the relationship

[0086] Input the query triple into the above model to obtain the semantic vector representation z of the query triple q ;

[0087] Calculate the similarity semantic vector representation between the query triple and the support set, and perform authenticity scoring on the query triple

[0088] Based on the score of the query triple, determine whether the triple really exists, and use the real triple as the result of knowledge graph completion.

[0089] It should be noted that the terms such as "upper", "lower", "left", "right", "front", "back", etc. cited in the invention are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments in their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0090] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A few-sample knowledge graph completion model, characterized in that: include: LLM-PCA semantic encoder, attention mapping network and authenticity scoring module: The LLM-PCA semantic encoder includes a large model semantic information extraction module and a principal component analysis module, which are used to extract text semantic features of entities and relationships in knowledge graph triples; extract semantic feature vectors of entity and relationship texts through a large language model, and perform principal component analysis PCA dimensionality reduction on the extracted semantic feature vectors to obtain an effective low-dimensional semantic representation of knowledge graph elements; The attention mapping network includes a self-attention mechanism layer and a multi-layer mapping network layer, which adaptively learns the interaction and mapping relationship between the text semantics of entity and relationship types through the self-attention mechanism, and maps the triple text semantic representation vector under the same relationship to a unified semantic space through the multi-layer mapping network layer and the contrastive learning method, so as to combine the structural information with the text semantics; The authenticity scoring module is used to calculate the similarity between the query triple and the support set, perform authenticity scoring on the query triple, and achieve few-sample knowledge graph completion.

2. A method for completing a few-sample knowledge graph using the few-sample knowledge graph completion model according to claim 1, characterized in that: include: 1) The processing flow of the LLM-PCA semantic encoder is as follows: Preprocess the knowledge graph entities and relationship texts to remove spaces between words; Encode textual descriptions of triple entities and relations using a pre-trained large-scale language model; Perform principal component analysis on the encoded feature vector to reduce its dimension and obtain a compact semantic representation vector; 2) The processing flow of the attention mapping network is as follows: The semantic vector of the triplet is input into the multi-head self-attention network layer to dynamically assign the importance weight of each component; The semantic vector after attention aggregation is mapped to the semantic space through a multi-layer perceptron network; Construct negative sample triples and calculate the similarity between the true triples and the negative sample triples; Design a contrast loss function so that the semantic similarity of true triplets under the same relation is high and the semantic similarity with negative sample triplets is low; 3) The processing flow of the authenticity scoring module is as follows: The known triples under the same relation are taken as the support set, and the support set semantic vector representation of the relation is obtained by averaging the semantic vector representations of each triple; Input the query triple into the above model to obtain the semantic vector representation of the query triple; Calculate the similarity semantic vector representation between the query triple and the support set, perform authenticity scoring on the query triple, and achieve few-sample knowledge graph completion.

3. The method for completing a knowledge graph using a small number of samples according to claim 2, wherein: The specific steps of the LLM-PCA semantic encoder are as follows: S1: Preprocess entity h and relation r names, remove spaces between words, and ensure that multi-word entity and relation names are processed as a single unit; S2: Input the preprocessed text into the word segmenter of the large-scale language model for word segmentation; S3: Use the word embedding layer of the large-scale language model and the multi-layer Transformer encoder to encode the sequence after word segmentation to obtain the i-th hidden layer vector S4: Extract the hidden state vectors of the first and last layers of the Transformer encoder, perform average pooling on all word units, and obtain the semantic vector representation of the text S5: Perform principal component analysis (PCA) on the semantic vector to reduce its dimension and obtain the final compact semantic representation.

4. The method for completing a knowledge graph using a small number of samples according to claim 2, wherein: The process of the attention mapping network is as follows: S1: Arrange the semantic vectors of triples into a matrix S2: Information fusion of X is performed through a multi-head self-attention network layer to obtain the output vector y = att(X); S3: Input y into the multi-layer perceptron network to obtain the final triple semantic representation vector: z=MLP(y)=W n (…(W2(W1y+b1)+b2)…)+b n 。 5. The method for completing a knowledge graph using a few samples as claimed in claim 4, wherein: The process of contrastive learning is as follows: S1: For each true triplet h, r and t, construct negative sample triplet h, r and t' by replacing the tail entity; S2: Input the positive and negative sample triplets into the above model to obtain the final semantic vector; S3: Calculate the cosine similarity between the true triples and the negative sample triples; S4: Design contrast loss function: S5: Use the Adam optimizer to perform parameter tuning on the training set.

6. The few-sample knowledge graph completion method according to claim 2, characterized in that: The process of the authenticity scoring module is as follows: S1: Take the known triples under the same relationship as the support set Sr, and get the support set semantic vector representation of the relationship by averaging the semantic vector representation of each triple. S2: Input the query triple into the above model to obtain the semantic vector representation z of the query triple q ; S3: Calculate the similarity semantic vector representation between the query triple and the support set, and perform authenticity scoring on the query triple: S4: Determine whether the triple really exists based on the authenticity score of the query triple, and add the real triple to the existing knowledge graph to achieve the purpose of knowledge graph completion.

Images