A knowledge representation learning model construction method for entity and triple joint reasoning

Abstract

The present invention belongs to the field of knowledge graph representation learning, and specifically relates to a method for constructing a knowledge representation learning model for entity and triple joint reasoning, comprising: using an existing triple dataset as a positive sample set for entity reasoning, sampling and constructing multiple triple pairs from it, and randomly assigning logical relationships to each triple pair to form a positive sample set for triple reasoning; constructing a model training sample set, each training sample including: for each positive sample in the entity reasoning positive sample set, sequentially extracting the entity reasoning positive sample and generating a corresponding negative sample, and performing the same operation on the triple reasoning positive sample set; using the training samples, optimizing the model parameters with the goal of minimizing the entity-triplet joint reasoning loss, and obtaining a knowledge representation learning model for entity and triple joint reasoning. The present invention can simultaneously implement knowledge graph representation learning for joint reasoning of entity relationships and triple relationships.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph representation learning, and more specifically, to a method for constructing a knowledge representation learning model for entity and triple joint reasoning. Background Art

[0002] A knowledge graph is a summary and abstract representation of real-world objects. It abstracts real-world objects or concepts into entities, or points in the knowledge graph, and abstracts the connections between objects into lines, or edges, between points in the knowledge graph. It often uses triples consisting of a head entity, a tail entity, and a relationship to describe, represent, and store the relationships between entities. The overlap between entities in different triples allows the knowledge graph to form a mesh structure, providing a visual representation of the knowledge graph. The nodes in the mesh represent entities, and the edges connecting the nodes represent the connections between entities. This knowledge graph, stored in a primitive negative format, relies on computationally complex graph algorithms due to its unique graph structure. Furthermore, some points in the knowledge graph have too few edges connecting them, resulting in a weak connection to the knowledge graph and significant data sparsity. Therefore, the information stored in the knowledge graph is difficult to use.

[0003] To address the aforementioned issues, knowledge representation learning (KRL), based on the concept of distributed representation learning, has emerged as a trendsetter. It cleverly uses vectors in a real-valued semantic space to represent entities and relationships in knowledge graphs, using distance to represent the similarity between two entities. This allows for the representation of relationships between entities through vector calculations. After years of research, the mainstream research direction of KRL has emerged: based on triple structure information and the fusion of multi-source information.

[0004] However, these methods focus solely on the relationship between the head and tail entities of a triple, as well as external information related to the head and tail entities, while ignoring the rich logical information between triples. Utilizing as much logical relationship information between triples as possible within a knowledge base for effective knowledge representation learning is crucial for tasks such as knowledge graph-based search, personalized recommendations, intelligent question answering, knowledge graph completion, knowledge acquisition, fusion, and reasoning. Summary of the Invention

[0005] In response to the defects of the existing technology and the need for improvement, the present invention provides a method for constructing a knowledge representation learning model for joint reasoning of entities and triples, which aims to simultaneously realize knowledge graph representation learning for joint reasoning of entity relationships and triple relationships.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for constructing a knowledge representation learning model for entity and triple joint reasoning is provided, comprising:

[0007] The existing triple data set is used as a positive sample set for entity reasoning, and multiple triple pairs are sampled and constructed from it in a random sampling manner. A logical relationship is randomly assigned to each triple pair to form a positive sample set for triple reasoning;

[0008] Constructing a model training sample set, each training sample comprising: an entity reasoning positive sample obtained by randomly sampling from the entity reasoning positive sample set, a negative sample constructed by processing the entity reasoning positive sample, a triple reasoning positive sample obtained by randomly sampling from the triple reasoning positive sample set, and a negative sample constructed by processing the triple reasoning positive sample;

[0009] Each training sample is initialized with uniform distribution. Using all the initialized training samples, the parameters in the entity-triplet joint representation learning model are optimized with the goal of minimizing the entity-triplet joint reasoning loss, and a knowledge graph representation learning model for entity and triplet joint reasoning is obtained.

[0010] Furthermore, the negative samples for entity reasoning are generated as follows:

[0011] The head entity or tail entity of each triple (h, r, t) in the entity reasoning positive sample set is modified, where the probability of selecting the modified head entity is P, and the probability of selecting the modified tail entity is 1-P. The modified triple is expressed as (h′, r′, t′). If the modified triple does not appear in the entity reasoning positive sample set, (h′, r′, t′) is used as an entity reasoning negative sample.

[0012] Furthermore, the generation method of triplet inference negative samples is as follows:

[0013] For each triple inference sample (tri h , tri r , tri t ) is modified, where the probability of selecting the modified head triplet is P, and the probability of selecting the modified tail triplet is 1-P. The modified triplet reasoning sample is represented as (tri′ h , tri′ r , tri′ t ), if the modified triple inference sample does not appear in the triple inference positive sample set, then (tri′ h , tri′ r , tri′ t ) as a triplet inference negative sample.

[0014] Furthermore, the loss function corresponding to the entity-triplet joint reasoning loss is defined as:

[0015]

[0016] L=λl1+(1-λ)l2;

[0017] f(α,β,γ)=|α+β-γ|;

[0018] Wherein, L1 represents the loss function of the entity reasoning part, L2 represents the loss function of the triple reasoning part, L represents the loss function of the entity and triple joint reasoning model, f is a function that measures the distance between the entity vector and the relationship vector or the triple vector and the relationship vector, α, β, γ represent different vectors, the value ranges of α and γ are both entity vectors and triple vectors, and the value range of β is entity relationship vectors and triple relationship vectors; S represents the entity reasoning positive sample set, S′ represents the entity reasoning negative sample set, Tri represents the triple reasoning positive sample set, and Tri′ represents the triple reasoning negative sample set; [x] + Indicates that the x part takes a non-negative value; γ1 is a hyperparameter that controls the separation distance of the positive and negative sample representation vectors of entity reasoning, and γ2 is a hyperparameter that controls the separation distance of the positive and negative sample representation vectors of triple reasoning.

[0019] Furthermore, the entity-triplet joint representation learning model adopts the TransE, TransR or RotatE network framework.

[0020] The present invention also provides a knowledge representation learning method for entity and triple joint reasoning, which uses the knowledge representation learning model constructed by the above-mentioned knowledge representation learning model construction method for entity and triple joint reasoning to perform representation learning on entity reasoning facts-triplet reasoning facts.

[0021] The present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the above-mentioned entity-oriented, triple-joint reasoning knowledge representation learning method and / or the above-mentioned entity-oriented, triple-joint reasoning knowledge representation learning method.

[0022] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0023] (1) The present invention is a knowledge graph representation learning method that can simultaneously perform joint reasoning on entity relationships and triple relationships. First, multiple triple reasoning data sets are created, and the logical information between triples in the knowledge base is innovatively and fully utilized. By learning the logical relationships implied between triples, a vector representation of the triples is obtained, and the joint reasoning of entity relationships and triple relationships is realized, the expressive power of the knowledge representation learning model is improved, and a vector representation of higher quality is obtained.

[0024] (2) The present invention proposes an entity-triplet joint reasoning loss, which enables the representation learning model to fully utilize the implicit information between triplets and realize the joint reasoning of the relationship between entities and triplets. This modeling method fully utilizes the information of different dimensions and granularities in the knowledge graph, enabling the representation learning model to learn vector representations with stronger expressive power, which can better serve the downstream tasks of knowledge representation learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart of a method for constructing a knowledge representation learning model for entity and triple joint reasoning provided by an embodiment of the present invention;

[0026] Figure 2 Schematic diagram of the application scenario of the triple joint reasoning part provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0028] Example 1

[0029] A knowledge representation learning model construction method for entity and triple joint reasoning, such as Figure 1 Shown, including:

[0030] S1. Use the existing triple dataset as the entity reasoning positive sample set, and randomly sample and construct multiple triple pairs from it. For each triple pair, randomly assign one of the preset logical relationships to form the triple reasoning positive sample set.

[0031] S2. Construct a model training sample set, where each training sample consists of: an entity reasoning positive sample obtained by randomly sampling from the entity reasoning positive sample set, a negative sample constructed by processing the entity reasoning positive sample, a triple reasoning positive sample obtained by randomly sampling from the triple reasoning positive sample set, and a negative sample constructed by processing the triple reasoning positive sample;

[0032] S3. Initialize each training sample with uniform distribution; use all the initialized training samples to minimize the entity-triplet joint reasoning loss, optimize the parameters in the entity-triplet joint representation learning model, and obtain a knowledge graph representation learning model for entity and triplet joint reasoning.

[0033] It should be noted that before using training samples to train the model, the dataset's entity vectors, entity relationship vectors, triple vectors, and triple relationship vectors must be initialized. The parameters involved in the loss function of the entity-triplet joint reasoning model (entity vectors, entity relationship vectors, triple vectors, and triple relationship vectors) are updated using a stochastic gradient descent algorithm. After continuous iterative updates, a knowledge graph representation learning model for entity-triplet joint reasoning is ultimately obtained. Furthermore, each triple pair and its logical relationship in the triple reasoning positive sample set constitutes a triple reasoning positive sample.

[0034] As a specific implementation, the aforementioned existing triple datasets can be the public datasets FB15K or WN18. Based on observations of the triples provided in these training sets (i.e., triple factual information), the present invention discovered that a large number of triple pairs within the training sets exhibit causal relationships. Previous knowledge representation learning models have failed to recognize and utilize this information, which could provide valuable insights for knowledge representation learning. Therefore, corresponding triple inference datasets with causal relationships are generated for the two original datasets mentioned above.

[0035] The following are examples of triples with logical relationships in the training set:

[0036] The entity reasoning dataset contains a large number of triples, such as Figure 2As shown in the figure, there is a causal logical relationship between the two triplets (director, filming, Movie 2) and (director, winning the Best Director Award): (director, filming, Movie 2) causes (director, winning the Best Director Award). Entity reasoning datasets contain a large number of triplets with certain logical relationships, which can be used to form a triple reasoning dataset for triple reasoning. Knowledge representation learning based on triple reasoning breaks the limitations of previous knowledge representation learning methods and utilizes the coarser-grained information in the original dataset. This fully utilizes the multi-dimensional and multi-granular information in the knowledge base.

[0037] For the publicly available datasets FB15K and WN18, corresponding triple reasoning datasets with causal relationships are generated: MT-FB15k-206, MT-FB15k-106, MT-FB15k-56, MT-FB15k-2018, MT-WN18-56, MT-WN18-106, MT-WN18-36, MT-WN18-518, ST-FB15k-201, ST-FB15k-10, ST-FB15k-5, ST-FB15k-202, ST-WN18-51, ST-WN18-10, ST-WN18-3, ST-WN18-52, a total of 16 datasets, and their specific information is shown in the figure below:

[0038]

[0039]

[0040] The following uses FB15K as an example to illustrate how to generate the corresponding triple inference dataset:

[0041] FB15K is a knowledge base that stores a large number of entities and relationships in the form of triples, where each triple represents a fact. The training set of FB15K is a set of facts, and each piece of data is a triple. Random sampling is performed on the 480,000 triples in the training set of FB15K to generate 200,000 initial triple pairs δ, which are then deduplicated to obtain δ*. Subsequently, 80% of the data in δ* is randomly selected as the training set, 10% of the data as the validation set, and 10% of the data as the test set. All triples involved in δ* are further deduplicated, and the triple set T of the dataset is obtained after deduplication. The same steps are used to generate the corresponding triple inference dataset for WN18.

[0042] As a specific implementation plan, each training sample is initialized with uniform distribution, specifically: for each entity vector representation and entity relationship vector representation in the original training set FB15K or WN18, use The uniform distribution is used for initialization, and the triple vector representation and triple relationship vector representation of the simulation data set generated in S1 (i.e., the triple reasoning positive sample set) are also used. The negative samples are initialized with a uniform distribution. The negative samples are initialized in the same way. Other uniform distributions can also be used for initialization, which is not limited by the present invention.

[0043] Preferably, the negative samples for entity reasoning are generated as follows:

[0044] The head entity or tail entity of each triple (h, r, t) in the entity reasoning positive sample set is modified, where the probability of selecting the modified head entity is P, and the probability of selecting the modified tail entity is 1-P. The modified triple is expressed as (h′, r′, t′). If the modified triple does not appear in the entity reasoning positive sample set, (h′, r′, t′) is used as an entity reasoning negative sample.

[0045] Preferably, the negative sample for triple inference is generated as follows:

[0046] For each triple inference sample (tri h , tri r , tri t ) is modified, where the probability of selecting the modified head triplet is P, and the probability of selecting the modified tail triplet is 1-P. The modified triplet reasoning sample is represented as (tri′ h , tri′ r , tri′ t ), if the modified triple reasoning sample does not appear in the triple reasoning positive sample set, then (tri′ h , tri′ r , tri′ t ) as a triplet inference negative sample. h ∈T.

[0047] Preferably, the loss function corresponding to the entity-triplet joint reasoning loss is defined as:

[0048]

[0049] L=λl1+(1-λ)l2;

[0050] f(α,β,γ)=|α+β-γ|;

[0051] Where l1 represents the loss function for entity reasoning, l2 represents the loss function for triple reasoning, L represents the loss function for the joint entity and triple reasoning model, α, β, and γ represent the head entity vector (head triple vector), entity relationship vector (triplet relationship vector), and tail entity vector (tail triple vector), respectively. α and γ both range from entity vectors to triple vectors, while β ranges from entity relationship vectors to triple relationship vectors. Specifically, the head entity vector and the tail entity vector are both entity vectors, but their positions within a triple are different, so they are referred to as the head entity vector and the tail entity vector. The same distinction is also used for the head triple vector and the tail triple vector. γ1 is a hyperparameter that controls the separation distance between the positive and negative sample representation vectors for entity reasoning, and γ2 is a hyperparameter that controls the separation distance between the positive and negative sample representation vectors for triple reasoning. S is the set of positive examples for entity reasoning, S′ is the set of negative examples for entity reasoning, Tri is the set of positive examples for triple reasoning, and Tri′ is the set of negative examples for triple reasoning. [x] + Indicates that the x part takes a non-negative value.

[0052] Substitute (h, r, t) and (h′, r′, t′) into formula l1 to calculate the loss of the entity reasoning part, where the function f is used to calculate the distance between (h, r, t) and (h′, r′, t′) respectively, and (tri h , tri r , tri t ) and (tri′ h , tri′ r tri′ t ) is brought into formula l2 to calculate the loss of the triple reasoning part, where the function f is used to calculate (tri h , tri r , tri t ) and (tri′ h , tri′ r , tri′ t ), and then bring the two parts of loss into formula 1 to obtain the weighted sum of the total loss of the entity part and the triple part, that is, the loss of the entity and triple joint reasoning model designed by the present invention.

[0053] Preferably, the above entity-triplet joint representation learning model can not only improve the TransE model, but also be applied to models such as TransR and RotatE in the form of a plug-in.

[0054] This embodiment incorporates the idea of ​​entity and triple joint reasoning into the modification of its model, adds a triple reasoning module, and merges the original entity reasoning module with the newly added triple reasoning module.

[0055] Example 2

[0056] A knowledge representation learning method for entity and triple joint reasoning uses a knowledge representation learning model constructed by a knowledge representation learning model construction method for entity and triple joint reasoning as described in Example 1 above to perform representation learning on entity reasoning facts-triplet reasoning facts.

[0057] The relevant technical solutions are the same as those in Example 1 and will not be described again here.

[0058] Existing knowledge graph representation learning models all model the relationship between head and tail entities at the triple level. This fine-grained modeling approach is too limited in scope, using only the structural information of the triple itself and ignoring the potential relationship between triples. The present invention first opens up a new idea for knowledge representation learning: using triple reasoning for knowledge representation learning, and secondly provides a method for constructing a data set for triple reasoning. Finally, a knowledge graph representation learning method for entity and triple joint reasoning is provided. Without the need to introduce external information, it fully utilizes information of different dimensions and granularities in the knowledge base to learn a more expressive vector representation, which can better serve the downstream tasks of knowledge representation learning.

[0059] Example 3

[0060] A computer-readable storage medium, comprising a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute a knowledge representation learning method for entity-oriented and triple joint reasoning as described in the first embodiment above and / or a knowledge representation learning method for entity-oriented and triple joint reasoning as described in the second embodiment above.

[0061] The relevant technical solutions are the same as those in Example 1 and will not be described again here.

[0062] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a knowledge representation learning model for entity and triple joint reasoning, characterized by: include: The existing triple data set is used as a positive sample set for entity reasoning, and multiple triple pairs are sampled and constructed from it in a random sampling manner. A logical relationship is randomly assigned to each triple pair to form a positive sample set for triple reasoning; Constructing a model training sample set, each training sample comprising: an entity reasoning positive sample obtained by randomly sampling from the entity reasoning positive sample set, a negative sample constructed by processing the entity reasoning positive sample, a triple reasoning positive sample obtained by randomly sampling from the triple reasoning positive sample set, and a negative sample constructed by processing the triple reasoning positive sample; Each training sample is initialized with a uniform distribution. Using all the initialized training samples, the parameters of the entity-triplet joint representation learning model are optimized with the goal of minimizing the entity-triplet joint reasoning loss, resulting in a knowledge graph representation learning model for entity and triplet joint reasoning. Among them, the generation method of entity reasoning negative samples is: Modify the head entity or tail entity of each triple (h, r, r) in the entity reasoning positive sample set, where the probability of selecting the modified head entity is P, and the probability of selecting the modified tail entity is 1-P. The modified triple is represented as (h ′ ,r ′ ,t ′ ), if the modified triple does not appear in the entity reasoning positive sample set, then (h ′ ,r ′ ,t ′ ) as a negative sample for entity reasoning; The generation method of triplet inference negative samples is: For each triple inference sample (tri h ,tri r ,tri t ) is modified, where the probability of selecting the modified head triplet is P, and the probability of selecting the modified tail triplet is 1-P. The modified triplet reasoning sample is represented as (tri′ h ,tri′ r ,tri′ t ), if the modified triple inference sample does not appear in the triple inference positive sample set, then (tri′ h ,tri′ r ,tri′ t ) as a triplet inference negative sample; The triple entity is text.

2. The method for constructing a knowledge representation learning model for entity and triple joint reasoning according to claim 1, characterized in that: The loss function corresponding to the entity-triplet joint reasoning loss is defined as: l1=∑ (h,r,t)∈S ∑ (h′,r′,t′)∈S′ [γ1+f(h,r,t)-f(h′,r′,t′)] + ; L=λl1+(1-λ)l2; f(α,β,γ)=|α+β-γ|; Wherein, L1 represents the loss function of the entity reasoning part, L2 represents the loss function of the triple reasoning part, L represents the loss function of the entity and triple joint reasoning model, f is a function that measures the distance between the entity vector and the relationship vector or the triple vector and the relationship vector, α, β, γ represent different vectors, the value ranges of α and γ are both entity vectors and triple vectors, and the value range of β is entity relationship vectors and triple relationship vectors; S represents the positive sample set of entity reasoning, S′ represents the negative sample set of entity reasoning, Tri represents the positive sample set of triple reasoning, and Tri′ represents the negative sample set of triple reasoning; [x] + Indicates that the x part takes a non-negative value; γ1 is a hyperparameter that controls the separation distance of the positive and negative sample representation vectors of entity reasoning, and γ2 is a hyperparameter that controls the separation distance of the positive and negative sample representation vectors of triple reasoning.

3. The method for constructing a knowledge representation learning model for entity and triple joint reasoning according to claim 1, characterized in that: The entity-triplet joint representation learning model adopts the TransE, TransR or RotatE network framework.

4. A knowledge representation learning method for entity and triple joint reasoning, characterized by: A knowledge representation learning model constructed by a knowledge representation learning model construction method for entity and triple joint reasoning as described in any one of claims 1 to 3 is used to perform representation learning on entity reasoning facts-triple reasoning facts.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by the processor, the device where the storage medium is located is controlled to execute a knowledge representation learning model construction method for entity-oriented and triple joint reasoning as described in any one of claims 1 to 3 and / or a knowledge representation learning method for entity-oriented and triple joint reasoning as described in claim 4.

Images