A knowledge graph completion method based on pre-trained models and graph neural networks.

Abstract

This invention provides a graph neural network knowledge graph completion method based on a pre-trained model, characterized by the following steps: Step S1, obtaining the natural language representations corresponding to each described entity and relation; Step S2, obtaining the natural language representations corresponding to each undescribed entity based on all the natural language representations corresponding to described entities; Step S3, training a knowledge graph completion model, along with the representations corresponding to each relation and the initial semantic representations corresponding to each undescribed entity; Step S4, inputting the triples, the natural language representations corresponding to each described entity, and the representations corresponding to the relations in the triples into the trained knowledge graph completion model to obtain the missing entities. In summary, this method can improve the completion effect of knowledge graphs.

Description

Technical Field

[0001] This invention relates to a knowledge graph completion method, specifically a graph neural network knowledge graph completion method based on a pre-trained model. Background Technology

[0002] Knowledge graphs, based on knowledge associations, can depict the relationships and patterns between things from multiple perspectives and levels. However, current real-world knowledge graphs are often incomplete, with many missing connections. On the other hand, new knowledge is constantly emerging, and how to integrate new knowledge into existing knowledge graphs is also a problem that needs to be solved. Knowledge Graph Completion (KGC) is used to update and expand knowledge graphs, addressing the aforementioned issues of incomplete graphs and the integration of new knowledge.

[0003] While various knowledge graph completion techniques have achieved some success, many challenges remain, such as the long-tail data problem and poor model generalization. The long-tail data phenomenon refers to situations where some entities and relations appear frequently but are few in number, while others appear infrequently but are numerous, with few triple instances containing these entities and relations. This severely impacts the knowledge graph completion process, and this effect becomes more pronounced with increasing data volume. Regarding the generalization problem of knowledge graph completion models, most current models, such as translation and tensor decomposition models, typically only complete static knowledge graph content. These models rely solely on existing triple data during training and cannot infer information about unknown triples. This static completion approach struggles to adapt to changes in the external environment, resulting in poor model generalization. In the real world, new knowledge emerges constantly, and old knowledge may need to be updated or corrected over time. Although models based on neural networks and graph neural networks can process newly added information, they mostly use random initialization to process it, which makes it difficult to achieve the desired effect.

[0004] In summary, there is still considerable room for improvement in the completion effect of existing knowledge graph completion methods. Summary of the Invention

[0005] This invention is made to solve the above problems, and aims to provide a graph neural network knowledge graph completion method based on a pre-trained model.

[0006] This invention provides a graph neural network knowledge graph completion method based on a pre-trained model, used to complete triples of missing entities in a knowledge graph. The entities in the knowledge graph include both descriptive and undescriptive entities, and have the following characteristics: Step S1, inputting all descriptive entities and relations in the knowledge graph into an existing pre-trained language model to obtain the natural language representations corresponding to each descriptive entity and relation; Step S2, based on the natural language representations corresponding to all descriptive entities, obtaining the natural language representations corresponding to each undescriptive entity in the knowledge graph through a multivariate Gaussian distribution; Step S3, constructing a knowledge graph completion model, and based on the knowledge graph, combining the natural language representations corresponding to all descriptive entities, relations, and undescriptive entities, completing the knowledge graph... The entire model is trained to obtain a trained knowledge graph completion model, as well as the representations corresponding to each relation and the initial semantic representations corresponding to each entity without description. In step S4, the triples, the natural language representations corresponding to each entity with description, and the representations corresponding to the relations in the triples are input into the trained knowledge graph completion model to obtain the missing entities. The knowledge graph completion model includes: a graph representation generation module, which generates corresponding graph representations for each natural language representation with description and for each initial semantic representation without description; and a score calculation module, which calculates the predicted scores for the graph representations corresponding to the existing entities in the triples, the representations corresponding to the relations, and the graph representations corresponding to each entity, and takes the entity with the highest predicted score as the missing entity.

[0007] The graph neural network knowledge graph completion method based on a pre-trained model provided by this invention may also have the following feature: wherein, in step S2, the expression for the natural language representation without descriptive entities obtained through multivariate Gaussian distribution is: In the formula For the set of all undescribed entities, For set The natural language representation of the h-th undescribed entity in the dataset, μ E σ is the mean of all natural language representations that describe entities. E Let E be the variance of all natural language representations with descriptive entities, E be the set of all entities with descriptive entities, |E| be the total number of entities in set E, and e be the variance of all natural language representations with descriptive entities. h Let h be the natural language representation of the h-th descriptive entity in set E.

[0008] The graph neural network knowledge graph completion method based on a pre-trained model provided in this invention may also have the following feature: the graph representation generation module includes R-GCN, and in the l-th layer of R-GCN, for the i-th node, its propagation mechanism is as follows: In the formula Let c be the set of neighboring nodes of the i-th node under relation r. i,r As a preset constant, Let be the trainable parameter matrix of the l-th layer for relation r. The trainable parameter matrix for node self-loops. This represents the hidden state of the i-th node at level l. For activation function, For the i-th node, which corresponds to either a natural language representation of the entity with description or an initial semantic representation without description, the output of the i-th node in the last layer of R-GCN is... This is the graph representation of the entity corresponding to this node.

[0009] The graph neural network knowledge graph completion method based on a pre-trained model provided by this invention may also have the following feature: the score calculation module calculates the predicted score through the scoring function of DistMult.

[0010] The graph neural network knowledge graph completion method based on a pre-trained model provided by this invention may also have the following features: Step S3 includes the following sub-steps: Step S3-1, inputting the natural language representations corresponding to each relation into the graph representation generation module to generate corresponding training graph representations; Step S3-2, using the natural language representations corresponding to each described entity as the initial training semantic representations corresponding to that described entity, and using the natural language representations corresponding to each undescribed entity as the initial training semantic representations corresponding to that undescribed entity; Step S3-3, inputting the initial training semantic representations corresponding to each entity into the graph representation generation module to generate corresponding training graph representations; Step S3-4, randomly sampling a batch of triples from the knowledge graph to construct... Step S3-5: Calculate the loss result based on the batch training data and the corresponding training graph representation, combined with the loss function. Step S3-6: Optimize the trainable parameter matrix of the graph representation generation module, the training graph representation corresponding to each relation, and the initial semantic representation of the undescribed entity based on the loss result. Step S3-7: Repeat steps S3-3 to S3-6 until the preset termination condition is met. Then, the knowledge graph completion model with the optimized graph representation generation module is used as the trained knowledge graph completion model, the optimized training graph representation corresponding to the relation is used as the representation corresponding to that relation, and the optimized initial semantic representation of the undescribed entity is used as the initial semantic representation of the undescribed entity.

[0011] The graph neural network knowledge graph completion method based on a pre-trained model provided in this invention may also have the following features: The batch training data includes B triples existing in the knowledge graph as positive sample triples, and multiple negative sample triples constructed from the positive sample triples using an in-batch negative sampling method. The negative sample triples include tail entity replacement triples, head entity replacement triples, head self-replacement triples, and tail self-replacement triples. The tail entity replacement triples are formed by replacing the tail entity in each positive sample triple with the tail entity from another positive sample triple, and removing the tail entity from each positive sample triple. The triplet obtained after the tuple is as follows: Head entity replacement triplet is obtained by replacing the head entity in each positive sample triplet with the head entity in another positive sample triplet, and then removing the positive sample triplet. Head self-replacement triplet is obtained by replacing the tail entity in each positive sample triplet with the head entity in that positive sample triplet, and then removing the positive sample triplet. Tail self-replacement triplet is obtained by replacing the head entity in each positive sample triplet with the tail entity in that positive sample triplet, and then removing the positive sample triplet.

[0012] The graph neural network knowledge graph completion method based on a pre-trained model provided in this invention may also have the following feature: where the expression for the loss function is:

[0013] In the formula Let λ1 and λ2 be the loss function, and λ1 and λ2 be the weight coefficients of the corresponding terms, respectively. For the expectation, For triples (h) i ,r i ,t i The predicted score of ) For batch training data The sum of the predicted scores of all negative triples corresponding to the i-th positive triple in the dataset after exponential activation, where ε is the set of all entities, and x... h Let h be the training graph representation of entity h in set ε. Let x be the set of all relations. r For set The training graph representation corresponding to relation r is given by E, where E is the set of all entities with descriptions. Let e ​​be the training graph representation of entity h in set E. h For the natural language representation of entity h, e r Let r be the natural language representation of relation r.

[0014] The role and effect of invention

[0015] According to the graph neural network knowledge graph completion method based on a pre-trained model of the present invention, on the one hand, by constructing tail-self-substitution triples and head-self-substitution triples as part of the negative samples, the adverse effects of entity asymmetry in the knowledge graph distribution on the training of the knowledge graph completion model are reduced, thereby improving the performance of the knowledge graph completion model. On the other hand, by generating deep language representation information (i.e., natural language representation) for each entity with description through a pre-trained language model, and using a multivariate Gaussian distribution to generate natural language representations without description through the natural language representations with description, and then continuously optimizing the natural language representations without description and the representation of relations through training, high-quality representations of each entity without description and relation are obtained. Therefore, the graph neural network knowledge graph completion method based on a pre-trained model of the present invention can improve the completion effect of the knowledge graph. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the graph neural network knowledge graph completion method based on a pre-trained model in an embodiment of the present invention.

[0017] Figure 2 This is a block diagram of the knowledge graph completion model in an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram illustrating the training principle of the knowledge graph completion model in an embodiment of the present invention. Detailed Implementation

[0019] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the graph neural network knowledge graph completion method based on the pre-trained model of this invention.

[0020] The graph neural network knowledge graph completion method based on the pre-trained model in this embodiment is used to complete the triples of missing entities in the knowledge graph.

[0021] This knowledge graph can be represented as... In the formula, ε represents the set of entities. A set representing relations. Represents the set of triples, and For a triple h represents the head entity, r represents the relation, and t represents the tail entity. Another set of descriptions is given. For some entities h∈ε or relations exist and Where dh and d r Each represents its natural language description, and any w i and v i All are vocabulary lists The words in the knowledge graph. The contents can be described by the set The entities to be described include descriptive entities and descriptive relationships, and also include sets that cannot be described. Entities that are described are undescribed entities, and relationships that are undescribed relationships are undescribed relationships. This embodiment uses a knowledge graph. All relations are treated as descriptive relations. The treatment of undescriptive relations is the same as that of undescriptive entities in the following text, and therefore will not be repeated.

[0022] Triple completion uses information from an existing knowledge graph to infer missing facts. For example, given the head entity and relation, it predicts the tail entity (h, r, ?), or given the tail entity and relation, it predicts the head entity (?, r, t). Essentially, this is a binary classification problem involving triples, i.e., predicting whether a triple exists.

[0023] Figure 1 This is a flowchart illustrating the graph neural network knowledge graph completion method based on a pre-trained model in an embodiment of the present invention.

[0024] like Figure 1 As shown, the graph neural network knowledge graph completion method based on a pre-trained model includes the following steps:

[0025] Step S1: Input all descriptive entities and relations in the knowledge graph into the existing pre-trained language model to obtain the natural language representations corresponding to each descriptive entity and relation. In this embodiment, the existing pre-trained language model is BERT.

[0026] Step S2: Based on the natural language representations corresponding to all entities with descriptions, obtain the natural language representations corresponding to each entity without descriptions in the knowledge graph through a multivariate Gaussian distribution.

[0027] The expression for the natural language representation of the undescribed entity obtained through multivariate Gaussian distribution is as follows:

[0028]

[0029] In the formula For the set of all undescribed entities, For set The natural language representation of the h-th undescribed entity in the dataset, μ E σ is the mean of all natural language representations that describe entities. ELet E be the variance of all natural language representations with descriptive entities, E be the set of all entities with descriptive entities, |E| be the total number of entities in set E, and e be the variance of all natural language representations with descriptive entities. h Let h be the natural language representation of the h-th descriptive entity in set E.

[0030] Step S3: Construct a knowledge graph completion model, and train the knowledge graph completion model based on the knowledge graph and the natural language representations corresponding to all entities with descriptions, relations and entities without descriptions, to obtain the trained knowledge graph completion model, the representations corresponding to each relation and the initial semantic representations corresponding to each entity without descriptions.

[0031] Figure 2 This is a block diagram of the knowledge graph completion model in an embodiment of the present invention.

[0032] like Figure 2 As shown, the knowledge graph completion model 100 includes a graph representation generation module 10 and a score calculation module 20.

[0033] The graph representation generation module 10 is used to generate corresponding graph representations for each natural language representation with a descriptive entity, and to generate corresponding graph representations for each initial semantic representation without a descriptive entity.

[0034] The graph representation generation module 10 includes an R-GCN. In the l-th layer of the R-GCN, for the i-th node, the propagation mechanism is as follows:

[0035]

[0036] In the formula Let c be the set of neighboring nodes of the i-th node under relation r. i,r As a preset constant, Let be the trainable parameter matrix of the l-th layer for relation r. The trainable parameter matrix for node self-loops. This represents the hidden state of the i-th node at level l. This is the activation function.

[0037] This represents the natural language representation of the entity corresponding to the i-th node, or the initial semantic representation of the entity without a description. The output of the i-th node in the last layer of R-GCN... This is the graph representation of the entity corresponding to the node. In this embodiment, in training the knowledge graph completion model 100, the graph representation generation module 10 is also used to generate a training graph representation corresponding to the relation based on the natural language representation corresponding to the relation, and to generate a training graph representation corresponding to the entity without description based on the natural language representation corresponding to the entity without description, i.e. If there is no natural language representation describing the entity or relation, then the output of the i-th node in the last layer of R-GCN is... It also provides a training graph representation for the entity or relationship corresponding to the node.

[0038] The score calculation module 20 is used to calculate the predicted score by combining the graph representations corresponding to the existing entities and the relation representations in the triplet with the graph representations corresponding to each entity, and to take the entity with the highest predicted score as the missing entity.

[0039] The score calculation module 20 calculates the predicted score using the DistMult scoring function. In this embodiment, the DistMult scoring function is used to calculate the predicted score of the triples formed by the existing entities and relations in the triples and the triples formed by each entity. The scoring function gives a higher predicted score to the more closely matched triples. Therefore, the entity in the triple corresponding to the highest predicted score can be selected as the entity that best matches the triple, i.e., the missing entity.

[0040] Figure 3 This is a schematic diagram illustrating the training principle of the knowledge graph completion model in an embodiment of the present invention.

[0041] like Figure 3 As shown, a training dataset is constructed based on the knowledge graph, and batch training data is constructed based on the training dataset, combined with a loss function. The knowledge graph completion model 100 was trained.

[0042] Therefore, step S3 includes the following sub-steps:

[0043] Step S3-1: Input the natural language representations corresponding to each relation into the graph representation generation module to generate the corresponding training graph representations.

[0044] Step S3-2: Use the natural language representation corresponding to each entity with description as the initial semantic representation for training that entity, and use the natural language representation corresponding to each entity without description as the initial semantic representation for training that entity.

[0045] Step S3-3: Input the initial semantic representation of each entity into the graph representation generation module to generate the corresponding training graph representation.

[0046] Steps S3-4 involve randomly sampling a batch of triples from the knowledge graph to construct batch training data.

[0047] The batch training data includes B triples existing in the knowledge graph as positive sample triples, and multiple negative sample triples constructed from the positive sample triples using an in-batch negative sampling method. The negative sample triples include tail entity substitution triples, head entity substitution triples, head self-substitution triples, and tail self-substitution triples.

[0048] Tail entity replacement triples are obtained by replacing the tail entity in each positive sample triplet with the tail entity from another positive sample triplet, and then removing the positive sample triplets. Therefore, for a batch of training data, the matrix S representing the predicted scores of the tail entity replacement triplets is... T for:

[0049]

[0050] In the formula X H ,X R ,X T ∈R N×D Let S represent the head entity, relation, and tail entity in this batch of training data, respectively, and ⊙ represent the Haldman product. That is, the above process is equivalent to binding the head entity and relation (i.e., performing the Haldman product) and then calculating the dot product with different tail entities. Therefore, matrix S... T Only include (h) i ,r i ,t j ), i,j=1,…,B of the fractions.

[0051] The head entity replacement triplet is obtained by replacing the head entity in each positive sample triplet with the head entity from another positive sample triplet, and then removing the positive sample triplet. The resulting triplet then contains the matrix S of predicted scores corresponding to the head entity replacement triplet. H for:

[0052]

[0053] Due to the scoring function For three-vector dot product operations, higher scores are usually obtained when the head and tail entities are the same. Furthermore, because the distribution of entities in some knowledge graphs is asymmetrical—for example, in a financial equity knowledge graph, there are many triples containing relationships such as region, industry, and shareholder type—the tail entities of these triples only appear at the end of the triples. Therefore, head-self-substitution triples and tail-self-substitution triples were added.

[0054] The head-self-replacement triplet is obtained by replacing the tail entity in each positive sample triplet with the head entity from that positive sample triplet, and then removing the positive sample triplet. The matrix S corresponding to the predicted scores of the head-self-replacement triplet is then...h_self for:

[0055]

[0056] In the formula, 1∈1 D Given a D-dimensional column vector, used to sum the rows of the matrix after the Hadman product to obtain a fraction.

[0057] Tail-self-replacement triplets are triplets obtained by replacing the head entity in each positive triplet with the tail entity from the previous positive triplet, and then removing the previous positive triplet. The matrix S corresponding to the predicted scores of the tail-self-replacement triplet is then... t_self for:

[0058]

[0059] In the formula, 1∈1 D Represent a D-dimensional column vector used to sum the rows of the matrix after the Hadman product to obtain a fraction.

[0060] In this embodiment, the above four matrices are concatenated to obtain the complete fractional matrix S, whose expression is:

[0061]

[0062] Because different triples within a batch of training data may have the same head or tail entities, this could lead to positive samples appearing in the off-diagonal negative sample positions of these constructs. To ensure the proper functioning of contrastive learning, only diag(S) is retained. T The elements of diag(S) are used as the only positive sample targets, ignoring diag(S). H Possible positive samples (i.e., false negatives) and other off-diagonal elements are considered to prevent duplicate calculations, while the remaining positions are the actual negative samples that need to be compared with the positive samples. To achieve this process, filtering is performed within each batch, and the scores of the false negative samples are manually set to small negative numbers, such as -10. 8 Thus, after softmax activation, its contribution to the overall probability calculation approaches 0.

[0063] Step S3-5: Based on the batch training data and the corresponding training graph representation, and combined with the loss function, calculate the loss result.

[0064] Among them, the loss function The expression is:

[0065]

[0066]

[0067] In the formula Let λ1 and λ2 be the loss function, and λ1 and λ2 be the weight coefficients of the corresponding terms, respectively. For the expectation, For triples (h) i ,r i ,t i The predicted score of ) For batch training data The sum of the predicted scores of all negative triples corresponding to the i-th positive triple in the dataset after exponential activation, where ε is the set of all entities, and x... h Let h be the training graph representation of entity h in set ε. Let x be the set of all relations. r For set The training graph representation corresponding to relation r is given by E, where E is the set of all entities with descriptions. Let e ​​be the training graph representation of entity h in set E. h For the natural language representation of entity h, e r Let r be the natural language representation of relation r.

[0068] Steps S3-6: Based on the loss results, optimize the trainable parameter matrix of the graph representation generation module, the training graph representations corresponding to each relation, and the initial training semantic representations corresponding to entities without description.

[0069] Step S3-7: Repeat steps S3-3 to S3-6 until a preset termination condition is met. Then, the knowledge graph completion model with the optimized graph representation generation module is used as the trained knowledge graph completion model, the optimized training graph representation corresponding to the relation is used as the representation corresponding to that relation, and the optimized training initial semantic representation corresponding to the entity without description is used as the initial semantic representation corresponding to that entity without description. In this embodiment, the preset termination condition is the convergence of the knowledge graph completion model.

[0070] Step S4: Input the triples, the natural language representations corresponding to each entity, and the representations corresponding to the relations in the triples into the trained knowledge graph completion model to obtain the missing entities.

[0071] The role and effect of the embodiments

[0072] According to the graph neural network knowledge graph completion method based on a pre-trained model involved in this embodiment, on the one hand, tail-self-substitution triples and head-self-substitution triples are constructed as part of the negative samples, thereby reducing the adverse effects of entity asymmetry in the knowledge graph on the training of the knowledge graph completion model and improving the performance of the knowledge graph completion model. On the other hand, deep language representation information, i.e., natural language representation, of each entity with description is generated through a pre-trained language model, and natural language representation without description is generated from the natural language representation with description of entities using a multivariate Gaussian distribution. Then, through training, the natural language representation without description of entities and the representation of relations are continuously optimized, thereby obtaining high-quality representations of each entity without description and relation. In summary, this method can improve the completion effect of knowledge graphs.

[0073] Those skilled in the art will appreciate that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A graph neural network knowledge graph completion method based on a pre-trained model, used to complete triples of missing entities in a knowledge graph, wherein the entities in the knowledge graph include entities without description and entities with description, characterized in that, Includes the following steps: Step S1: Input all the described entities and relations in the knowledge graph into the existing pre-trained language model to obtain the natural language representations corresponding to each described entity and relation; Step S2: Based on the natural language representations corresponding to all the entities with descriptions, obtain the natural language representations corresponding to each entity without description in the knowledge graph through a multivariate Gaussian distribution; Step S3: Construct a knowledge graph completion model, and train the knowledge graph completion model based on the knowledge graph and the natural language representations corresponding to all described entities, relations and undescribed entities, to obtain a trained knowledge graph completion model, as well as the representations corresponding to each relation and the initial semantic representations corresponding to each undescribed entity. Step S4: Input the triples, the natural language representations corresponding to each described entity, and the representations corresponding to the relations in the triples into the trained knowledge graph completion model to obtain the missing entities. The knowledge graph completion model includes: The graph representation generation module is used to generate corresponding graph representations for each of the natural language representations corresponding to the described entities, and to generate corresponding graph representations for each of the initial semantic representations corresponding to the undescribed entities. The score calculation module is used to calculate predicted scores for the graph representations corresponding to the existing entities in the triples and the representations corresponding to the relations, and for the graph representations corresponding to each entity, respectively, and to select the entity with the highest predicted score as the missing entity. The expression for the loss function during training is: In the formula Let λ1 and λ2 be the weight coefficients of the corresponding terms, representing the loss function. For the expectation, For triples (h) i ,r i ,t i The predicted score of ) For batch training data The sum of the predicted scores of all negative triples corresponding to the i-th positive triple in the equation after exponential activation, where ε is the set of all the entities, and x h Let h be the training graph representation of entity h in set ε. Let x be the set of all relations. r For set The training graph representation corresponding to relation r is given by E, where E is the set of all described entities. Let e ​​be the training graph representation of entity h in set E. h For the natural language representation of entity h, e r Let r be the natural language representation of relation r.

2. The graph neural network knowledge graph completion method based on a pre-trained model according to claim 1, characterized in that: in, In step S2, the expression for the natural language representation of the undescribed entity is obtained through a multivariate Gaussian distribution as follows: In the formula The set formed by all the undescribed entities. For set The natural language representation of the h-th undescribed entity in the dataset, μ E Let σ be the mean of all the natural language representations of the described entities. E Let E be the variance of the natural language representations of all the described entities, E be the set of all the described entities, |E| be the total number of entities in set E, and e be the variance of the natural language representations of all the described entities. h Let h be the natural language representation of the h-th descriptive entity in set E.

3. The graph neural network knowledge graph completion method based on a pre-trained model according to claim 1, characterized in that: in, The graph representation generation module includes R-GCN. In the l-th layer of the R-GCN, for the i-th node, the propagation mechanism is as follows: In the formula Let c be the set of neighboring nodes of the i-th node under relation r. i,r As a preset constant, Let be the trainable parameter matrix of the l-th layer for relation r. The trainable parameter matrix for node self-loops. This represents the hidden state of the i-th node at level l. For activation function, This refers to the natural language representation of the described entity or the initial semantic representation of the undescribed entity corresponding to the i-th node. The output of the i-th node in the last layer of the R-GCN This is the graph representation corresponding to the entity corresponding to the node.

4. The graph neural network knowledge graph completion method based on a pre-trained model according to claim 1, characterized in that: in, The score calculation module calculates the predicted score using the DistMult scoring function.

5. The graph neural network knowledge graph completion method based on a pre-trained model as described in claim 1, Its features are: Step S3 includes the following sub-steps: Step S3-1: Input the natural language representations corresponding to each of the relationships into the graph representation generation module to generate the corresponding training graph representations; Step S3-2: Take the natural language representation corresponding to each described entity as the initial semantic representation for training corresponding to the described entity, and take the natural language representation corresponding to each undescribed entity as the initial semantic representation for training corresponding to the undescribed entity. Step S3-3: Input the initial semantic representation of each entity into the graph representation generation module to generate the corresponding training graph representation; Steps S3-4: Randomly sample a batch of triples from the knowledge graph to construct batch training data; Step S3-5: Based on the batch training data and the corresponding training graph representation, and combined with the loss function, calculate the loss result; Steps S3-6: Based on the loss results, optimize the trainable parameter matrix of the graph representation generation module, the training graph representation corresponding to each relation, and the initial training semantic representation corresponding to the undescribed entity; Step S3-7: Repeat steps S3-3 to S3-6 until the preset termination condition is met. Then, the knowledge graph completion model with the optimized graph representation generation module is used as the trained knowledge graph completion model, the optimized training graph representation corresponding to the relation is used as the representation corresponding to the relation, and the optimized training initial semantic representation corresponding to the undescribed entity is used as the initial semantic representation corresponding to the undescribed entity.

6. The graph neural network knowledge graph completion method based on a pre-trained model according to claim 5, characterized in that: in, The batch training data includes B triples existing in the knowledge graph as positive sample triples, and multiple negative sample triples constructed from the positive sample triples using an in-batch negative sampling method. The negative sample triples include tail entity substitution triples, head entity substitution triples, head self-substitution triples, and tail self-substitution triples. The tail entity replacement triplet is obtained by replacing the tail entity in each of the positive sample triplets with the tail entity from other positive sample triplets, and then removing the tail entity from each of the positive sample triplets. The head entity replacement triplet is obtained by replacing the head entity in each of the positive sample triplets with the head entity from other positive sample triplets, and then removing the positive sample triplets. The head-replacement triplet is obtained by replacing the tail entity in each of the positive sample triplets with the head entity from the positive sample triplet, and then removing the positive sample triplets. The tail-replacement triplet is obtained by replacing the head entity in each positive sample triplet with the tail entity in the same positive sample triplet, and then removing the positive sample triplet.

Images