Graph neural network knowledge graph completion method based on pre-training model

Abstract

The invention provides a graph neural network knowledge graph completion method based on a pre-training model, and the method is characterized in that the method comprises the steps: S1, obtaining the natural language representation corresponding to each description entity and relation; s2, according to the natural language representations corresponding to all the entities with description, natural language representations corresponding to all the entities without description are obtained; s3, training to obtain a knowledge graph completion model, expressions corresponding to all the relations and initial semantic expressions corresponding to all the non-description entities; and S4, inputting the triad, the natural language representation corresponding to each description entity and the representation corresponding to the relationship in the triad into the trained knowledge graph completion model to obtain a missing entity. In a word, the method can improve the completion effect of the knowledge graph.

Description

Technical Field

[0001] The present invention relates to a knowledge graph completion method, and in particular to a graph neural network knowledge graph completion method based on a pre-trained model. Background Art

[0002] The knowledge graph based on knowledge association can depict the related facts and laws between things in a multi-angle and multi-level way. However, the knowledge graph in the real world is often incomplete and there are a lot of missing associations. On the other hand, new knowledge emerges in an endless stream, and how to integrate new knowledge into the existing knowledge graph is also a problem that needs to be solved. Knowledge Graph Completion, or KGC, is used to update and expand the knowledge graph to solve the problems of incomplete graphs and integration of new knowledge mentioned above.

[0003] Although various methods of knowledge graph completion technology have achieved certain results, there are still many difficulties and challenges, such as the long-tail data problem and poor generalization of the model. The long-tail data phenomenon refers to the fact that some entities and relationships appear frequently, but the number of these entities and relationships is small, while others appear less frequently but in large numbers, and there are few triple instances containing these entities and relationships. This will have a serious impact on the knowledge graph completion process, and as the amount of data increases, this situation will become more significant. As for the generalization problem of the knowledge graph completion model, most of the current knowledge graph completion models, such as translation models and tensor decomposition models, usually only complete static knowledge graph content. These models only rely on existing triple data during training and cannot infer unknown triple information. This static completion method is difficult to cope with changes in the external environment, resulting in poor generalization of the model. In the real world, new knowledge emerges in an endless stream, and old knowledge may need to be updated or revised over time. Although models based on neural networks and graph neural networks can process newly added information, most of them only use random initialization to process it, and it is difficult to achieve ideal results.

[0004] In short, there is still much room for improvement in the completion effect of existing knowledge graph completion methods. Summary of the invention

[0005] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a graph neural network knowledge graph completion method based on a pre-trained model.

[0006] The present invention provides a graph neural network knowledge graph completion method based on a pre-trained model, which is used to complete the triples of missing entities in the knowledge graph. The entities in the knowledge graph include non-descriptive entities and descriptive entities, and have such characteristics, including the following steps: step S1, inputting all descriptive entities and relationships in the knowledge graph into an existing pre-trained language model respectively, and obtaining the natural language representations corresponding to each descriptive entity and relationship; step S2, according to the natural language representations corresponding to all descriptive entities, through multivariate Gaussian distribution, obtaining the natural language representations corresponding to each non-descriptive entity in the knowledge graph; step S3, constructing a knowledge graph completion model, and according to the knowledge graph, combining the natural language representations corresponding to all descriptive entities, relationships and non-descriptive entities, to complete the knowledge graph. The whole model is trained to obtain a trained knowledge graph completion model, as well as representations corresponding to each relationship and initial semantic representations corresponding to each entity without description; step S4, the triples, the natural language representations corresponding to each descriptive entity and the representations corresponding to the relationships in the triples are input into the trained knowledge graph completion model to obtain missing entities, wherein the knowledge graph completion model includes: a graph representation generation module, which is used to generate a corresponding graph representation for the natural language representation corresponding to each descriptive entity, and to generate a corresponding graph representation for the initial semantic representation corresponding to each entity without description; a score calculation module, which is used to calculate the prediction scores of the graph representations corresponding to the existing entities in the triples and the representations corresponding to the relationships with the graph representations corresponding to each entity, and take the entity corresponding to the highest prediction score as the missing entity.

[0007] In the graph neural network knowledge graph completion method based on the pre-trained model provided by the present invention, it can also have the following characteristics: wherein, in step S2, the expression of the natural language representation of the entity without description obtained by multivariate Gaussian distribution is: In the formula is the set of all undescribed entities, For collection The natural language representation of the h-th undescribed entity in E is the mean of all natural language representations of described entities, σ E is the variance of the natural language representation of all described entities, E is the set of all described entities, |E| is the total number of entities in set E, and e h is the natural language representation of the h-th described entity in the set E.

[0008] In the graph neural network knowledge graph completion method based on the pre-trained model provided by the present invention, it can also have the following characteristics: wherein the graph representation generation module includes R-GCN, and in the lth layer of R-GCN, for the i-th node, its propagation mechanism is: In the formula is the set of neighbor nodes of the ith node under relation r, c i,r is a preset constant, is the trainable parameter matrix of the lth layer for relation r, is the trainable parameter matrix of the node self-loop, is the hidden state of the ith node in layer l, is the activation function, is the natural language representation of the entity with description corresponding to the i-th node or the initial semantic representation of the entity without description. The output of the i-th node in the last layer of R-GCN is The graph representation of the entity corresponding to the node.

[0009] The graph neural network knowledge graph completion method based on the pre-trained model provided by the present invention may also have the following feature: wherein the score calculation module calculates the prediction score through the scoring function of DistMult.

[0010] In the graph neural network knowledge graph completion method based on the pre-trained model provided by the present invention, it can also have the following characteristics: wherein, step S3 includes the following sub-steps: step S3-1, inputting the natural language representation corresponding to each relationship into the graph representation generation module respectively to generate the corresponding training graph representation; step S3-2, using the natural language representation corresponding to each descriptive entity as the training initial semantic representation corresponding to the descriptive entity, and using the natural language representation corresponding to each non-descriptive entity as the training initial semantic representation corresponding to the non-descriptive entity; step S3-3, inputting the training initial semantic representation corresponding to each entity into the graph representation generation module respectively to generate the corresponding training graph representation; step S3-4, randomly sampling a batch of triples from the knowledge graph to construct Create batch training data; step S3-5, calculate the loss result according to the batch training data and the corresponding training graph representation, combined with the loss function; step S3-6, according to the loss result, optimize the trainable parameter matrix of the graph representation generation module, the training graph representation corresponding to each relationship, and the training initial semantic representation corresponding to the undescription entity; step S3-7, repeat steps S3-3 to S3-6 until the preset termination condition is reached, then the knowledge graph completion model with the optimized graph representation generation module is used as the trained knowledge graph completion model, the optimized training graph representation corresponding to the relationship is used as the representation corresponding to the relationship, and the optimized training initial semantic representation corresponding to the undescription entity is used as the initial semantic representation corresponding to the undescription entity.

[0011] In the graph neural network knowledge graph completion method based on the pre-training model provided by the present invention, it can also have the following characteristics: wherein, the batch training data includes B triplets existing in the knowledge graphs as positive sample triplets, and multiple negative sample triplets constructed according to the positive sample triplets by the batch negative sampling method, the negative sample triplets include tail entity replacement triplets, head entity replacement triplets, head self-replacement triplets and tail self-replacement triplets, the tail entity replacement triplet is to replace the tail entity in each positive sample triplet with the tail entity in other positive sample triplets, and remove each positive sample triplet. The head entity replacement triplet is for each positive sample triplet, the head entity in the positive sample triplet is replaced by the head entity in other positive sample triplet, and the triplet is obtained after removing each positive sample triplet. The head self-replacement triplet is for each positive sample triplet, the tail entity in the positive sample triplet is replaced by the head entity in the positive sample triplet, and the triplet is obtained after removing each positive sample triplet. The tail self-replacement triplet is for each positive sample triplet, the head entity in the positive sample triplet is replaced by the tail entity in the positive sample triplet, and the triplet is obtained after removing each positive sample triplet.

[0012] In the graph neural network knowledge graph completion method based on the pre-trained model provided by the present invention, it can also have the following characteristics: wherein the expression of the loss function is:

[0013]

[0014]

[0015]

[0016] In the formula is the loss function, λ1 and λ2 are the weight coefficients of the corresponding items, For expectations, is a triple (h i ,r i ,t i ), Training data for batches The predicted scores of all negative sample triplets corresponding to the ith positive sample triplet in are summed after exponential activation, ε is the set of all entities, x h is the training graph representation corresponding to entity h in the set ε, is the set of all relations, x r For collection The training graph corresponding to the relation r is represented as follows, E is the set of all described entities, is the training graph representation corresponding to entity h in set E, e h is the natural language representation of entity h, e r is the natural language representation corresponding to the relation r.

[0017] Functions and Effects of the Invention

[0018] According to the graph neural network knowledge graph completion method based on the pre-trained model involved in the present invention, on the one hand, the tail self-replacement triples and the head self-replacement triples are constructed as part of the negative samples, thereby reducing the adverse effects of the asymmetric entity distribution in the knowledge graph on the training knowledge graph completion model and improving the performance of the knowledge graph completion model; on the other hand, the deep language representation information, i.e., the natural language representation, of each descriptive entity is generated through the pre-trained language model, and the natural language representation of the non-descriptive entity is generated through the natural language representation of the descriptive entity using the multivariate Gaussian distribution, and then the natural language representation of the non-descriptive entity and the representation of the relationship are continuously optimized through training, thereby obtaining high-quality representations of each non-descriptive entity and relationship. Therefore, the graph neural network knowledge graph completion method based on the pre-trained model of the present invention can improve the completion effect of the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a method for completing a graph neural network knowledge graph based on a pre-trained model in an embodiment of the present invention;

[0020] Figure 2 is a block diagram of a knowledge graph completion model in an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of the training principle of the knowledge graph completion model in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the graph neural network knowledge graph completion method based on the pre-trained model of the present invention.

[0023] The graph neural network knowledge graph completion method based on a pre-trained model in this embodiment is used to complete the triples of missing entities in the knowledge graph.

[0024] The knowledge graph can be represented as Where ε represents the set of entities, represents a collection of relations, represents a set of triples, and For a triple h represents the head entity, r represents the relationship, and t represents the tail entity. Given a description set For some entity h∈ε or relation exist and where d h and d r Respectively represent their natural language descriptions, and any w i and v i All are vocabulary That is, the knowledge graph Contains a collection that can be described The entities being described have described entities and the relationships have described relationships, including those that cannot be described through collections. The entity being described is the entity without description and the relationship is the relationship without description. All relations are considered as descriptive relations, and the treatment of undescriptive relations is the same as the treatment of undescriptive entities below, so it will not be repeated.

[0025] Triple completion is to use the information of the existing knowledge graph to infer the missing facts, such as predicting the tail entity when the head entity and relationship are known, that is, (h, r, ?), or predicting the head entity when the tail entity and relationship are known, that is, (?, r, t). In essence, this is a binary classification problem of triples, that is, predicting whether a triple exists.

[0026] Figure 1 It is a flow chart of the method for completing the graph neural network knowledge graph based on a pre-trained model in an embodiment of the present invention.

[0027] like Figure 1 As shown in the figure, the graph neural network knowledge graph completion method based on the pre-trained model includes the following steps:

[0028] Step S1: Input all described entities and relations in the knowledge graph into an existing pre-trained language model to obtain the natural language representation corresponding to each described entity and relation. In this embodiment, the existing pre-trained language model is BERT.

[0029] Step S2, based on the natural language representations corresponding to all descriptive entities, obtain the natural language representations corresponding to each non-descriptive entity in the knowledge graph through multivariate Gaussian distribution.

[0030] Among them, the expression of natural language representation of the entity without description obtained through multivariate Gaussian distribution is:

[0031]

[0032] In the formula is the set of all undescribed entities, For collection The natural language representation of the h-th undescribed entity in E is the mean of all natural language representations of described entities, σ E is the variance of the natural language representation of all described entities, E is the set of all described entities, |E| is the total number of entities in set E, and e h is the natural language representation of the h-th described entity in the set E.

[0033] Step S3, construct a knowledge graph completion model, and train the knowledge graph completion model based on the knowledge graph and the natural language representations corresponding to all descriptive entities, relationships, and non-descriptive entities to obtain a trained knowledge graph completion model, as well as the representations corresponding to each relationship and the initial semantic representations corresponding to each non-descriptive entity.

[0034] Figure 2 It is a block diagram of the knowledge graph completion model in an embodiment of the present invention.

[0035] like Figure 2 As shown, the knowledge graph completion model 100 includes a graph representation generation module 10 and a score calculation module 20.

[0036] The graph representation generation module 10 is used to generate a corresponding graph representation for the natural language representation corresponding to each descriptive entity, and to generate a corresponding graph representation for the initial semantic representation corresponding to each non-descriptive entity.

[0037] The figure shows that the generation module 10 includes R-GCN. In the lth layer of R-GCN, for the i-th node, its propagation mechanism is:

[0038]

[0039] In the formula is the set of neighbor nodes of the ith node under relation r, c i,r is a preset constant, is the trainable parameter matrix of the lth layer for relation r, is the trainable parameter matrix of the node self-loop, is the hidden state of the ith node in layer l, is the activation function.

[0040] is the natural language representation of the entity with description corresponding to the i-th node or the initial semantic representation of the entity without description. The output of the i-th node in the last layer of R-GCN is is the graph representation corresponding to the entity corresponding to the node. In the training knowledge graph completion model 100 in this embodiment, the graph representation generation module 10 is also used to generate the training graph representation corresponding to the relationship according to the natural language representation corresponding to the relationship, and to generate the training graph representation corresponding to the descriptionless entity according to the natural language representation corresponding to the descriptionless entity, that is, There is also a natural language representation corresponding to the entity or relationship, then the output of the i-th node in the last layer of R-GCN is It also represents the training graph corresponding to the described entity or relationship corresponding to the node.

[0041] The score calculation module 20 is used to calculate the prediction scores of the graph representations corresponding to the existing entities and the representations corresponding to the relationships in the triples and the graph representations corresponding to the entities, and take the entity corresponding to the highest prediction score as the missing entity.

[0042] Among them, the score calculation module 20 calculates the prediction score through the scoring function of DistMult. In this embodiment, the prediction score of the triple composed of the existing entities and relations in the triple and each entity is calculated through the scoring function of DistMult. The scoring function gives a higher prediction score to the more matching triples, so the entity in the triple corresponding to the highest prediction score is selected, which can be used as the entity that best matches the triple, that is, the missing entity.

[0043] Figure 3 It is a schematic diagram of the training principle of the knowledge graph completion model in an embodiment of the present invention.

[0044] like Figure 3 As shown, a training dataset is constructed based on the knowledge graph, and batch training data is constructed based on the training dataset, and the loss function is combined The knowledge graph completion model 100 is trained.

[0045] Then, step S3 includes the following sub-steps:

[0046] Step S3-1, input the natural language representation corresponding to each relationship into the graph representation generation module to generate the corresponding training graph representation.

[0047] Step S3-2, using the natural language representation corresponding to each descriptive entity as the training initial semantic representation corresponding to the descriptive entity, and using the natural language representation corresponding to each non-descriptive entity as the training initial semantic representation corresponding to the non-descriptive entity.

[0048] Step S3-3, input the training initial semantic representation corresponding to each entity into the graph representation generation module to generate the corresponding training graph representation.

[0049] Step S3-4, randomly sample a batch of triplets from the knowledge graph to construct batch training data.

[0050] The batch training data includes B triplets in the knowledge graph as positive sample triplets, and multiple negative sample triplets constructed by the batch negative sampling method based on the positive sample triplets. Negative sample triplets include tail entity replacement triplets, head entity replacement triplets, head self-replacement triplets, and tail self-replacement triplets.

[0051] The tail entity replacement triple is a triple obtained by replacing the tail entity in each positive sample triple with the tail entity in other positive sample triples and removing each positive sample triple. Then, for a batch of training data, the matrix S of the prediction score corresponding to the tail entity replacement triple is T for:

[0052]

[0053] Where X H ,X R ,X T ∈R N×D are the representations of the head entity, relation, and tail entity in this batch of training data, respectively, and ⊙ represents the Hadamard product. That is, the above process is equivalent to binding the head entity and the relation, that is, performing the Hadamard product, and then performing the calculation with different tail entities, that is, the dot product. Therefore, the matrix S T Only include (h i ,r i ,t j ),i,j=1,…,the score of B.

[0054] The head entity replacement triplet is a triplet obtained by replacing the head entity in each positive sample triplet with the head entity in other positive sample triplet and removing each positive sample triplet. Then, the matrix S of the prediction score corresponding to the head entity replacement triplet is H for:

[0055]

[0056] Since the scoring function It is a three-vector dot product operation. When the head and tail entities are the same, it is usually easy to get a higher score. In addition, since the distribution of entities in some knowledge graphs is asymmetric, for example, the financial equity knowledge graph contains a large number of triples containing relationship types such as region, industry, and shareholder type. The tail entity of these triples only appears at the tail of the triple. Therefore, head self-replacement triples and tail self-replacement triples are added.

[0057] The head self-replacement triple is a triple obtained by replacing the tail entity in each positive sample triple with the head entity in the positive sample triple and removing each positive sample triple. Then, the matrix S of the prediction score corresponding to the head self-replacement triple is h_self for:

[0058]

[0059] Where 1∈1 D is a D-dimensional column vector used to sum the matrix row by row after the Hadamard product operation to obtain the score

[0060] The tail self-replacement triple is a triple obtained by replacing the head entity in each positive sample triple with the tail entity in the positive sample triple and removing each positive sample triple. Then, the matrix S of the prediction score corresponding to the tail self-replacement triple is t_self for:

[0061]

[0062] Where 1∈1 D Represents a D-dimensional column vector used to sum the matrix row by row after the Hadamard product operation to obtain the score

[0063] In this embodiment, the above four matrices are concatenated to obtain a complete score matrix S, which is expressed as follows:

[0064]

[0065] Since different triplets may have the same head or tail entity in a batch of training data, this may cause positive samples to appear in the non-diagonal negative sample positions of these constructs. In order to ensure the normal progress of contrastive learning, only diag(S T ) as the only positive sample target, ignoring diag(S H ) and other non-diagonal elements are possible positive samples, namely false negative samples, to prevent repeated calculations, while the remaining positions are real negative samples that need to be compared with positive samples. In order to achieve the above process, filtering is performed within each batch, and the score of the above false negative samples is manually set to a small negative number, such as -10 8 , so that after softmax activation, the contribution to the overall probability calculation approaches 0.

[0066] Step S3-5, based on the batch training data and the corresponding training graph representation, combined with the loss function, calculate the loss result.

[0067] Among them, the loss function The expression is:

[0068]

[0069]

[0070] In the formula is the loss function, λ1 and λ2 are the weight coefficients of the corresponding items, For expectations, is a triple (h i ,r i ,t i ), Training data for batches The predicted scores of all negative sample triplets corresponding to the ith positive sample triplet in are summed after exponential activation, ε is the set of all entities, x h is the training graph representation corresponding to entity h in the set ε, is the set of all relations, x r For collection The training graph corresponding to the relation r is represented as follows, E is the set of all described entities, is the training graph representation corresponding to entity h in set E, e h is the natural language representation of entity h, e r is the natural language representation corresponding to the relation r.

[0071] Step S3-6, based on the loss results, optimize the trainable parameter matrix of the graph representation generation module, the training graph representations corresponding to each relationship, and the training initial semantic representations corresponding to the descriptionless entities.

[0072] Step S3-7, repeating steps S3-3 to S3-6 until the preset termination condition is reached, then the knowledge graph completion model with the optimized graph representation generation module is used as the trained knowledge graph completion model, the optimized training graph representation corresponding to the relationship is used as the representation corresponding to the relationship, and the optimized training initial semantic representation corresponding to the descriptionless entity is used as the initial semantic representation corresponding to the descriptionless entity. In this embodiment, the preset termination condition is the convergence of the knowledge graph completion model.

[0073] Step S4, input the triples, the natural language representations corresponding to each descriptive entity, and the representations corresponding to the relationships in the triples into the trained knowledge graph completion model to obtain the missing entities.

[0074] Functions and Effects of the Embodiments

[0075] According to the graph neural network knowledge graph completion method based on the pre-trained model involved in this embodiment, on the one hand, tail self-replacement triples and head self-replacement triples are constructed as part of negative samples, thereby reducing the adverse effects of the asymmetric entity distribution in the knowledge graph on the training knowledge graph completion model and improving the performance of the knowledge graph completion model; on the other hand, the deep language representation information, i.e., the natural language representation, of each descriptive entity is generated through the pre-trained language model, and the natural language representation of the non-descriptive entity is generated through the natural language representation of the descriptive entity using the multivariate Gaussian distribution, and then the natural language representation of the non-descriptive entity and the representation of the relationship are continuously optimized through training, thereby obtaining high-quality representations of each non-descriptive entity and relationship. In short, this method can improve the completion effect of the knowledge graph.

[0076] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A graph neural network knowledge graph completion method based on a pre-trained model, used to complete the triples of missing entities in a knowledge graph, wherein the entities in the knowledge graph include entities without description and entities with description, characterized in that: The following steps are involved: Step S1, inputting all the described entities and relations in the knowledge graph into the existing pre-trained language model respectively to obtain the natural language representation corresponding to each of the described entities and the relations; Step S2, obtaining the natural language representation corresponding to each of the non-descriptive entities in the knowledge graph through multivariate Gaussian distribution according to the natural language representations corresponding to all the descriptive entities; Step S3, constructing a knowledge graph completion model, and training the knowledge graph completion model according to the knowledge graph, combined with the natural language representations corresponding to all the descriptive entities, the relationships and the non-descriptive entities, to obtain a trained knowledge graph completion model, as well as the representations corresponding to each of the relationships and the initial semantic representations corresponding to each of the non-descriptive entities; Step S4, inputting the triples, the natural language representations corresponding to the described entities, and the representations corresponding to the relationships in the triples into the trained knowledge graph completion model to obtain missing entities, Wherein, the knowledge graph completion model includes: A graph representation generation module, used to generate a corresponding graph representation for the natural language representation corresponding to each of the descriptive entities, and to generate a corresponding graph representation for the initial semantic representation corresponding to each of the non-descriptive entities; The score calculation module is used to calculate the prediction scores of the graph representations corresponding to the existing entities in the triples, the representations corresponding to the relationships, and the graph representations corresponding to the entities, and take the entity corresponding to the highest prediction score as the missing entity.

2. The method for completing a knowledge graph using a graph neural network based on a pre-trained model according to claim 1, characterized in that: in, In step S2, the natural language expression of the descriptionless entity is obtained through multivariate Gaussian distribution: In the formula is the set of all such entities without description, For collection The natural language representation of the h-th undescribed entity in E is the mean of the natural language representations of all described entities, σ E is the variance of the natural language representation of all described entities, E is the set of all described entities, |E| is the total number of entities in set E, e h is the natural language representation of the h-th described entity in the set E.

3. The method for completing a knowledge graph using a graph neural network based on a pre-trained model according to claim 1, characterized in that: in, The figure shows that the generation module includes R-GCN, In the lth layer of the R-GCN, for the i-th node, its propagation mechanism is: In the formula is the set of neighbor nodes of the ith node under relation r, c i,r is a preset constant, is the trainable parameter matrix of the lth layer for relation r, is the trainable parameter matrix of the node self-loop, is the hidden state of the ith node in layer l, is the activation function, is the natural language representation of the described entity corresponding to the i-th node or the initial semantic representation of the undescribed entity, The output of the i-th node in the last layer of the R-GCN is The graph representation corresponding to the entity corresponding to the node.

4. The method for completing a knowledge graph using a graph neural network based on a pre-trained model according to claim 1, characterized in that: in, The score calculation module calculates the prediction score through the scoring function of DistMult.

5. According to claim 1, the graph neural network knowledge graph completion method based on the pre-training model, Features: Wherein, the step S3 includes the following sub-steps: Step S3-1, inputting the natural language representations corresponding to the respective relationships into the graph representation generation module to generate corresponding training graph representations; Step S3-2, using the natural language representation corresponding to each of the described entities as the training initial semantic representation corresponding to the described entity, and using the natural language representation corresponding to each of the non-described entities as the training initial semantic representation corresponding to the non-described entity; Step S3-3, inputting the training initial semantic representation corresponding to each of the entities into the graph representation generation module to generate the corresponding training graph representation; Step S3-4, randomly sampling a batch of triplets from the knowledge graph to construct batch training data; Step S3-5, calculating a loss result based on the batch training data and the corresponding training graph representation in combination with a loss function; Step S3-6, optimizing the trainable parameter matrix of the graph representation generation module, the training graph representations corresponding to the respective relationships, and the training initial semantic representations corresponding to the descriptionless entities according to the loss results; Step S3-7, repeat steps S3-3 to S3-6 until the preset termination condition is reached, then the knowledge graph completion model with the optimized graph representation generation module is used as the trained knowledge graph completion model, the optimized training graph representation corresponding to the relationship is used as the representation corresponding to the relationship, and the optimized training initial semantic representation corresponding to the non-descriptive entity is used as the initial semantic representation corresponding to the non-descriptive entity.

6. The method for completing a knowledge graph using a graph neural network based on a pre-trained model according to claim 5, characterized in that: in, The batch training data includes B triplets existing in the knowledge graph as positive sample triplets, and a plurality of negative sample triplets constructed according to the positive sample triplets by a batch negative sampling method, The negative sample triples include a tail entity replacement triple, a head entity replacement triple, a head self-replacement triple and a tail self-replacement triple. The tail entity replacement triple is a triple obtained by replacing the tail entity in each positive sample triple with the tail entity in other positive sample triples and removing each positive sample triple. The head entity replacement triplet is a triplet obtained by replacing the head entity in each positive sample triplet with the head entity in other positive sample triplet and removing each positive sample triplet. The head self-replacement triple is a triple obtained by replacing the tail entity in each positive sample triple with the head entity in the positive sample triple and removing each positive sample triple. The tail self-replacement triplet is a triplet obtained by replacing the head entity in each of the positive sample triples with the tail entity in the positive sample triplet and removing each of the positive sample triples.

7. The method for completing a knowledge graph using a graph neural network based on a pre-trained model according to claim 5, characterized in that: in, The expression of the loss function is: In the formula is the loss function, λ1 and λ2 are the weight coefficients of the corresponding items, For expectations, is a triple (h i , r i , t i ), Training data for batches The sum of the predicted scores of all negative sample triplets corresponding to the i-th positive sample triplet after exponential activation, ε is the set of all the entities, x h is the training graph representation corresponding to entity h in the set ε, is the set of all relations, x r For collection The training graph corresponding to the relation r is represented as follows, E is the set of all described entities, is the training graph representation corresponding to entity h in set E, e h is the natural language representation of entity h, e r is the natural language representation corresponding to the relation r.

Images