Knowledge graph link prediction method

Abstract

The present invention relates to the technical field of knowledge graph completion tasks, and particularly relates to a knowledge graph link prediction method. The method comprises: using a precoding model to obtain an embedding layer vector, and constructing a corresponding masked triple; adding a corresponding position code to each element in the masked triple, so as to obtain a corresponding input sequence, inputting the input sequence into a trained main masking model, and outputting an entity classification probability; and on the basis of the entity classification probability, predicting potential candidate entities. The method further comprises: concatenating semantic information corresponding to the embedding layer vector and structural information obtained by an embedding model, so as to obtain fused head entity and relation representations, and constructing a corresponding fused masked triple; and adding a corresponding position code to each element in the fused masked triple, so as to obtain a corresponding fused input sequence. The present invention uses a precoding method, thereby effectively reducing the training burden on a model, and improving the inference speed of a model; and a fusion module is used before inputs are fed into a main masked model, thereby ensuring the integrity of textual description information and improving prediction accuracy.

Description

A knowledge graph link prediction method Technical Field

[0001] The present invention relates to the technical field of knowledge graph completion tasks, and in particular to a knowledge graph link prediction method. Background Art

[0002] A knowledge graph is a graphical model for representing and organizing knowledge. It describes the relationships between entities in the form of triples. Entities include real-world entities such as objects, concepts, and events, while relationships represent their connections and interactions. A triple is structured in the form of (head, relationship, tail), where the head and tail are entities, and the relationship connects the head and tail entities. Currently, many artificially generated knowledge graphs exist, but many missing relationships between entities remain. Knowledge graphs aim to leverage known entities and relationships to accurately predict unknown relationships, thereby enhancing knowledge graph information.

[0003] Existing knowledge graph completion methods are mainly divided into: completion based on embedding models, completion based on description models, and combining graph structure information and semantic description information (joint models); among them, description models use text descriptions of entities and relationships, use pre-trained language models to encode descriptions as feature representations, and use features to match missing parts, such as KG-BERT and MTL-KGC; although such models can utilize information from outside the graph structure, they need to input text directly into the pre-trained language model, resulting in a large amount of memory and time required for training, and the inference speed is significantly slowed down; there is little research on joint models, and existing research does not fully utilize the structural information in the graph; most models directly align description features to the graph structure, but inappropriate alignment methods will destroy the text description information.

[0004] In summary, the existing technologies for knowledge graph completion algorithms have the following problems: (1) Description-based knowledge completion algorithms rely on pre-trained language models, but the text encoding of pre-trained language models will seriously affect the reasoning speed; (2) When fusing information and semantic description information, the description features are directly aligned to the graph structure, and inappropriate alignment will destroy the text description information.

[0005] Summary of the Invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the description-based knowledge completion algorithm in the existing technology relies on a pre-trained language model, but the text encoding of the pre-trained language model will seriously affect the reasoning speed; when fusing the combined information and semantic description information, the description features are directly aligned to the graph structure, and the inappropriate alignment method will destroy the text description information.

[0007] To solve the above technical problems, the present invention provides a knowledge graph link prediction method, comprising:

[0008] Obtain a data set; wherein the data set includes an entity description text set and a relationship description text set; obtain an index value corresponding to each entity description text and construct an index set;

[0009] The data set is input into the initialized precoding model, and each entity description text and each relationship description text are preprocessed respectively; a pooling operation is performed on the preprocessed entity description text and each relationship description text to obtain the entity semantic expression of each entity description text and the relationship semantic expression of each relationship description text; an embedding operation is performed on all entity semantic expressions and all relationship semantic expressions to obtain entity embedding vectors and relationship embedding vectors;

[0010] Based on the entity embedding vector and the relationship embedding vector, combined with the index set, a triple set is constructed, and the triple set is divided into a training set and a test set. The inverse relationship triples corresponding to the training set are added to the training set to obtain the target training set; the inverse relationship triples corresponding to the test set are added to the test set to obtain the target test set;

[0011] Initialize the main mask model and extract samples from the target training set according to the preset batch size to obtain a sample set; construct each mask triplet corresponding to each sample in the sample set; add the corresponding position vector to each element in each mask triplet, and combine it with the start and end flag vectors of the main mask model to obtain the input sequence corresponding to each sample;

[0012] The input sequences corresponding to each sample are simultaneously input into the main mask model. After being processed by multiple layers of Transformer blocks, the vectors at the last mask position corresponding to each input sequence are output. The vectors at the last mask position corresponding to each input sequence are input into the classification layer. After a softmax operation, the probability of each entity distribution corresponding to each input sequence is output.

[0013] Construct a loss function and use the Adam optimizer to minimize the loss function value, update the parameters of the main mask model, and obtain the parameters of the main mask model of the current round after extracting all samples in the target training set. Use the evaluation function to evaluate the main mask model of the current round. If the performance of the main mask model of the current round is better than that of the main mask model of the previous round, save the weight of the main mask model of the current round and conduct the next round of training until the preset number of training rounds is reached. End the training and obtain the trained main mask model.

[0014] Based on the target test set, a test set mask triplet is constructed to obtain the test set input sequence; the test set input sequence is input into the trained main mask model to output the entity classification probability; and potential entities are predicted through the entity classification probability.

[0015] Preferably, the step of inputting the data set into the initialized precoding model, preprocessing each entity description text and each relationship description text respectively; performing a pooling operation on each preprocessed entity description text and each relationship description text to obtain an entity semantic expression of each entity description text and a relationship semantic expression of each relationship description text; and performing an embedding operation on all entity semantic expressions and all relationship semantic expressions to obtain entity embedding vectors and relationship embedding vectors includes:

[0016] Through the pooling operation, the entity description text and the relationship description text are summarized into semantic expressions, which are expressed as follows:

[0017] Among them, entity_des represents entity description; relation_des represents relationship description; Represents the description text of the i-th entity; Represents the j-th relationship description text; e i Represents the semantic expression of the i-th entity, i∈[1,n], n represents the number of entities; r j Represents the jth relation semantic expression, j∈[1,m], m represents the number of relations;

[0018] Embed the entity semantic expression and the relationship semantic expression to obtain the entity embedding vector and the relationship embedding vector, whose expressions are: E = [e1; e2; ...; e n ] R=[r1;r2;...;r m ] Embeddings=[E,R]

[0019] Where n represents the number of entities; m represents the number of relations; E represents the entity embedding vector; R represents the relationship embedding vector; Embeddings = [E, R] represents the embedding layer vector.

[0020] Preferably, constructing each mask triplet corresponding to each sample in the sample set; adding a corresponding position vector to each element in each mask triplet, and combining the start and end flag vectors of the main mask model to obtain each input sequence corresponding to each sample includes:

[0021] Select a sample A from the sample set and construct the mask triplet corresponding to sample A. Add the corresponding position vector to each element in the mask triplet corresponding to sample A, and combine it with the start and end flag vectors of the main mask model to obtain the input sequence corresponding to sample A. Its expression is:

[0022] Among them, x input represents the input sequence corresponding to sample A; E CLS Indicates the start symbol of the sequence; E SEP Indicates the end symbol of the sequence; E h Indicates the head entity feature representation corresponding to sample A; E r Indicates the relational feature representation corresponding to sample A; E mask represents the mask vector; Indicates the position code corresponding to the s-th position of the sequence;

[0023] Based on the above operations, each mask triplet corresponding to each sample is constructed at the same time, and each input sequence corresponding to the remaining samples is obtained.

[0024] Preferably, the steps of simultaneously inputting the input sequences corresponding to the samples into the main mask model, processing them through multiple layers of Transformer blocks, and outputting the vectors at the mask positions of the last layer corresponding to the input sequences; and inputting the vectors at the mask positions of the last layer corresponding to the input sequences into the classification layer, performing a softmax operation, and outputting the distribution probabilities of the entities corresponding to the input sequences include:

[0025] Perform the following operations on the input sequence corresponding to sample A in the sample set:

[0026] The input sequence corresponding to sample A is input into the main mask model. The input sequence corresponding to sample A will be processed by the 12-layer Transformer block. The vector at the Emask position of the last layer can be used as the expression of the predicted entity, and its expression is: num =Transformer(z num-1 ),num=1,2,...,12 z0=x input

[0027] Among them, z num Represents the input of the numth layer; z0 represents the input of the first layer; Indicates the last layer E mask vectors on positions; Indicates the final output E mask The value of

[0028] The classification layer is a feedforward neural network, and the input dimension of the classification layer is The output dimension is the number of entities; Input into the classification layer, and then through the softmax operation, the entity classification probability is obtained, and its expression is:

[0029] in, represents the entity classification probability;

[0030] Based on the above operations, each input sequence corresponding to each sample in the sample set is simultaneously input into the main mask model, and the distribution probability of each entity corresponding to each input sequence is output.

[0031] Preferably, constructing the loss function includes:

[0032] Based on the distribution probability of each entity corresponding to each input sequence, a loss function is constructed; wherein the loss function is a cross entropy loss function, and its expression is:

[0033] Among them, y i represents the actual label of the i-th entity, y i Takes 0 or 1; when y i When 1 is taken, it means that the i-th entity is a label; when y i When it is 0, it means that the i-th entity is not a label; represents the i-th probability corresponding to the i-th entity in the entity distribution probability; N represents the number of all entities; i represents the index value.

[0034] Preferably, the initialization of the main mask model further comprises extracting samples from the target training set according to a preset batch size, and obtaining the sample set:

[0035] Within a preset range, randomly initialize each entity structure embedding expression and each relationship structure embedding expression of the embedding model, and integrate each entity structure embedding expression and each relationship structure embedding expression into an entity embedding matrix and a relationship embedding matrix respectively;

[0036] The entity embedding vector obtained by the precoding model contains multiple entity semantic expressions; the relationship embedding vector obtained by the precoding model contains multiple relationship semantic expressions;

[0037] The entity semantic expression and relationship semantic expression corresponding to each sample in the sample set are spliced ​​with the entity structure embedding expression and relationship structure embedding expression corresponding to the embedding model to obtain the spliced ​​features corresponding to each sample; the spliced ​​features corresponding to each sample are input into the fusion module to obtain the fused head entity expression and each relationship expression corresponding to each sample; based on the fused head entity expression and each relationship expression corresponding to each sample, the fused mask triples corresponding to each sample are constructed; the corresponding position vector is added to each element in each fused mask triple, and combined with the start and end flag vectors of the main mask model, the fused input sequence corresponding to each sample is obtained;

[0038] Each fused input sequence corresponding to each sample is simultaneously input into the main mask model.

[0039] Preferably, the step of splicing the entity semantic expression and the relationship semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and the relationship structure embedding expression corresponding to the embedding model to obtain splicing features corresponding to each sample; and inputting the splicing features corresponding to each sample into a fusion module to obtain fused head entity expressions and relationship expressions corresponding to each sample includes:

[0040] Select any sample B from the sample set, define the entity semantic expression and relationship semantic expression corresponding to sample B as the semantically rich description expression u from the precoding model; define the corresponding entity structure embedding expression and relationship structure embedding expression in the embedding model as the embedding expression v with structural information from the embedding model;

[0041] The semantically rich description expression u from the pre-coding model and the structurally rich embedded expression v from the embedding model are concatenated to obtain the concatenated feature c. The concatenated feature is the concatenation result of the two expressions, and its expression is: c(u,v)=[u,v]

[0042] The splicing features corresponding to sample B are input into the fusion module. By reducing the dimension, the fusion information is extracted to obtain the fused head entity and relationship expression, which is expressed as follows:

[0043] in, Indicates the fused head entity expression corresponding to sample B; Represents the fused relational expression corresponding to sample B;

[0044] Based on the above operations, each sample in the sample set is spliced ​​simultaneously to obtain each splicing feature corresponding to each sample, and each fused head entity expression and each relationship expression corresponding to each sample are obtained.

[0045] Preferably, constructing each fusion mask triple corresponding to each sample based on each fused head entity expression and each relationship expression corresponding to each sample; adding a corresponding position vector to each element in each fusion mask triple, and combining the start and end flag vectors of the main mask model to obtain each fusion input sequence corresponding to each sample includes:

[0046] Based on the fused head entity expression and relation expression corresponding to sample B, the fusion mask triplet corresponding to sample B is constructed; the corresponding position vector is added to each element in the fusion mask triplet corresponding to sample B, and combined with the start and end flag vectors of the main mask model, the fusion input sequence corresponding to sample B is obtained, which is expressed as follows:

[0047] Among them, xinput_fusion represents the fusion input sequence corresponding to sample B; E CLS Indicates the start symbol of the sequence; E SEP Indicates the end symbol of the sequence; Indicates the fused head entity expression corresponding to sample B; represents the fused relational expression corresponding to sample B; E mask represents the mask vector; Indicates the position code corresponding to the s-th position of the sequence;

[0048] Based on the above operations, through the fused head entity expressions and relationship expressions corresponding to each sample, each fusion mask triple corresponding to each sample is constructed at the same time, and each fusion input sequence corresponding to each sample is obtained.

[0049] Preferably, the embedding model is a translation model.

[0050] Preferably, constructing the loss function includes:

[0051] Based on the translation model principle, the scoring function is the cosine similarity function, and the sum of the embedding vectors of the head entity and the relationship is defined as e hr , whose expression is: e hr =|e h +e r |

[0052] Among them, e h Represents the embedding vector of the head entity in the translation model; e r Embedding vector representing the relationship in the translation model; e t represents the embedding vector of the tail entity in the translation model; |||| represents the two-norm calculation; f(h,r,t) represents the scoring function;

[0053] During the training process, negative sampling is added to the translation model loss function for calculation as the contrast loss L S , so that the translation model learns an embedding representation with a positive proportion score higher than a negative proportion score, which is expressed as:

[0054] Among them, K represents the entity set of negative sampling; t k Represents the kth entity in the negatively sampled entity set; the cross entropy loss function is used as the classification loss L CE , whose expression is:

[0055] Among them, y i represents the actual label of the i-th entity, y i Takes 0 or 1; when y i When 1 is taken, it means that the i-th entity is a label; when y i When it is 0, it means that the i-th entity is not a label; represents the i-th probability corresponding to the i-th entity in the entity distribution probability; N represents the number of all entities; i represents the index value;

[0056] The total loss function L optimized by the main mask model is the weighted sum of classification loss and contrast loss, and its expression is L = L CE +αL S

[0057] Among them, L CE represents the classification loss; α represents the weight of the contrast loss; L S represents contrast loss.

[0058] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0059] (1) The knowledge graph link prediction method described in the present invention uses a precoding model to encode and summarize long text descriptions of entities and relationships into features of semantic information; since the text description does not change, the precoding operation only needs to be performed once before model training. The resulting embedding layer represents the semantic expression of all entities and relationships. Subsequent training and reasoning use this embedding layer, which alleviates the problems caused by long text encoding; this method innovatively uses the precoding method, effectively reducing the training pressure of the model and improving the reasoning speed of the model;

[0060] (2) The knowledge graph link prediction method described in the present invention takes into account that semantic information should be independent of structural information, and the two are expressions from different information sources; and is inspired by early fusion in multimodal learning. A simple fusion module is used before inputting the main mask model to first splice the semantic expression and the embedded expression into one expression, and then the most suitable fusion expression is obtained through a learnable fusion module; this method does not require the description features to be aligned to the graph structure, and there is no need to consider that inappropriate alignment will destroy the text description information, thereby ensuring the integrity of the text description information and improving the accuracy of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0062] FIG1 is a schematic diagram of the structure of a knowledge graph algorithm based on a pre-coded masked language model provided by the present invention;

[0063] Figure 2 shows the input vector of the main mask model in the pre-encoded mask language model;

[0064] FIG3 is a schematic diagram of a knowledge graph algorithm based on a framework of pre-encoded fusion structural information and semantic information provided by the present invention;

[0065] Figure 4 is an example analysis of 6 missing triplets selected from the WN18RR dataset. DETAILED DESCRIPTION

[0066] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0067] Referring to FIG1 , FIG1 is a schematic diagram of the structure of the knowledge graph algorithm based on the pre-coded masked language model provided by the present invention. In order to solve the problem that the training language model requires inputting long texts, which slows down the inference speed, the present invention adopts a pre-coded masked language model to predict the knowledge graph link. The model is composed of two BERTs (bidirectional Transformer encoder representation). The specific steps include:

[0068] S101: Obtain data set S d ={(des e ,des r )}; where the data set S d Includes entity description text collection des e , relationship description text set des r; Get the index value corresponding to each entity description text and build an index set;

[0069] S102: Input the data set into the initialized precoding model, preprocess each entity description text and each relationship description text respectively; perform a pooling operation on each preprocessed entity description text and each relationship description text to obtain the entity semantic expression of each entity description text and the relationship semantic expression of each relationship description text, which are expressed as follows:

[0070] Among them, entity_des represents entity description; relation_des represents relationship description; Represents the description text of the i-th entity; Represents the j-th relationship description text; e i Represents the semantic expression of the i-th entity, i∈[1,n], n represents the number of entities; r j Represents the jth relation semantic expression, j∈[1,m], m represents the number of relations;

[0071] Perform embedding operations on all entity semantic expressions and all relationship semantic expressions to obtain entity embedding vectors and relationship embedding vectors, whose expressions are: E = [e1; e2; ...; e n ] R=[r1;r2;...;r m ] Embeddings=[E,R]

[0072] Where n represents the number of entities; m represents the number of relations; E represents the entity embedding vector; R represents the relationship embedding vector; Embeddings = [E, R] represents the embedding layer vector;

[0073] S103: Based on the entity embedding vector and the relationship embedding vector, combined with the index set, a triple set is constructed, and the triple set is divided into a training set S train = {h, ​​r, t} and test set S test ={h,r,t}, the training set S train The corresponding inverse relation triple (t,r - ,h) Add to the training set to obtain the target training set The test set S test The corresponding inverse relation triple (t,r - ,h) Add to the test set to get the target test set

[0074] S104: Initialize the main mask model and extract batch_size samples from the target training set according to the preset batch size batch_size to obtain a sample set;

[0075] S105: Construct each mask triplet corresponding to each sample in the sample set; add the corresponding position vector to each element in each mask triplet, and combine the start and end flag vectors of the main mask model to obtain each input sequence corresponding to each sample, including:

[0076] Select a sample A from the sample set and take sample A as an example to construct the mask triplet corresponding to sample A. Add the corresponding position vector to each element in the mask triplet corresponding to sample A, and combine it with the start and end flag vectors of the main mask model to obtain the input sequence corresponding to sample A. Its expression is:

[0077] Among them, x input represents the input sequence corresponding to sample A; E CLS Indicates the start symbol of the sequence; E SEP Indicates the end symbol of the sequence; E h Indicates the head entity feature representation corresponding to sample A; E r Indicates the relational feature representation corresponding to sample A; E mask represents the mask vector; Indicates the position code corresponding to the s-th position of the sequence;

[0078] Based on the above operations, each mask triple corresponding to each sample in the sample set is constructed at the same time, and each input sequence corresponding to each sample is obtained;

[0079] The input sequence structure of the main mask model is shown in Figure 2. Since the BERT model needs to consider the relative position of each element in the input sequence to better capture the structure and relationship in the sequence, when using BERT for knowledge graph completion tasks, adding corresponding position encodings to the input at each position can provide relative position information of entities and relationships, allowing the model to better understand the relationship between different elements in the input sequence.

[0080] S106: Input each input sequence corresponding to each sample into the main mask model at the same time. After being processed by multiple layers of Transformer blocks, the vector at the last mask position corresponding to each input sequence is output. The vector at the last mask position corresponding to each input sequence is input into the classification layer. After a softmax operation, the entity distribution corresponding to each input sequence is output, including:

[0081] Taking sample A as an example, the input sequence corresponding to sample A is input into the main mask model. The input sequence corresponding to sample A will be processed by the 12-layer Transformer block. The vector at the Emask position of the last layer can be used as the expression of the predicted entity, and its expression is: num =Transformer(z num-1 ),num=1,2,...,12 z0=x input

[0082] Among them, z num Represents the input of the numth layer; z0 represents the input of the first layer; Indicates the last layer E mask vectors at positions; Indicates the final output E mask The value of

[0083] Link prediction (head entity → relation →?) can be regarded as a multi-classification problem of the mask vector; therefore, the classification layer is used to obtain the entity classification probability; the classification layer is a feedforward neural network, and the input dimension of the classification layer is The output dimension is the number of entities; after the softmax operation, the entity classification probability is obtained, and its expression is:

[0084] in, represents the entity classification probability;

[0085] Based on the above operations, each input sequence corresponding to each sample in the sample set obtained in S105 is simultaneously input into the main mask model, and the distribution probability of each entity corresponding to each input sequence is output;

[0086] S107: Construct a loss function based on the distribution probability of each entity corresponding to each input sequence; wherein the loss function is a cross entropy loss function, and its expression is:

[0087] Among them, y i represents the actual label of the i-th entity, y i Takes 0 or 1; when y i When 1 is taken, it means that the i-th entity is a label; when y i When it is 0, it means that the i-th entity is not a label; represents the probability of the i-th entity; N represents the number of all entities; i represents the index value; the purpose of this loss function is to make the probability of the correct entity as large as possible;

[0088] Assume that a dataset contains 40,943 entities, each with an index value. The batch size is preset to 256. Then, 256 data items are read each time, and 256 mask triplets are constructed, resulting in an input sequence of 256 rows and 5 columns of data. This input sequence is fed into the main mask model, and the final output is 256 rows and 40,943 columns of probabilities. The entity classification probabilities output by the classification layer are the probabilities distributed between 0 and 1 after the softmax operation, and the sum of the 40,943 probabilities is 1.

[0089] S108: Using the Adam optimizer to minimize the loss function value, update the parameters of the main mask model until all samples in the target training set are extracted, and obtain the parameters of the main mask model of the current round; using the evaluation function to evaluate the main mask model of the current round, if the performance of the main mask model of the current round is better than that of the main mask model of the previous round, save the weight of the main mask model of the current round, and perform the next round of training until the preset number of training rounds is reached, and then terminate the training to obtain the trained main mask model;

[0090] S109: Based on the target test set, construct a test set mask triplet to obtain a test set input sequence; input the test set input sequence into the trained main mask model and output the entity classification probability; predict potential entities through the entity classification probability.

[0091] According to the above steps, the algorithm flow of the model can be obtained as follows:

[0092] Based on the above embodiment, in order to improve the prediction accuracy of the knowledge graph link in this embodiment, based on the framework of pre-encoded fusion structural information and semantic information, each fusion input sequence corresponding to each sample is obtained as the input sequence of the main mask model;

[0093] Referring to FIG3 , FIG3 is a schematic diagram of a knowledge graph algorithm based on a framework of pre-encoding fusion structural information and semantic information provided by the present invention; the knowledge graph completion based on the framework of fusion structural information and semantic information consists of three stages, namely, the pre-encoding stage, the fusion stage, and the encoding stage; the specific steps include:

[0094] S301: Obtain data set S d ={(des e ,des r )}; where the data set S d Includes entity description text collection des e , relationship description text set des r ; Get the index value corresponding to each entity description text and build an index set;

[0095] S302: Input the data set into the initialized precoding model, preprocess each entity description text and each relationship description text respectively; perform a pooling operation on each preprocessed entity description text and each relationship description text to obtain the entity semantic expression of each entity description text and the relationship semantic expression of each relationship description text, which are expressed as follows:

[0096] Among them, entity_des represents entity description; relation_des represents relationship description; Represents the description text of the i-th entity; Represents the j-th relationship description text; e i Represents the semantic expression of the i-th entity, i∈[1,n], n represents the number of entities; r j Represents the jth relation semantic expression, j∈[1,m], m represents the number of relations;

[0097] Perform embedding operations on all entity semantic expressions and all relationship semantic expressions to obtain entity embedding vectors and relationship embedding vectors, whose expressions are: E = [e1; e2; ...; e n ] R=[r1;r2;...;r m ] Embeddings=[E,R]

[0098] Where n represents the number of entities; m represents the number of relations; E represents the entity embedding vector; R represents the relationship embedding vector; Rmbeddings = [E, R] represents the embedding layer vector;

[0099] S303: Based on the entity embedding vector and the relationship embedding vector, combined with the index set, a triple set is constructed, and the triple set is divided into a training set S train = {h, ​​r, t} and test set S test ={h,r,t}, the training set S train The corresponding inverse relation triple (t,r - ,h) Add to the training set to obtain the target training set The test set S test The corresponding inverse relation triple (t,r - ,h) Add to the test set to get the target test set

[0100] exist Within the range, randomly initialize the embedding expressions of each entity structure and each relationship structure of the embedding model, and integrate the embedding expressions of each entity structure and each relationship structure into the entity embedding matrix and the relationship embedding matrix respectively; where dim represents the feature dimension of the embedding layer vector;

[0101] S304: Initialize the main mask model and extract batch_size samples from the target training set according to the preset batch size batch_size to obtain a sample set;

[0102] S305: Concatenate the entity semantic expression and relationship semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and relationship structure embedding expression corresponding to the embedding model to obtain concatenated features; input each concatenated feature into the fusion module to obtain the fused head entity expression and relationship expression corresponding to each sample, including:

[0103] Select any sample B from the sample set and take sample B as an example. Define the entity semantic expression and relationship semantic expression corresponding to sample B as the semantically rich description expression u from the precoding model; define the corresponding entity structure embedding expression and relationship structure embedding expression in the embedding model as the embedding expression v with structural information from the embedding model.

[0104] The semantically rich description expression u from the pre-coding model and the structurally rich embedded expression v from the embedding model are concatenated to obtain the concatenated feature c. The concatenated feature is the concatenation result of the two expressions, and its expression is: c(u,v)=[u,v]

[0105] The splicing features corresponding to sample B are input into the fusion module. By reducing the dimension, the fusion information is extracted to obtain the fused head entity and relationship expression, which is expressed as follows:

[0106] in, Indicates the fused head entity expression corresponding to sample B; Represents the fused relational expression corresponding to sample B;

[0107] Based on the above operations, each sample in the sample set is spliced ​​at the same time to obtain the splicing features corresponding to each sample, and obtain the fused head entity expressions and relationship expressions corresponding to each sample;

[0108] S306: Constructing fusion mask triples corresponding to each sample based on the fused head entity expressions and relationship expressions corresponding to each sample; adding corresponding position vectors to each element in each fusion mask triple, and combining the start and end flag vectors of the main mask model to obtain fusion input sequences corresponding to each sample, including:

[0109] Taking sample B as an example, based on the fused head entity expression and relation expression corresponding to sample B, the fused mask triplet corresponding to sample B is constructed; the corresponding position vector is added to each element in the fused mask triplet corresponding to sample B, and combined with the start and end flag vectors of the main mask model, the fused input sequence corresponding to sample B is obtained, which is expressed as:

[0110] Among them, xinput_fusion represents the fusion input sequence corresponding to sample B; E CLS Indicates the start symbol of the sequence; E SEP Indicates the end symbol of the sequence; Indicates the fused head entity expression corresponding to sample B; represents the fused relational expression corresponding to sample B; E mask represents the mask vector; Indicates the position code corresponding to the s-th position of the sequence;

[0111] Based on the above operations, through the fused head entity expressions and relationship expressions corresponding to each sample, each fusion mask triple corresponding to each sample is constructed, and each fusion input sequence corresponding to each sample is obtained;

[0112] Among them, S305 and S306 belong to the fusion module in the framework of pre-encoded fusion structural information and semantic information. Through this fusion module, the prediction accuracy of the overall fusion framework can be improved by fusing structural information and semantic information, while ensuring the integrity of text description information;

[0113] S307: Input each fused input sequence corresponding to each sample into the main mask model at the same time, and after being processed by multiple layers of Transformer blocks, output the vector at the last mask position corresponding to each fused input sequence; input the vector at the last mask position corresponding to each fused input sequence into the classification layer, and after a softmax operation, output the distribution probability of each entity corresponding to each fused input sequence;

[0114] S308: In a specific embodiment of the present invention, the adopted embedding model is a translation model; then constructing the loss function includes:

[0115] The translation model principle is to transform the relationship into a transformation between entities, and point the sum of the head entity and the relationship vector to the tail entity. The expected vector performance is as follows: h+r≈t; the scoring function is the cosine similarity function, and the sum of the embedding vectors of the head entity and the relationship is defined as e hr , whose expression is: e hr =|e h +e r |

[0116] Among them, e h Represents the embedding vector of the head entity in the translation model; e r Embedding vector representing the relationship in the translation model; e t represents the embedding vector of the tail entity in the translation model; |||| represents the two-norm calculation; f(h,r,t) represents the scoring function;

[0117] In the training process of traditional translation models, negative sampling operations are usually added, which can effectively improve the generalization performance of the model, solve the imbalance problem between samples, and enhance the robustness of the model. In this paper, negative sampling is added to the loss function for calculation as a contrast loss, so that the translation model learns an embedding representation with a higher positive proportional score than a negative proportional score. The expression is:

[0118] Among them, K represents the entity set of negative sampling; t k Represents the kth entity in the negatively sampled entity set;

[0119] The score of the scoring function f(h,r,t) is between [-1,1]. The closer the score is to 1, the more the mapping between the head entity vector and the relationship vector can point to the tail entity, that is, the better the model performance is. In the problem of predicting the tail entity (head entity, relationship,?), the translation model's predicted output is the entity with the highest score, and its expression is:

[0120] Where E is the set of all entities; t res_i represents the entity with the highest score;

[0121] The cross entropy loss function is used as the classification loss L CE , whose expression is

[0122] Among them, y i represents the actual label of the i-th entity, y i Takes 0 or 1; when y i When 1 is taken, it means that the i-th entity is a label; when y i When it is 0, it means that the i-th entity is not a label; represents the i-th probability corresponding to the i-th entity in the entity distribution probability; N represents the number of all entities; i represents the index value;

[0123] The total loss function L optimized by the main mask model is the weighted sum of classification loss and contrast loss, and its expression is L = L CE +αL S

[0124] Among them, L CErepresents the classification loss; α represents the weight of the contrast loss; L S represents contrast loss;

[0125] S309: Using the Adam optimizer to minimize the loss function value, update the parameters of the main mask model until all samples in the target training set are extracted, and obtain the parameters of the main mask model of the current round; using the evaluation function to evaluate the main mask model of the current round, if the performance of the main mask model of the current round is better than that of the main mask model of the previous round, save the weight of the main mask model of the current round, and perform the next round of training until the preset number of training rounds is reached, and then terminate the training to obtain the trained main mask model;

[0126] S310: Based on the target test set, construct a test set fusion mask triplet to obtain a test set fusion input sequence; input the test set fusion input sequence into the trained main mask model and output the entity classification probability; predict potential entities through the entity classification probability.

[0127] According to the above steps, the algorithm flow of this model is as follows:

[0128] In order to verify the reasoning performance of the pre-coding-based masked language model and the framework based on pre-coding fusion of structural information and semantic information in the present invention, experiments were conducted on the pre-coding-based masked language model and the framework based on pre-coding fusion of structural information and semantic information:

[0129] 1. Experimental Dataset

[0130] This experiment uses two benchmark datasets: WN18RR and UMLS (Uniform Medical Language System). WN18RR is a subset of the WordNet data set that describes the association features between English words while retaining the symmetric, asymmetric, and compositional relationships in the WordNet dataset. It is used to evaluate link prediction tasks. The UMLS (Uniform Medical Language System) dataset is a small knowledge graph for the biomedical and health fields. The scales of the two benchmark datasets are shown in Table 1, namely:

[0131] Table 1 Dataset specifications

[0132] It should be noted that the entities have filtered out triplets that did not appear in the training set, and the entities and relations in the validation set and test set have also appeared in the training set; the data in the validation set and test set do not participate in model optimization, and the subsequent experimental conclusions are all evaluation results of the two models on the test set; the text descriptions of entities and relations in the dataset use the content defined by the BERT-based knowledge graph embedding algorithm (KG-BERT).

[0133] 2. Experimental evaluation indicators

[0134] The missing triples (head entity, relationship,?) of the knowledge graph may correspond to multiple tail entities. According to convention, when evaluating, it is necessary to remove all possible entities (except label entities) that have appeared in the training set, validation set, and test set from the candidate triples; among the filtered entities, the softmax value output by the classification layer is used for ranking; the true answer of the i-th triple in the test set is ranked among all entities. i .

[0135] 1. Average ranking (MR)

[0136] MR refers to all ranks in the test set i The average value of is expressed as:

[0137] This indicator reflects the ranking of the true answer among the candidate entities under average circumstances. The lower the MR, the better the reasoning performance of the model.

[0138] 2. Mean Reciprocal Ranking (MRR)

[0139] MRR refers to the rank of all the ranks in the test set i The average value of the reciprocal of is expressed as:

[0140] Compared to MR, this metric focuses more on whether the model can find a high ranking of the true answer among the candidate entities; the higher the MRR, the better the model's reasoning performance;

[0141] 3. Average number of top n nodes (Hits@n)

[0142] Hits@n refers to all ranks in the test set i ≤ the average value of n, where n is usually 10, 3, or 1. The expression is:

[0143] This indicator model reflects whether the model can rank the true answer into the top n, focusing only on the model's high ranking performance; the larger the Hits@n, the better the model's inference performance.

[0144] 3. Experimental Environment

[0145] The experimental environment will affect the experimental results. The experimental environment is shown in Table 2 for research reference, namely:

[0146] Table 2 Experimental environment

[0147] 4. Parameter settings

[0148] Parameter settings will affect the experimental results. The detailed parameter settings of this experiment are shown in Table 3, namely:

[0149] Table 3 Parameter settings

[0150] 5. Experimental Results

[0151] In order to compare the inference speed advantage of the precoding mechanism in the present invention, KG-BERT, the masked language model based on precoding and the framework for fusing structural information and semantic information based on precoding provided by the present invention were selected in the experiment for comparison on the WN18RR and UMLS test sets; the input of KG-BERT is text, while the input of the model provided by the present invention is all precoded expressions; the inference speed of KG-BERT is related to the entity scale in the dataset, and the more entities, the longer the inference speed; as shown in Table 4, Table 4 shows the accelerated inference effect brought about by avoiding encoding long text under the precoding mechanism; among them, Ours-1 represents the masked language model based on precoding provided by the present invention, and Ours-2 represents the framework for fusing structural information and semantic information based on precoding provided by the present invention; it can be seen from Table 4 that the model provided by the present invention has a significant effect on the improvement of inference speed. In addition, since Ours-2 needs to use an embedding model, the inference speed is not as good as Ours-1;

[0152] Table 4. Comparison of inference speed

[0153] In order to fairly and reasonably compare the reasoning performance of the model of the present invention and the fusion framework, the selected benchmark models are divided into three categories: embedding-based models, description-based models and joint models; among them, the embedding-based models include: the transformation embedding model (TransE) for modeling multi-relational data, the knowledge graph embedding model based on complex spatial relationship rotation (RotatE), the bilinear diagonal model (DistMult), and the convolutional two-dimensional knowledge graph embedding model (ConvE); the description-based models include: the BERT-based knowledge graph embedding algorithm (KG-BERT), the multi-task knowledge graph completion algorithm (MTL_KGC); the joint models include: the pre-trained knowledge graph embedding model (Pretrain-KGE), the structure-enhanced knowledge graph text representation learning (StAR), and the joint language semantics and structure embedding model (LASS) for knowledge graph completion.

[0154] Referring to Table 5, Table 5 shows the specific experimental results of Ours-1 and Ours-2 provided by the present invention on the WN18RR dataset and UMLS dataset, respectively. On the WN18RR dataset, MRR and Hits@1, Hits@3, and Hits@10 are used as evaluation indicators; on the UMLS dataset, MR and Hits@10 are used as evaluation indicators. In addition, except for Ours-1 and Ours-2 provided by the present invention, the rest of the experimental results are taken from the results of the original paper. In Table 5, “-” indicates data not described in the paper, bold indicates the best result under the indicator, and underline indicates the next best result under the indicator.

[0155] Table 5 Experimental results

[0156] According to the experimental results, on the UMLS dataset, Ours-1 and Ours-2 provided by the present invention achieved the best and second best results respectively; on the WN18RR dataset, Ours-1 and Ours-2 provided by the present invention achieved the best results in terms of mean reciprocal rank (MRR), average number of top-ranked nodes (Hits@1), and average number of top-three nodes (Hits@3), but our model ranked third in terms of Hits@10; through analysis, it can be concluded that the model proposed in the present invention has more advantages in predicting high rankings; from the intuitive results, it can be concluded that the framework based on pre-coding fusion of structural information and semantic information (Ours-2) is optimized in all indicators compared with the masked language model based on pre-coding without fusion (Ours-1), so the fusion framework provided by the present invention is effective.

[0157] In order to more intuitively demonstrate the significance of the knowledge graph completion algorithm, as shown in Figure 4, Figure 4 shows an example analysis of 6 missing triplets selected from the WN18RR dataset; among them, bold indicates that the prediction result is consistent with the label, and the number before the result is the ranking of the true answer; on these incomplete triplets, the results of the framework based on pre-coding that integrates structural information and semantic information (Ours-2) are better than those of the masked language model based on pre-coding (Ours-1).

[0158] Knowledge graphs can play a vital role in medical aided diagnosis. Medical knowledge graphs can be constructed by drawing information from a variety of data sources, including clinical trials, academic papers, and medical records. This helps provide more comprehensive information and offers doctors multi-dimensional data support. After constructing a structured medical knowledge graph, knowledge graph completion can fill in the gaps in the knowledge graph, providing more comprehensive and accurate medical knowledge, thereby enhancing understanding of the patient's condition and providing more precise diagnoses. This completed knowledge can provide doctors with more options and decision-making support. When faced with complex cases, doctors can rely on a more comprehensive and detailed knowledge graph to make more informed diagnostic and treatment decisions.

[0159] Knowledge graphs may contain incomplete information, especially when it comes to new diseases, treatments or the latest research advances; knowledge graph completion can provide doctors with the latest and most comprehensive medical information by integrating the latest medical literature, research data and clinical practice.

[0160] In addition, a knowledge graph with a large amount of medical information can be used to build an auxiliary medical question-and-answer system; combined with entity recognition, relationship recognition, intent inference, question-and-answer matching, language generation and other technologies in natural language processing, the question-and-answer system can give suggestive answers to questions raised by patients, provide real-time information, and help patients better understand their condition, symptoms and treatment options; for common questions raised by patients that are not difficult or complicated, the system can automatically reply to reduce the burden on doctors to answer, allowing doctors to focus more on complex cases and clinical decisions; in some areas with insufficient medical resources, the medical question-and-answer system can use the Internet to spread medical knowledge more widely and make up for the problem of insufficient medical resources.

[0161] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A knowledge graph link prediction method, characterized in that: include: Obtain a data set; wherein the data set includes an entity description text set and a relationship description text set; obtain an index value corresponding to each entity description text and construct an index set; The data set is input into the initialized precoding model, and each entity description text and each relationship description text are preprocessed respectively; a pooling operation is performed on the preprocessed entity description text and each relationship description text to obtain the entity semantic expression of each entity description text and the relationship semantic expression of each relationship description text; an embedding operation is performed on all entity semantic expressions and all relationship semantic expressions to obtain entity embedding vectors and relationship embedding vectors; Based on the entity embedding vector and the relationship embedding vector, combined with the index set, a triple set is constructed, and the triple set is divided into a training set and a test set. The inverse relationship triples corresponding to the training set are added to the training set to obtain the target training set; the inverse relationship triples corresponding to the test set are added to the test set to obtain the target test set; Initialize the main mask model and extract samples from the target training set according to the preset batch size to obtain a sample set; construct each mask triplet corresponding to each sample in the sample set; add the corresponding position vector to each element in each mask triplet, and combine it with the start and end flag vectors of the main mask model to obtain the input sequence corresponding to each sample; The input sequences corresponding to each sample are simultaneously input into the main mask model. After being processed by multiple layers of Transformer blocks, the vectors at the last mask position corresponding to each input sequence are output. The vectors at the last mask position corresponding to each input sequence are input into the classification layer. After a softmax operation, the probability of each entity distribution corresponding to each input sequence is output. Construct a loss function and use the Adam optimizer to minimize the loss function value, update the parameters of the main mask model, and obtain the parameters of the main mask model of the current round after extracting all samples in the target training set. Use the evaluation function to evaluate the main mask model of the current round. If the performance of the main mask model of the current round is better than that of the main mask model of the previous round, save the weight of the main mask model of the current round and conduct the next round of training until the preset number of training rounds is reached. End the training and obtain the trained main mask model. Based on the target test set, construct the test set mask triplet to obtain the test set input sequence; The test set input sequence is input into the trained main mask model, and the entity classification probability is output; the potential entities are predicted through the entity classification probability.

2. A knowledge graph link prediction method according to claim 1, characterized in that: The data set is input into the initialized precoding model, and each entity description text and each relationship description text are preprocessed respectively; a pooling operation is performed on each preprocessed entity description text and each relationship description text to obtain an entity semantic expression of each entity description text and a relationship semantic expression of each relationship description text; an embedding operation is performed on all entity semantic expressions and all relationship semantic expressions to obtain entity embedding vectors and relationship embedding vectors, including: Through the pooling operation, the entity description text and the relationship description text are summarized into semantic information features, which are expressed as follows: Among them, entity_des represents entity description; relation_des represents relationship description; Represents the description text of the i-th entity; Represents the j-th relationship description text; e i Represents the semantic expression of the i-th entity, i∈[1,n], n represents the number of entities; r j Represents the jth relation semantic expression, j∈[1,m], m represents the number of relations; Embed the entity semantic expression and the relationship semantic expression to obtain the entity embedding vector and the relationship embedding vector, which are expressed as follows: E=[e1;e2;...;e n ] R＝[r1；r2；...；r m ] Embeddings=[E,R] Where n represents the number of entities; m represents the number of relations; E represents the entity embedding vector; R represents the relationship embedding vector; Embeddings = [E, R] represents the embedding layer vector.

3. A knowledge graph link prediction method according to claim 1, characterized in that: The method constructs mask triples corresponding to each sample in the sample set; adds corresponding position vectors to each element in each mask triple, and combines the start and end flag vectors of the main mask model to obtain input sequences corresponding to each sample, including: Select a sample A from the sample set and construct the mask triplet corresponding to sample A; Add the corresponding position vector to each element in the corresponding mask triplet, and combine it with the start and end flag vectors of the main mask model to obtain the input sequence corresponding to sample A, which is expressed as: Among them, x input represents the input sequence corresponding to sample A; E CLS Indicates the start symbol of the sequence; E SEP Indicates the end symbol of the sequence; E h Indicates the head entity feature representation corresponding to sample A; E r Indicates the relationship feature representation corresponding to sample A; E mask represents the mask vector; Indicates the position code corresponding to the s-th position of the sequence; Based on the above operations, each mask triple corresponding to each sample is constructed at the same time, and each input sequence corresponding to each sample is obtained.

4. A knowledge graph link prediction method according to claim 3, characterized in that: The input sequences corresponding to the samples are simultaneously input into the main mask model, processed by multiple layers of Transformer blocks, and the vectors at the last mask position corresponding to the input sequences are output; the vectors at the last mask position corresponding to the input sequences are input into the classification layer, and after the softmax operation, the probability of the entity distribution corresponding to the input sequences is output, including: Perform the following operations on the input sequence corresponding to sample A in the sample set: The input sequence corresponding to sample A is input into the main mask model. The input sequence corresponding to sample A will be processed by the 12-layer Transformer block; the last layer E mask The vector at the position can be used as an expression with a predicted entity, and its expression is: z num =Transformer(z num-1 ),num=1,2,...,12 z0=x input Among them, z num Represents the input of the numth layer; z0 represents the input of the first layer; Indicates the last layer E mask vectors on positions; Indicates the final output E mask The value of The classification layer is a feedforward neural network, and the input dimension of the classification layer is The output dimension is the number of entities; Input into the classification layer, and then through the softmax operation, the entity classification probability is obtained, and its expression is: in, represents the entity classification probability; Based on the above operations, each input sequence corresponding to each sample in the sample set is simultaneously input into the main mask model, and the distribution probability of each entity corresponding to each input sequence is output.

5. A knowledge graph link prediction method according to claim 1, characterized in that: The construction of the loss function includes: Based on the distribution probability of each entity corresponding to each input sequence, a loss function is constructed; wherein the loss function is a cross entropy loss function, and its expression is: Among them, y i represents the actual label of the i-th entity, y i Takes 0 or 1; when y i When 1 is taken, it means that the i-th entity is a label; when y i When it is 0, it means that the i-th entity is not a label; represents the i-th probability corresponding to the i-th entity in the entity distribution probability; N represents the number of all entities; i represents the index value.

6. A knowledge graph link prediction method according to claim 1, characterized in that: Initializing the main mask model, extracting samples from the target training set according to a preset batch size, and obtaining the sample set further includes: Within a preset range, randomly initialize each entity structure embedding expression and each relationship structure embedding expression of the embedding model, and integrate each entity structure embedding expression and each relationship structure embedding expression into an entity embedding matrix and a relationship embedding matrix respectively; The entity embedding vector obtained by the precoding model contains multiple entity semantic expressions; the relationship embedding vector obtained by the precoding model contains multiple relationship semantic expressions; The entity semantic expression and relationship semantic expression corresponding to each sample in the sample set are spliced ​​with the entity structure embedding expression and relationship structure embedding expression corresponding to the embedding model to obtain the spliced ​​features corresponding to each sample; the spliced ​​features corresponding to each sample are input into the fusion module to obtain the fused head entity expression and each relationship expression corresponding to each sample; based on the fused head entity expression and each relationship expression corresponding to each sample, the fused mask triples corresponding to each sample are constructed; the corresponding position vector is added to each element in each fused mask triple, and combined with the start and end flag vectors of the main mask model, the fused input sequence corresponding to each sample is obtained; Each fused input sequence corresponding to each sample is simultaneously input into the main mask model.

7. A knowledge graph link prediction method according to claim 6, characterized in that: The entity semantic expression and relationship semantic expression corresponding to each sample in the sample set are spliced ​​with the entity structure embedding expression and relationship structure embedding expression corresponding to the embedding model to obtain each splicing feature corresponding to each sample; Input the splicing features corresponding to each sample into the fusion module to obtain the fused head entity expressions and relationship expressions corresponding to each sample, including: Select any sample B from the sample set, define the entity semantic expression and relationship semantic expression corresponding to sample B as the semantically rich description expression u from the precoding model; define the corresponding entity structure embedding expression and relationship structure embedding expression in the embedding model as the embedding expression v with structural information from the embedding model; The semantically rich description expression u from the pre-coding model and the structurally information embedded expression v from the embedding model are concatenated to obtain the concatenated feature c. The concatenated feature is the concatenation result of the two expressions, and its expression is: c(u,v)=[u,v] The splicing features corresponding to sample B are input into the fusion module, and the fusion information is extracted by dimensionality reduction to obtain the fused head entity and relationship expression, which is expressed as follows: in, Indicates the fused head entity expression corresponding to sample B; Represents the fused relational expression corresponding to sample B; Based on the above operations, each sample in the sample set is spliced ​​simultaneously to obtain each splicing feature corresponding to each sample, and each fused head entity expression and each relationship expression corresponding to each sample are obtained.

8. A knowledge graph link prediction method according to claim 7, characterized in that: The method constructs each fusion mask triple corresponding to each sample based on the fused head entity expressions and each relationship expression corresponding to each sample; adds a corresponding position vector to each element in each fusion mask triple, and combines the start and end flag vectors of the main mask model to obtain each fusion input sequence corresponding to each sample, including: Based on the fused head entity expression and relation expression corresponding to sample B, the fusion mask triplet corresponding to sample B is constructed; the corresponding position vector is added to each element in the fusion mask triplet corresponding to sample B, and combined with the start and end flag vectors of the main mask model, the fusion input sequence corresponding to sample B is obtained, which is expressed as follows: Among them, xinput_fusion represents the fusion input sequence corresponding to sample B; E CLS Indicates the start symbol of the sequence; E SEP Indicates the end symbol of the sequence; Indicates the fused head entity expression corresponding to sample B; represents the fused relational expression corresponding to sample B; E mask represents the mask vector; Indicates the position code corresponding to the s-th position of the sequence; Based on the above operations, through the fused head entity expressions and relationship expressions corresponding to each sample, each fusion mask triple corresponding to each sample is constructed at the same time, and each fusion input sequence corresponding to each sample is obtained.

9. A knowledge graph link prediction method according to claim 6, characterized in that: The embedding model is a translation model.

10. A knowledge graph link prediction method according to claim 9, characterized in that: The construction of the loss function includes: Based on the translation model principle, the scoring function is the cosine similarity function, and the sum of the embedding vectors of the head entity and the relationship is defined as e hr , whose expression is: And hr =|and h +e r | Among them, e h Represents the embedding vector of the head entity in the translation model; e r Embedding vector representing the relationship in the translation model; e t represents the embedding vector of the tail entity in the translation model; || || represents the two-norm calculation; f(h,r,t) represents the scoring function; During the training process, negative sampling is added to the translation model loss function for calculation as the contrast loss L S , so that the translation model learns an embedding representation with a positive proportion score higher than a negative proportion score. The expression is: Among them, K represents the entity set of negative sampling; t k Represents the kth entity in the negatively sampled entity set; The cross entropy loss function is used as the classification loss L CE , whose expression is: Among them, y i represents the actual label of the i-th entity, y i Takes 0 or 1; when y i When 1 is taken, it means that the i-th entity is a label; when y i When it is 0, it means that the i-th entity is not a label; represents the i-th probability corresponding to the i-th entity in the entity distribution probability; N represents the number of all entities; i represents the index value; The total loss function L optimized by the main mask model is the weighted sum of classification loss and contrast loss, and its expression is: L=L CE +αL S Among them, L CE represents the classification loss; α represents the weight of the contrast loss; L S represents contrast loss.

Images