Knowledge graph link prediction method

Abstract

The present invention relates to the technical field of knowledge graph completion tasks, and particularly relates to a knowledge graph link prediction method. The method comprises: using a precoding

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of knowledge graph completion tasks, and in particular to a knowledge graph link prediction method. BACKGROUND

[0002] A knowledge graph is a graphical model for representing and organizing knowledge, which describes a relation between entities in a form of triples. An entity includes a real-world entity, such as an object, a concept, time, etc. The relation represents connections and interactions. A composition paradigm of triples is (a head, a relation and a tail), in which the head and the tail are entities, and the relation connects the head entity and the tail entity. At present, there are many knowledge graphs of artificial mobile phones, but there are still many missing relations between entities. The knowledge graph aims to use known entities and relations to accurately predict unknown relations and enhance knowledge graph information.

[0003] The existing knowledge graph completion methods are mainly divided into: completion based on an embedding model, completion based on a description model, and a combination of graph structural information and semantic description information (a joint model). The description model uses the text description of entities and relations, uses a pre-training language model to represent description encoding as features, and uses features to match missing parts, such as a Knowledge Graph based on Bidirectional Encoder Representations from Transformers (KG-BERT) and Multi-Task Learning for Knowledge Graph Completion (MTL-KGC). Although such a model can make use of information outside the graph structure, the text needs to be directly input into the pre-training language model, which leads to a lot of memory and time for training and a significant slowdown in reasoning speed. There is little research on a joint model, and the existing research does not make full use of the structural information in the graph. Most models directly align the description features into the graph structure, but improper alignment methods may destroy the text description information.

[0004] To sum up, the problems existing in a knowledge graph completion algorithm in the prior art are as follows. (1) The description-based knowledge completion algorithm relies on the pretraining language model, but text encoding of the pre-training language model may seriously affect the reasoning speed. (2) When fusing the combination information and the semantic description information, the description features are directly aligned into the graph structure, and an improper alignment method may destroy the text description information. SUMMARY

[0005] Therefore, the technical problem to be solved by the present disclosure is to overcome the problem that the description-based knowledge completion algorithm in the prior art relies on the pre-training language model, but text encoding of the pre-training language model may seriously affect the reasoning speed, when fusing the combination information and the semantic description information, the description features are directly aligned into the graph structure, and an improper alignment method may destroy the text description information.

[0006] In order to solve the above technical problems, the present disclosure provides a knowledge graph link prediction method, including:

[0007] acquiring a data set; where the data set includes an entity description text set and a relation description text set; acquiring an index value corresponding to each entity description text, and constructing an index set;

[0008] inputting the data set into an initialized precoding model, and preprocessing each entity description text and each relation description text, respectively; performing a pooling operation on each entity description text and each relation description text which have been preprocessed to obtain an entity semantic expression of each entity description text and a relation semantic expression of each relation description text; and performing an embedding operation on all entity semantic expressions and all relation semantic expressions to obtain an entity embedding vector and a relation embedding vector;

[0009] constructing a triplet set in combination with the index set according to the entity embedding vector and the relation embedding vector, dividing the triplet set into a training set and a test set, adding an inverse relation triplet corresponding to the training set to the training set to obtain a target training set; and adding an inverse relation triplet corresponding to the test set to the test set to obtain a target test set;

[0010] initializing a primary mask model, and extracting a sample from the target training set according to a preset batch size to obtain a sample set; constructing each mask triplet corresponding to each sample in the sample set; adding a corresponding position vector to each element in each mask triplet to obtain each input sequence corresponding to each sample in combination with start and end flag vectors of the primary mask model;

[0011] inputting each input sequence corresponding to each sample into the primary mask model simultaneously, and outputting a vector at a mask position of the last layer corresponding to each input sequence after multi-layer Transformer block processing; inputting the vector at the mask position of the last layer corresponding to each input sequence into a classification layer, and outputting each entity distribution probability corresponding to each input sequence through softmax operation;

[0012] constructing a loss function, minimizing a value of the loss function by using an Adam optimizer, and updating a parameter of the primary mask model until all samples in the target training set are extracted to obtain the parameter of the primary mask model in a current epoch; evaluating the primary mask model in the current epoch by using an evaluation function, if the performance of the primary mask model in the current epoch is better than that of the primary mask model in a previous epoch, saving a weight of the primary mask model in the current epoch, carrying out a next epoch of training until a preset number of training epochs is reached, and ending the training to obtain a trained primary mask model; and

[0013] constructing a test set mask triplet based on the target test set to obtain a test set input sequence; inputting the test set input sequence into the trained primary mask model, and outputting an entity classification probability; and predicting a potential entity through the entity classification probability.

[0014] Preferably, the step of: inputting the data set into an initialized precoding model, and preprocessing each entity description text and each relation description text, respectively; performing a pooling operation on each entity description text and each relation description text which have been preprocessed to obtain an entity semantic expression of each entity description text and a relation semantic expression of each relation description text; and performing an embedding operation on all entity semantic expressions and all relation semantic expressions to obtain an entity embedding vector and a relation embedding vector, includes:

[0015] summarizing the entity description text and the relation description text into semantic expressions through the pooling operation, in which the expression is as follows:

[0016] et = Pool(BertEncoder(x^ntltyydes))

[0017] Tj = Pool(B ert Encoder (xjelation-desy)

[0018] where entityjdes denotes entity description; relation_d.es denotes relation description; denotes an i-th entity description text; xJelatlon^des denotes a j-th relation description text; et denotes an i-th entity semantic expression, i 6 [1, n], n denotes the number of entities; r, denotes a j-th relation semantic expression, j 6 [1, m], and m denotes the number of relations;

[0019] performing an embedding operation on the entity semantic expression and the relation semantic expression to obtain an entity embedding vector and a relation embedding vector, in which the expression is as follows:

[0020] E = [ei;e2;...;en]

[0021] R = [ri;r2;...;rm]

[0022] Embeddings = [E, / ?]

[0023] where n denotes the number of entities; m denotes the number of relations; E denotes an entity embedding vector; R denotes a relation embedding vector; and Embeddings = [E, / ?] denotes an embedding layer vector.

[0024] Preferably, the step of: constructing each mask triplet corresponding to each sample in the sample set; and adding a corresponding position vector to each element in each mask triplet to obtain each input sequence corresponding to each sample in combination with start and end flag vectors of the primary mask model, includes:

[0025] selecting any one sample A from the sample set, and constructing a mask triplet corresponding to the sample A; and adding the corresponding position vector to each element in the mask triplet corresponding to the sample A to obtain an input sequence corresponding to the sample A in combination with start and end flag vectors of the primary mask model, in which the expression is as follows: |0026| V"”” = [£c„ + E^. Eh + E^, Er + Emast + E^, ESISP + E^]

[0027] where xinput denotes an input sequence corresponding to the sample A; ECLS denotes a start symbol of the sequence; Esep denotes an end symbol of the sequence; Eh denotes a head entity feature representation corresponding to the sample A; Er denotes the relation feature representation corresponding to the sample A ; Emask denotes a mask vector; Efos denotes position encoding corresponding to an s-th position of the sequence;

[0028] constructing each mask triplet corresponding to each sample simultaneously based on the above operations, and obtaining each input sequence corresponding to each of the remaining samples.

[0029] Preferably, the step of: inputting each input sequence corresponding to each sample into the primary mask model simultaneously, and outputting a vector at a mask position of the last layer corresponding to each input sequence after multi-layer Transformer block processing; inputting the vector at the mask position of the last layer corresponding to each input sequence into a classification layer, and outputting each entity distribution probability corresponding to each input sequence through softmax operation, includes:

[0030] performing the following operations on the input sequence corresponding to the sample A in the sample set:

[0031] inputting the input sequence corresponding to the sample A into the primary mask model, in which the input sequence corresponding to the sample A is subjected to 12-layer Transformer block processing, a vector at the Emask position of the last layer is used as the expression of the entity to be predicted, and the expression is as follows:

[0032] znum = Trans f ormer (znum_f),num = 1,2,...,12

[0033] zQ = (00341 Fout, = z12 , ^mask ^mask

[0035] where znum denotes the input of a num-th layer; z0 denotes the input of a first layer; zmask denotes the vector at the Emask position of the last layer; and E™fsk denotes the value of the finally output Emask \

[0036] using the classification layer as a feedforward neural network, where an input dimension of the classification layer is a dimension of E^sk, and an output dimension is the number of entities; inputting E^sk into the classification layer, and then obtaining the entity classification probability through softmax operation, in which the expression is as follows:

[0037] y = softmax(MLP(E™fsk))

[0038] where y denotes an entity classification probability;

[0039] inputting each input sequence corresponding to each sample in the sample set into the primary mask model simultaneously based on the above operations, and outputting each entity distribution probability corresponding to each input sequence.

[0040] Preferably, constructing a loss function includes:

[0041] constructing a loss function based on each entity distribution probability corresponding to each input sequence; where the loss function is a cross entropy loss function, in which the expression is as follows:

[0042] Lce = -^=1yt log^i)

[0043] where yt denotes an actual label of the i-th entity, and yt takes the value of 0 or 1; when yi takes the value of 1, it indicates that the i-th entity is a label; when yt takes the value of 0, it indicates that the i-th entity is not a label; yt denotes the i-th probability corresponding to the i-th entity in the entity distribution probability; N denotes the number of all entities; and i denotes an index value.

[0044] Preferably, after initializing a primary mask model, and extracting a sample from the target training set according to a preset batch size to obtain a sample set, the method further includes:

[0045] randomly initializing each entity structure embedding expression and each relation structure embedding expression of the embedding model within a preset range, and integrating each entity structure embedding expression and each relation structure embedding expression into an entity embedding matrix and a relation embedding matrix, respectively;

[0046] containing a plurality of entity semantic expressions in the entity embedding vector obtained by the precoding model; and containing a plurality of relation semantic expressions in the relation embedding vector obtained by the precoding model;

[0047] splicing the entity semantic expression and the relation semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and the relation structure embedding expression corresponding to the embedding model to obtain each splicing feature corresponding to each sample; inputting each splicing feature corresponding to each sample into a fusion module to obtain each fused head entity expression and each fused relation expression corresponding to each sample; constructing each fusion mask triplet corresponding to each sample according to the each fused entity expression and each fused relation expression corresponding to each sample; and adding a corresponding position vector to each element in each fusion mask triplet to obtain each fusion input sequence corresponding to each sample in combination with the start and end flag vectors of the primary mask model; and

[0048] inputting each fusion input sequence corresponding to each sample into the primary mask model simultaneously.

[0049] Preferably, the step of: splicing the entity semantic expression and the relation semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and the relation structure embedding expression corresponding to the embedding model to obtain each splicing feature corresponding to each sample; and inputting each splicing feature corresponding to each sample into a fusion module to obtain each fused head entity expression and each fused relation expression corresponding to each sample, includes:

[0050] selecting any one sample B from the sample set, and defining the entity semantic expression and the relation semantic expression corresponding to the sample B as a description expression it rich in semantic information from the precoding model; defining the corresponding entity structure embedding expression and the relation structure embedding expression in the embedding model as an embedding expression v with structural information from the embedding model;

[0051] splicing the descriptive expression u rich in semantic information from the precoding model and the embedding expression v with structural information from the embedding model to obtain the splicing feature c, in which the splicing feature is a splicing result of the two expressions, and the expression is as follows:

[0052] c(u, v) = [u, v]

[0053] inputting the splicing feature corresponding to the sample B into the fusion module, and extracting fusion information in a dimension reducing manner to obtain the fused head entity and relation expression, in which the expression is as follows:

[0054] E^ = MLP(_c(uh,vh^

[0055] E? = MLP(c(ur,vry) f

[0056] where E^ denotes a fused head entity expression corresponding to the sample B; and E'. denotes a fused relation expression corresponding to the sample B:

[0057] splicing each sample in the sample set simultaneously based on the above operations to obtain each splicing feature corresponding to each sample, and obtain each fused head entity expression and each fused relation expression corresponding to each sample.

[0058] Preferably, the step of: constructing each fusion mask triplet corresponding to each sample according to the each fused entity expression and each fused relation expression corresponding to each sample; and adding a corresponding position vector to each element in each fusion mask triplet to obtain each fusion input sequence corresponding to each sample in combination with the start and end flag vectors of the primary mask model, includes:

[0059] constructing a fusion mask triplet corresponding to the sample B based on the fused head entity expression and the fused relation expression corresponding to the sample B; and adding a corresponding position vector to each element in the fusion mask triplet corresponding to the sample B to obtain the fusion input sequence corresponding to the sample B in combination with the start and end flag vectors of the primary mask model, in which the expression is as follows:

[0060] inPut-fusion _

[0061] + E^s, E' + E^s, E' + E^s, Emask + E^‘, Essp + E^s]

[0062] where xinput^uslon denotes a fusion input sequence corresponding to the sample B; Ecls denotes a start symbol of the sequence; Esep denotes an end symbol of the sequence; E^ f denotes a fused head entity expression corresponding to the sample B ; E^. denotes a fused relation expression corresponding to the sample B; Emask denotes a mask vector; and E^os denotes position encoding corresponding to an s-th position of the sequence;

[0063] constructing each fusion mask triplet corresponding to each sample simultaneously through each fused head entity expression and each fused relation expression corresponding to each sample based on the above operations, and obtaining each fusion input sequence corresponding to each sample.

[0064] Preferably, the embedding model is a translation model.

[0065] Preferably, constructing a loss function includes:

[0066] defining the sum ehr of embedding vectors of the head entity and the relation by using a score function as a cosine similarity function based on the principle of the translation model, in which the expression is as follows:

[0067] ehr = \eh + er\

[0068] f(h,r, t) = Cosine(ehr, = ..

[0069] where eh denotes an embedding vector of the head entity in the translation model; er denotes an embedding vector of the relation in the translation model; et denotes an embedding vector of a tail entity in the translation model; || || denotes the two-norm calculation; and f (h, r, t) denotes a score function;

[0070] in the training process, adding a negative sample to the loss function of the translation model for calculation as a contrast loss Ls, so that the translation model learns an embedding representation in which a positive proportional score is higher than a negative proportional score, in which the expression is as follows:

[0071] Ls = -log fth * f(hrt.

[0072] where K denotes an entity set of the negative sample; and tk denotes a fc-th entity in the entity set of the negative sample;

[0073] using a cross entropy loss function as a classification loss LCE, in which the expression is as follows:

[0074] LCB = -Z^yjog^)

[0075] where yt denotes an actual label of the i-th entity, and y, takes the value of 0 or 1; when yt takes the value of 1, it indicates that the i-th entity is a label; when yt takes the value of 0, it indicates that the i-th entity is not a label; yt denotes the i-th probability corresponding to the i-th entity in the entity distribution probability; N denotes the number of all entities; and i denotes an index value;

[0076] optimizing, by the primary mask model, a total loss function L as a weighted sum of the classification loss and the contrast loss, in which the expression is as follows:

[0077] L = LEE + (xLE

[0078] where LCE denotes a classification loss; a denotes a weight of the contrast loss; and Ls denotes a contrast loss.

[0079] Compared with the prior art, the technical solution of the present disclosure has the following beneficial effects.

[0080] (1) A knowledge graph link prediction method according to the present disclosure uses a precoding model to generalize the long text description encoding of entities and relations into features of semantic information. Because the text description may not change, the precoding operation only needs to be carried out once before the model is trained. The embedding layer obtained represents the semantic expression of all entities and relations. The embedding layer is used in both subsequent training and reasoning, thus reducing the problems resulted from long text encoding. This method innovatively uses a precoding method, which effectively reduces the training pressure of the model and improves the reasoning speed of the model.

[0081] (2) The knowledge graph link prediction method of the present disclosure considers that semantic information should be independent of structural information, and both the semantic information and the structural information are expressions from different information sources. In addition, due to being inspired by early fusion in multimodal learning, a simple fusion module is used to splice the semantic expression and the embedding expression into one expression before a primary mask model is input, and then the most suitable fusion expression is obtained through a learnable fusion module. This method does not need to align the description features into the graph structure, and does not need to consider that an improper alignment method may destroy the text description information, thus ensuring the integrity of the text description information and improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0001] In order to make the contents of the present disclosure understood more clearly, the present disclosure will be described in further detail in the following according to the detailed description of the present disclosure with reference to the attached drawings, in which:

[0002] FIG. 1 is a schematic structural diagram of a knowledge graph algorithm of a precodingbased mask language model according to the present disclosure;

[0003] FIG. 2 is an input vector of a primary mask model in a precoding-based mask language model;

[0004] FIG. 3 is a schematic diagram of a knowledge graph algorithm of a precoding-based framework of fusing the structural information and the semantic information according to the present disclosure; and

[0005] FIG. 4 is an example analysis of six missing triples selected from WN18RR data set. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0006] The present disclosure will be further described with reference to the attached drawings and specific embodiments, so that those skilled in the art can better understand and implement the present disclosure, but the given embodiments are not taken as limitations of the present disclosure.

[0007] Referring to FIG. 1, FIG. 1 is a schematic structural diagram of a knowledge graph algorithm of a precoding-based mask language model according to the present disclosure. In order to solve the problem that the training language model needs to input a long text to slow down the reasoning speed, the present disclosure uses a precoding-based mask language model to predict the knowledge graph link. The model consists of two BERT (Bidirectional Encoder Representations from Transformers). The specific steps are as follows.

[0008] S101: a data set Sd = {(dese, desr)} is acquired; where the data set Sd includes an entity description text set dese and a relation description text set desr; an index value corresponding to each entity description text is acquired, and an index set is constructed.

[0009] S102: the data set is input into an initialized precoding model, and each entity description text and each relation description text are preprocessed, respectively; a pooling operation is performed on each entity description text and each relation description text which have been preprocessed to obtain an entity semantic expression of each entity description text and a relation semantic expression of each relation description text, in which the expression is as follows:

[0010] et = Pool(BertEncoder(xfnMyjlesy)

[0011] Tj = Pool(BertEncoder(x™lation^

[0012] where entity_des denotes entity description; relation_des denotes relation description; x^ntlty^des denotes an i-th entity description text; xJelatlon^des denotes a j -th relation description text; et denotes an i-th entity semantic expression, i G [1, n], n denotes the number of entities; Tj denotes a j-th relation semantic expression, j G [l,m], and m denotes the number of relations.

[0013] An embedding operation is performed on all of the entity semantic expressions and all of the relation semantic expressions to obtain an entity embedding vector and a relation embedding vector, in which the expression is as follows:

[0014] E = [e1;e2;...;en]

[0015] R =

[0016] Embeddings = [E, E]

[0017] where n denotes the number of entities; m denotes the number of relations; E denotes an entity embedding vector; R denotes a relation embedding vector; and Embeddings = [E, E] denotes an embedding layer vector.

[0018] SI03: a triplet set is constructed in combination with the index set according to the entity embedding vector and the relation embedding vector, the triplet set is divided into a training set Strain = and a test set Stest = {h,r,t} , an inverse relation triplet (t, r~, h) corresponding to the training set Strain is added to the training set to obtain a target training set S™ain; and an inverse relation triplet (t, r^, h) corresponding to the test set Stest is added to the test set to obtain a target test set

[0019] S104: a primary mask model is initialized, and batch_size samples are extracted from the target training set according to a preset batch size batch_size to obtain a sample set.

[0020] SI05: the step of: constructing each mask triplet corresponding to each sample in the sample set; and adding a corresponding position vector to each element in each mask triplet to obtain each input sequence corresponding to each sample in combination with start and end flag vectors of the primary mask model, includes:

[0021] selecting any one sample A from the sample set, taking the sample A as an example, and constructing a mask triplet corresponding to the sample A; and adding the corresponding position vector to each element in the mask triplet corresponding to the sample A to obtain an input sequence corresponding to the sample A in combination with start and end flag vectors of the primary mask model, in which the expression is as follows:

[0022] xmput = [ECLS + E%0S,Eh + Elos,Er + E^0S,Emask + EI°s,Esep + E?05]

[0023] where xinput denotes an input sequence corresponding to the sample A; ECLS denotes a start symbol of the sequence; Esep denotes an end symbol of the sequence; Eh denotes a head entity feature representation corresponding to the sample A; Er denotes the relation feature representation corresponding to the sample A; Emask denotes a mask vector; E^os denotes position encoding corresponding to an s-th position of the sequence;

[0024] constructing each mask triplet corresponding to each sample simultaneously based on the above operations, and obtaining each input sequence corresponding to each sample.

[0025] The input sequence of the primary mask model is shown in FIG. 2. Because the BERT model needs to take into account the relative position of each element in the input sequence in order to better capture the structure and the relation in the sequence, when using the BERT to perform a knowledge graph completion task, corresponding position encoding is added to the input of each position, which can provide the relative position information of the entity and the relation, so that the model can better understand the relation between different elements in the input sequence.

[0026] SI06: the step of: inputting each input sequence corresponding to each sample into the primary mask model simultaneously, and outputting a vector at a mask position of the last layer corresponding to each input sequence after multi-layer Transformer block processing; inputting the vector at the mask position of the last layer corresponding to each input sequence into a classification layer, and outputting each entity distribution probability corresponding to each input sequence through softmax operation, includes:

[0027] taking the sample A as an example, inputting the input sequence corresponding to the sample A into the primary mask model, in which the input sequence corresponding to the sample A is subjected to 12-layer Transformer block processing, a vector at the Emask position of the last layer is used as the expression of the entity to be predicted, and the expression is as follows:

[0028] znum = Trans f ormer (znum_f),num = 1,2,...,12

[0029] zQ = xin?ut

[0030] E^k = ^mask

[0031] where znum denotes the input of a num-th layer; z0 denotes the input of a first layer; zmask denotes the vector at the Emask position of the last layer; and E°ffsk denotes the value of the finally output Emask;

[0032] regarding link prediction (head entity —► relation —>? ) as a multi-classification problem of a masked vector; therefore, obtaining the entity classification probability by using the classification layer; using the classification layer as a feedforward neural network, where an input dimension of the classification layer is a dimension of E™fsk, and an output dimension is the number of entities; and then obtaining the entity classification probability through softmax operation, in which the expression is as follows:

[0033] y = softmax^MLP^^

[0034] where y denotes an entity classification probability;

[0035] inputting each input sequence corresponding to each sample in the sample set obtained in SI05 into the primary mask model simultaneously based on the above operations, and outputting each entity distribution probability corresponding to each input sequence.

[0036] SI07: a loss function is constructed based on each entity distribution probability corresponding to each input sequence; where the loss function is a cross entropy loss function, in which the expression is as follows:

[0037] LCE = -^iyilog(yd

[0038] where yt denotes an actual label of the i-th entity, and yt takesthe value of 0 or 1; when yt takes the value of 1, it indicates that the i-th entity is a label; when y, takes the value of 0, it indicates that the i-th entity is not a label; yt denotes the probability of the i-th entity; N denotes the number of all entities; and i denotes an index value. The objective of the loss function is to enable the probability of the correct entity to be as large as possible.

[0039] Suppose there are 40943 entities in a data set, and each entity corresponds to an index value. If the preset batch size is 256, 256 pieces of data are read each time, 256 mask triples are constructed, and the obtained input sequence is data of 256 rows and 5 columns. The input sequence is input into the primary mask model, and the probability of 256 rows and 40943 columns is finally output. The entity classification probability output by the classification layer is the probability distributed between 0 and 1 through softmax, and the sum of 40943 probabilities is 1.

[0040] SI08: a value of the loss function is minimized by using an Adam optimizer, and a parameter of the primary mask model is updated until all samples in the target training set are extracted to obtain the parameter of the primary7 mask model in a current epoch; the primary mask model in the current epoch is evaluated by using an evaluation function, if the performance of the primary mask model in the current epoch is better than that of the primary mask model in a previous epoch, a weight of the primary mask model in the current epoch is saved, a next epoch of training is carried out until a preset number of training epochs is reached, and the training ends to obtain a trained primary mask model.

[0041] S109: a test set mask triplet is constructed based on the target test set to obtain a test set input sequence; the test set input sequence is input into the trained primary mask model, and an entity classification probability is output; and a potential entity is predicted through the entity classification probability.

[0042] According to the above steps, the algorithm flow of the model can be obtained as follows:

[0043] Algorithm 1: Precoding-Based Mask Language Model

[0044] Precoding: inputting a data set

[0045] 1, initializing a precoder BERT, and loading a pre-training weight;

[0046] 2, data preprocessing:

[0047] 2.1 data cleaning: removing entities that are not in the training set;

[0048] 2.2 setting the length of the longest text; if the text length of the input set is greater than the preset length of the longest text, truncating the text length to the preset length of the longest text; otherwise, filling the text length to the preset length of the longest text;

[0049] 3, according to the pooling operation, calculating the description text expression of each entity and each relation;

[0050] 4, according to the embedding operation, constructing the embedding layer of the primary model, and obtaining the embedding layer vector;

[0051] Training: inputting the training set of the triplet set, the feature dimension dim, the learning rate Ir, the number of training epochs epochs, the number of labels label_num, and the batch size batch_size-,

[0052] 1, initializing the primary mask model, constructing a Tokenizer, constructing an inverse relation triplet, and adding a training set to obtain a target training set; and initializing the weight of the classification layer;

[0053] 2, when the number of current epochs is less than the number of training epochs;

[0054] 2.1 extracting batch size samples from the target training set;

[0055] 2.1.1 constructing a mask triplet;

[0056] 2.1.2 inputting the input sequence into the primary mask model to obtain the mask representation of the last layer;

[0057] 2.1.3 calculating the classification probability;

[0058] 2.1.4 calculating a cross entropy loss function, and using the Adam optimizer to update a model parameter;

[0059] 2.2 if not all of the samples have been extracted, returning to 2.1;

[0060] 2.3 evaluating the model, and if the model performance is the best at present, saving the weight of the model;

[0061] 2.4 the current epoch plus one;

[0062] 3, if the current epoch is greater than the number of training epochs, ending the training; otherwise, returning 2.

[0063] Inference: inputting the test set of the triplet set, the feature dimension dim, and the number of labels label_num',

[0064] 1, loading the training weight, constructing an inverse relation triplet, adding a test set to obtain the target training set; and sampling;

[0065] 2, constructing a mask triplet;

[0066] 3, adding a corresponding position vector to each vector in the mask triplet to obtain an input sequence;

[0067] 4, inputting the input sequence into the primary mask model, and outputting the entity classification probability;

[0068] 5, predicting a potential entity with the probability

[0069] Based on the above embodiment, in this embodiment, in order to improve the prediction accuracy of the knowledge graph link, based on the precoding framework of fusing the structural information and the semantic information, each fusion input sequence corresponding to each sample is acquired as the input sequence of the primary mask model.

[0070] Referring to FIG. 3, FIG. 3 is a schematic diagram of a knowledge graph algorithm of a precoding-based framework of fusing the structural information and the semantic information according to the present disclosure. The knowledge graph completion based on the framework of fusing the structural information and the semantic information consists of three stages, that is, a precoding stage, a fusion stage and an encoding stage. The specific steps are as follows.

[0071] S301: a data set Sd = {(dese, desr)} is acquired; where the data set Sd includes an entity description text set dese and a relation description text set desr; an index value corresponding to each entity description text is acquired, and an index set is constructed.

[0072] S302: the data set is input into an initialized precoding model, and each entity description text and each relation description text are preprocessed, respectively; a pooling operation is performed on each entity description text and each relation description text which have been preprocessed to obtain an entity semantic expression of each entity description text and a relation semantic expression of each relation description text, in which the expression is as follows:

[0073] et = PoolfBertEncodev^

[0074] rj = Pool(BertEncoder(Xjelation-des))

[0075] where entity_des denotes entity description; relation_des denotes relation description; xentlty~des denotes an i -th entity description text; Xj elatlon-des denotes a j-th relation description text; et denotes an i-th entity semantic expression, i G [1, n], n denotes the number of entities; rj denotes a J-th relation semantic expression, j G [l,m], and m denotes the number of relations.

[0076] An embedding operation is performed on all of the entity semantic expressions and all of the relation semantic expressions to obtain an entity embedding vector and a relation embedding vector, in which the expression is as follows:

[0077] E = [ei;e2;...;en]

[0078] R = [ri;r2;...;rm]

[0079] Embeddings = [E, / ?]

[0080] where n denotes the number of entities; m denotes the number of relations; E denotes an entity embedding vector; R denotes a relation embedding vector; and Embeddings = [E, / ?] denotes an embedding layer vector.

[0081] SI03: a triplet set is constructed in combination with the index set according to the entity embedding vector and the relation embedding vector, the triplet set is divided into a training set Strain = and a test set Stest = {h,r,t} , an inverse relation triplet h) corresponding to the training set Strain is added to the training set to obtain a target training set Spain’ and an inverse relation triplet (t, r^, h.) corresponding to the test set Stest is added to the test set to obtain a target test set S^st.

[0082] Each entity structure embedding expression and each relation structure embedding expression of the embedding model is randomly initialized within a [—6Vdim, 6Vdim] range, and each entity structure embedding expression and each relation structure embedding expression are integrated into an entity embedding matrix and a relation embedding matrix, respectively; where dim denotes the feature dimension of the embedding layer vector.

[0083] S304: a primary mask model is initialized, and batch_size samples are extracted from the target training set according to a preset batch size batch_size to obtain a sample set.

[0084] S305: the step of: splicing the entity semantic expression and the relation semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and the relation structure embedding expression corresponding to the embedding model to obtain each splicing feature; and inputting each splicing feature into a fusion module to obtain each fused head entity expression and each fused relation expression corresponding to each sample, includes:

[0085] selecting any one sample B from the sample set, taking the sample B as an example, and defining the entity semantic expression and the relation semantic expression corresponding to the sample B as a description expression u rich in semantic information from the precoding model; defining the corresponding entity structure embedding expression and the relation structure embedding expression in the embedding model as an embedding expression v with structural information from the embedding model;

[0086] splicing the descriptive expression u rich in semantic information from the precoding model and the embedding expression v with structural information from the embedding model to obtain the splicing feature c, in which the splicing feature is a splicing result of the two expressions, and the expression is as follows:

[0087] c(u, v) = [u, v]

[0088] inputting the splicing feature corresponding to the sample B into the fusion module, and extracting fusion information in a dimension reducing manner to obtain the fused head entity and relation expression, in which the expression is as follows:

[0089] E? = MLP(c(uh,

[0090] E? = MLP(c(ur,vr^

[0091] where E^ denotes a fused head entity expression corresponding to the sample B; and f E;. denotes a fused relation expression corresponding to the sample B;

[0092] splicing each sample in the sample set simultaneously based on the above operations to obtain each splicing feature corresponding to each sample, and obtain each fused head entity expression and each fused relation expression corresponding to each sample.

[0093] S306: the step of: constructing each fusion mask triplet corresponding to each sample according to the each fused entity expression and each fused relation expression corresponding to each sample; and adding a corresponding position vector to each element in each fusion mask triplet to obtain each fusion input sequence corresponding to each sample in combination with the start and end flag vectors of the primary mask model, includes:

[0094] taking the sample B as an example, constructing a fusion mask triplet corresponding to the sample B based on the fused head entity expression and the fused relation expression corresponding to the sample B; and adding a corresponding position vector to each element in the fusion mask triplet corresponding to the sample B to obtain the fusion input sequence corresponding to the sample B in combination with the start and end flag vectors of the primary mask model, in which the expression is as follows:

[0095] inPut-fusion _

[0096] [Ecls + E*os>+ E^05,El + Ep2°s,Emask + E^, Esep + E^]

[0097] where xinput-fuslon denotes a fusion input sequence corresponding to the sample B; Ecls denotes a start symbol of the sequence; Esep denotes an end symbol of the sequence; E^ denotes a fused head entity expression corresponding to the sample B, E]. denotes a fused relation expression corresponding to the sample B; Emask denotes a mask vector; and Epos denotes position encoding corresponding to an s-th position of the sequence;

[0098] constructing each fusion mask triplet corresponding to each sample simultaneously through each fused head entity expression and each fused relation expression corresponding to each sample based on the above operations, and obtaining each fusion input sequence corresponding to each sample.

[0099] S305 and S306 belong to the fusion module in the precoding-based framework of fusing the structural information and the semantic information. Through this fusion module, the prediction accuracy of the overall fusion framework can be improved by fusing the structural information and the semantic information, and the integrity of text description information can be ensured simultaneously.

[00100] S307: each fusion input sequence corresponding to each sample is input into the primary mask model simultaneously, and a vector at a mask position of the last layer corresponding to each fusion input sequence is output after multi-layer Transformer block processing; the vector at the mask position of the last layer corresponding to each fusion input sequence is input into a classification layer, and each entity distribution probability corresponding to each input sequence is output through softmax operation.

[00101] S308: in a specific embodiment of the present disclosure, the used embedding model is a translation model. Constructing a loss function is as follows.

[00102] The principle of the translation model is to turn the relation into the transformation between entities, and point the sum of the head entity and the relation vector to the tail entity. The performance of the expected vector is h + r « t. The sum ehr of embedding vectors of the head entity and the relation is defined by using a score function as a cosine similarity function, in which the expression is as follows:

[00103] ehr = \eh + er|

[00104] f(h,r, t) = Cosine (ehr, et) =

[00105] where eh denotes an embedding vector of the head entity in the translation model; er denotes an embedding vector of the relation in the translation model; et denotes an embedding vector of a tail entity in the translation model; || || denotes the two-norm calculation; and f (h, r, t) denotes a score function.

[00106] In the training process of a traditional translation model, a negative sample operation is usually added, which can effectively improve the generalization performance of the model, solve the imbalance problem between samples, and strengthen the robustness of the model. In the present disclosure, a negative sample is added to the loss function for calculation as a contrast loss, so that the translation model learns an embedding representation in which a positive proportional score is higher than a negative proportional score, in which the expression is as follows:

[00107] Ls = -log.............—................................i^i.........................7^.........-

[00108] where K denotes an entity set of the negative sample; and tk denotes a k-th entity in the entity set of the negative sample.

[00109] The score of the score function f(h,r,t) is between [-1,1]. The closer the score is to 1, the better the mapping between the head entity vector and the relation vector can point to the tail entity, which indicates that the performance of the model is better. In terms of predicting the tail entity (the head entity, the relation, ?), the prediction output of the translation model is the entity with the highest score, in which the expression is as follows:

[00110] argmax(ehr, e^, tresi G E tres_i

[00111] where E is the set of all entities; and tres t denotes the entity with the highest score.

[00112] A cross entropy loss function is used as a classification loss LCE, in which the expression is as follows: [00113^ = -1^ logtfd

[00114] where yt denotes an actual label of the L-th entity, and yt takes the value of 0 or 1; when yi takes the value of 1, it indicates that the i-th entity is a label; when yt takes the value of 0, it indicates that the i-th entity is not a label; yi denotes the i-th probability corresponding to the i-th entity in the entity distribution probability; N denotes the number of all entities; and i denotes an index value;

[00115] optimizing, by the primary mask model, a total loss function L as a weighted sum of the classification loss and the contrast loss, in which the expression is as follows:

[00116] L = Lce + aLs

[00117] where LCE denotes a classification loss; a denotes a weight of the contrast loss; and Ls denotes a contrast loss.

[00118] S309: a value of the loss function is minimized by using an Adam optimizer, and a parameter of the primary mask model is updated until all samples in the target training set are extracted to obtain the parameter of the primary mask model in a current epoch; the primary mask model in the current epoch is evaluated by using an evaluation function, if the performance of the primary mask model in the current epoch is better than that of the primary mask model in a previous epoch, a weight of the primary mask model in the current epoch is saved, a next epoch of training is carried out until a preset number of training epochs is reached, and the training ends to obtain a trained primary mask model.

[00119] S310: a test set fusion mask triplet is constructed based on the target test set to obtain a test set fusion input sequence; the test set fusion input sequence is input into the trained primary mask model, and an entity classification probability is output; and a potential entity is predicted through the entity classification probability.

[00120] According to the above steps, the algorithm flow of the model can be obtained as follows:

[00121] Algorithm 2: Precoding-Based Framework of Fusing the Structural Information and the Semantic Information

[00122] Precoding: inputting a data set

[00123] !, initializing a precoder BERT, and loading a pre-training weight;

[00124] 2, data preprocessing:

[00125] 2.1 data cleaning: removing entities that are not in the training set;

[00126] 2.2 setting the length of the longest text; if the text length of the input set is greater than the preset length of the longest text, truncating the text length to the preset length of the longest text; otherwise, filling the text length to the preset length of the longest text;

[00127] 3, according to the pooling operation, calculating the description text expression of each entity and each relation;

[00128] 4, according to the embedding operation, constructing the embedding layer of the primary model, and obtaining the embedding layer vector;

[00129] Training: inputting the training set of the triplet set, the feature dimension dim, the learning rate Lr, the number of training epochs epochs, the number of labels label_num, and the batch size batch_size-,

[00130] 1, initializing the primary mask model, constructing a Tokenizer, constructing an inverse relation triplet, and adding a training set to obtain a target training set; initializing the entity embedding and relation embedding matrices in the translation model randomly with the preset range; initializing the weight of the classification layer; and initializing the weight of the fusion module;

[00131] 2, when the number of current epochs is less than the number of training epochs;

[00132] 2.1 extracting batch size samples from the target training set;

[00133] 2.1.1 constructing a mask triplet;

[00134] 2.1.2 acquiring the fusion representation;

[00135] 2.1.3 inputting the fusion input sequence into the primary mask model to obtain the mask representation of the last layer;

[00136] 2.1.4 calculating the classification probability;

[00137] 2.1.5 calculating the contrast loss;

[00138] 2.1.6 calculating a total loss function of the model, and using the Adam optimizer to update a model parameter;

[00139] 2.2 if not all of the samples have been extracted, returning to (1); [00140J2.3 evaluating the model, and if the model performance is the best at present, saving the weight of the model;

[00141] 2 4 the current epoch plus one;

[00142] 3, if the current epoch is greater than the number of training epochs, ending the training; otherwise, returning 2.

[00143] Inference: inputting the test set of the triplet set, the feature dimension dim, and the numb er of 1 ab el s lab e l_num

[00144] 1, loading the training weight, constructing an inverse relation triplet, adding a test set to obtain the target training set; and sampling;

[00145] 2, constructing a mask triplet;

[00146] 3, acquiring the fusion representation;

[00147] 4, inputting the fusion representation into the primary mask model, and outputting the entity classification probability;

[00148] 5, predicting a potential entity with the probability

[00149] In order to verify the reasoning performance of the precoding-based mask language model and the precoding-based framework of fusing the structural information and the semantic information, experiments are conducted on the precoding-based mask language model and the precoding-based framework of fusing the structural information and the semantic information.

[00150] !. Experimental data set

[00151] The experiment uses two benchmark data sets: WN18RR and UMLS (Unified Medical Language System). WN18RR belongs to the WordNet data subset, which describes the data set of association features between English words and keeps the symmetry, asymmetry and combination relations in the WordNet data set, and is used to evaluate the link prediction task. The UMLS (Unified Medical Language System) data set is a small knowledge graph for biomedical and health fields. The scale of the two benchmark data sets is shown in Table 1, that is,

[00152] Table 1 : Data set specification data set entity number relation number training set length verification set test set length WN18RR 40559 11 86835 2824 2924 UMLS 135 46 5216 652 661

[00153] It should be noted that the entity has filtered out the triples that do not appear in the training set, and the entities and relations in the verification set and the test set have also appeared in the training set. The data in the verification set and the test set do not participate in model optimization, and the subsequent experimental conclusions are all the evaluation results of the two models on the test set. The text description of the entities and relations in the data set is defined by the knowledge graph embedding algorithm based on BERT (KG-BERT).

[00154] II. Experimental evaluation index

[00155] The defective triplet of the knowledge graph (the head entity, the relation, ?) may correspond to a plurality of tail entities, and it is customary to remove all possible entities (except the label entities) from all the training sets, verification sets and test sets that have appeared in the candidate triplet during evaluation. In the filtered entities, the softmax values output by the classification layer are ranked. The true answer of the i-th triplet in the test set is ranked as rankt among all entities.

[00156] 1, Mean Ranking (MR) [00157JMR refers to the mean value of all rankt in the test set, in which the expression is as follows:

[00158] MR = ^£"1ranki

[00159] The index reflects the ranking of true answers among candidate entities on average. The lower the MR, the better the reasoning performance of the model.

[00160] 2, Mean Ranking Reciprocal (MRR)

[00161] MRR refers to the mean value of the reciprocal of all rankt in the test set, in which the expression is as follows: [00162JMRR IV I CLICK. /

[00163] Compared with MR, the index pays more attention to whether the model can find the high ranking of real answers among candidate entities. The higher the MRR, the better the reasoning performance of the model.

[00164] 3, Mean number of top n nodes (Hits@n)

[00165] Hits@n refers to the mean value of all ranki <n in the test set, where n usually takes the values of 10, 3 and 1, in which the expression is as follows:

[00166] Hits@n = ^2^=1^(^^1 — n),n = 10,3,1

[00167] The index model reflects whether the model can rank the real answers in the top n, and only pays attention to the high ranking performance of the model. The larger the Hits@n, the better the reasoning performance of the model.

[00168] III. Experimental environment

[00169] The experimental environment may affect the experimental results. The experimental environment is shown in Table 2 for research reference, that is,

[00170] Table 2: Experimental environment Experimental environment Specification Processor Inter(R) Core(TM) i7-6850K CPU @ 3.60GHZ Graphics card NVIDIA TITAN Xp 12G Memory 128GDDR4 Development language Python 3.9.7 Development system Ubuntu 21.10 Development framework Pytorch 1.11.0 + cudal 0.2

[00171] IV. Parameter setting

[00172] The parameter setting may affect the experimental result. The detailed parameter setting in the experiment is shown in Table 3, that is,

[00173] Table 3: Parameter Setting Parameter Specification Precoding characterizing dimension 768 dimensions Primary mask characterizing dimension 768 dimensions Translation model characterizing dimension 768 dimensions Negative sampling size of the embedding model WN18RR: 2048 UMLS: 134 Learning rate of the primary mask WN18RR: 3e-5 UMLS: le-5 Learning rate of the embedding model WN18RR: le-3 UMLS: le-4 Number of training epochs WN18RR: 200 epochs UMLS: 1000 epochs Loss parameter WN18RR: 2.0 UMLS: 0.5 Training batch size 256 Number of self-attention heads of the primary mask model 4 Number of attention layers of the primary mask model 12 Length of the longest text allowed to be input by the precoding model 128 Device GPU

[00174] V. Experimental results

[00175] In order to compare the reasoning speed advantage of the precoding mechanism in the present disclosure, KG-BERT is selected in the experiment to compare with the precoding-based mask language model and the precoding-based framework of fusing the structural information and the semantic information according to the present disclosure in terms of WN18RR and UMLS test sets. The input of KG-BERT is the text, while each input of the model according to the present disclosure is the precoding expression. The reasoning speed of KG-BERT is related to the scale of entities in the data set. The more entities, the longer the reasoning speed. Referring to Table 4, Table 4 shows the acceleration inference effect due to avoiding encoding the long text under the precoding mechanism. Ours-1 denotes a precoding-based mask language model according to the present disclosure, and Ours-2 denotes a precoding-based framework for fusing the structural information and the semantic information according to the present disclosure. As can be seen from Table 4, the model according to the present disclosure has a significant effect on the improvement of the reasoning speed. In addition, because Ours-2 needs to use the embedding model, the reasoning speed is not as good as OURS-1.

[00176] Table 4: Comparison of reasoning speeds Model WN18RR reasoning speed (seconds per sample) UMLS reasoning speed (seconds per sample) KG-BERT 387.252 1.31 Ours-1 0.011 0.010 Ours-2 0.013 0.011

[00177] In order to compare the reasoning performances of the model and the fusion framework fairly and reasonably, the selected benchmark models are divided into three categories: an embedding-based model, a description-based model, and a joint model. The embedding-based model includes: a transformation embedding model for modeling multi-relation data, a knowledge graph embedding model based on complex spatial relation rotation (RotatE), a bilinear diagonal model (DistMult), and a convolution two-dimensional knowledge graph embedding model (Conve). The description-based model includes: a knowledge graph embedding algorithm based on BERT (KG-BERT) and a multi-task knowledge graph completion algorithm (MTL _ KGC). The joint model includes: a pre-training knowledge graph embedding model (Pretrain-KGE), a structure-augmented knowledge graph text representation learning (StAR), and a joint language semantic and structure embedding model (LASS) for knowledge graph completion.

[00178] Referring to Table 5, Table 5 shows the specific experimental results of Ours-1 and Ours-2 on the WN18RR data set and the UMLS data set, respectively. On the WN18RR data set, MRR and Hits@l, Hits@3, and Hits@10 are used as evaluation indexes. On the UMLS data set, MR and Hits@10 are used as evaluation indexes. In addition, except for Ours-1 and Ours-2 provided by the present disclosure, other experimental results are taken from the original paper results; and further, the "-"in Table 5 denotes the data that is not explained in the paper, in which a bold part indicates the best result under this index, and an underlined part indicates the next good result of this index.

[00179] Table 5: Experimental results Data set WN18RR UMLS Evaluation MRR Hits@ 1 Hits@3 Hits@10 MR Hits@ 10 index Embedding-based model TransE 0.243 0.043 0.441 0.532 1.84 0.989 RotatE 0.470 0.443 0.482 0.526 — — DistMult 0.430 0.390 0.440 0.490 5.52 0.846 ConvE 0.456 0.419 0.470 0.531 1.51 0.990 Description-based model KG-BERT 0.219 0.095 0.243 0.497 1.47 0.990 MTL-KGC 0.331 0.203 0.383 0.597 — — Joint model Pretrain- 0.235 — — 0.557 — — KGE StAR 0.364 0.222 0.436 0.647 1.49 0.991 LASS-base — — — 0.725 1.39 0.991 The present disclosure Ours-1 0.492 0.459 0.503 0.555 1.39 0.994 Ours-2 0.516 0.474 0.532 0.601 1.29 0.995

[00180] According to the experimental results, it can be concluded that on the UMLS data set, Ours-1 and Ours-2 provided by the present disclosure have achieved the best effect and the second best effect, respectively. On the WN18RR data set, Ours-1 and Ours-2 provided by the present disclosure have achieved the best results in terms of three indexes, that is, the mean ranking reciprocal (MRR), the mean number of top-ranked nodes (Hits@l), and the mean number of the top three nodes (Hits@3), but the model according to the present disclosure has an effect of ranking third in terms of the Hits@10 index. Through analysis, it can be considered that the model according to the present disclosure has more advantages in terms of predicting high rankings. From the intuitive results, it can be considered that compared with the precoding-based mask language model without fusion (Ours-1), the precoding-based framework of fusing the structural information and the semantic information (Ours-2) has achieved improvements in all indexes, so that the fusion framework according to the present disclosure is effective.

[00181] In order to show the significance of the knowledge graph completion algorithm more intuitively, refer to FIG. 4, which shows an example analysis of six missing triples selected from WN18RR data set, in which a bold part indicates that the prediction result is consistent with the label, and the number before the result is the ranking of the real answer. On these incomplete triples, the results of the precoding-based framework of fusing the structural information and the semantic information (Ours-2) are better than those of the precoding-based mask language model (Ours-1).

[00182] The knowledge graph can play an important role in medical auxiliary diagnosis. The construction of the medical knowledge graph can acquire information from a variety of data sources, including clinical trials, academic papers, medical records and so on, which helps to provide more comprehensive information and provide more-dimensional data support for doctors. After constructing a structured medical knowledge graph, knowledge graph completion can fill the information loss in the knowledge graph and provide more comprehensive and accurate medical knowledge, thus enhancing the understanding of patient conditions and providing more accurate diagnosis. Complete knowledge can provide more choices and decision support for doctors. In the face of complex cases, doctors can rely on a more comprehensive and detailed knowledge graph to make more informed diagnosis and treatment decisions.

[00183] There may be incomplete information in the knowledge graph, especially when new diseases, treatment methods or the latest research progress are involved. Knowledge graph completion can provide doctors with the latest and comprehensive medical information by integrating the latest medical literatures, research data and clinical practice.

[00184] In addition, a knowledge graph with a large amount of medical information can be used to construct an auxiliary medical question-answering system. In combination with the technologies of entity recognition, relation recognition, intention inference, question-answering matching and language generation in natural language processing, the question-answering system can give suggestive answers to questions raised by patients and provide instant information to help patients better understand their illness, symptoms and treatment options. The system can automatically reply to the common questions of non-incurable diseases raised by patients, so as to reduce the burden on doctors of answering such questions and enable doctors to focus more on complex cases and clinical decisions. In some areas with insufficient medical resources, the medical questionanswering system can spread medical knowledge more widely through the Internet and make up for the shortage of medical resources.

[00185] Obviously, the foregoing embodiment is only an example for clear explanation, rather than a limitation of the implementation. For those skilled in the art, other changes or modifications in different forms can be made on the basis of the above description. It is not necessary and impossible to exhaust all the embodiments here. However, the obvious changes or modifications derived therefrom still fall within the scope of protection of the present disclosure.

Claims

WHAT IS CLAIMED IS:

1. A knowledge graph link prediction method, comprising:acquiring a data set; wherein the data set comprises an entity description text set and a relation description text set; acquiring an index value corresponding to each entity description text, and constructing an index set;inputting the data set into an initialized precoding model, and preprocessing each entity description text and each relation description text, respectively; performing a pooling operation on each entity description text and each relation description text which have been preprocessed to obtain an entity semantic expression of each entity description text and a relation semantic expression of each relation description text; and performing an embedding operation on all entity semantic expressions and all relation semantic expressions to obtain an entity embedding vector and a relation embedding vector;constructing a triplet set in combination with the index set according to the entity embedding vector and the relation embedding vector, dividing the triplet set into a training set and a test set, adding an inverse relation triplet corresponding to the training set to the training set to obtain a target training set; and adding an inverse relation triplet corresponding to the test set to the test set to obtain a target test set;initializing a primary mask model, and extracting a sample from the target training set according to a preset batch size to obtain a sample set; constructing each mask triplet corresponding to each sample in the sample set; adding a corresponding position vector to each element in each mask triplet to obtain each input sequence corresponding to each sample in combination with start and end flag vectors of the primary mask model;inputting each input sequence corresponding to each sample into the primary mask model simultaneously, and outputting a vector at a mask position of the last layer corresponding to each input sequence after multi-layer Transformer block processing; inputting the vector at the mask position of the last layer corresponding to each input sequence into a classification layer, and outputting each entity distribution probability corresponding to each input sequence through softmax operation;constructing a loss function, minimizing a value of the loss function by using an Adam optimizer, and updating a parameter of the primary mask model until all samples in the target training set are extracted to obtain the parameter of the primary mask model in a current epoch; evaluating the primary mask model in the current epoch by using an evaluation function, if theperformance of the primary mask model in the current epoch is better than that of the primary mask model in a previous epoch, saving a weight of the primary mask model in the current epoch, carrying out a next epoch of training until a preset number of training epochs is reached, and ending the training to obtain a trained primary mask model; andconstructing a test set mask triplet based on the target test set to obtain a test set input sequence; inputting the test set input sequence into the trained primary mask model, and outputting an entity classification probability; and predicting a potential entity through the entity classification probability.

2. The knowledge graph link prediction method according to claim 1, wherein the step of inputting the data set into an initialized precoding model, and preprocessing each entity description text and each relation description text, respectively; performing a pooling operation on each entity description text and each relation description text which have been preprocessed to obtain an entity semantic expression of each entity description text and a relation semantic expression of each relation description text; and performing an embedding operation on all entity semantic expressions and all relation semantic expressions to obtain an entity embedding vector and a relation embedding vector, comprises:summarizing the entity description text and the relation description text into semantic information features through the pooling operation, in which the expression is as follows:et = Pool(BertEncoder(xfnM^Tj = Pool(EertEncoder(xJelatl°n-desy)where entity_des denotes entity description; relation_des denotes relation description; x^ntity_des jenoteg an j enpty description text; elatlon-des denotes a j -th relation description text; et denotes an i -th entity semantic expression, i G [1, n], n denotes the number of entities; Tj denotes a j-th relation semantic expression, j G [l,m], and m denotes the number of relations;performing an embedding operation on the entity semantic expression and the relation semantic expression to obtain an entity embedding vector and a relation embedding vector, in which the expression is as follows:E [gj, 62,..., en]R =Embeddings = [£", / ?]where n denotes the number of entities; m denotes the number of relations; E denotes an entity embedding vector; R denotes a relation embedding vector; and Embeddings = [E, / ?]denotes an embedding layer vector.

3. The knowledge graph link prediction method according to claim 1, wherein the step of: constructing each mask triplet corresponding to each sample in the sample set; and adding a corresponding position vector to each element in each mask triplet to obtain each input sequence corresponding to each sample in combination with start and end flag vectors of the primary mask model, comprises:selecting any one sample A from the sample set, and constructing a mask triplet corresponding to the sample A; and adding the corresponding position vector to each element in the mask triplet corresponding to the sample A to obtain an input sequence corresponding to the sample A in combination with start and end flag vectors of the primary mask model, in which the expression is as follows:Yinput — rr । pP0S pt । pPos p । pP0S p । pP0S p । pP0Si co ’ c2 ’ ‘-‘mask ' >‘-'SEP c4 Jwhere xinput denotes an input sequence corresponding to the sample A, ECLS denotes a start symbol of the sequence; Esep denotes an end symbol of the sequence; Eh denotes a head entity feature representation corresponding to the sample A; Er denotes the relation feature representation corresponding to the sample A; Emask denotes a mask vector; E^os denotes position encoding corresponding to an s-th position of the sequence;constructing each mask triplet corresponding to each sample simultaneously based on the above operations, and obtaining each input sequence corresponding to each sample.

4. The knowledge graph link prediction method according to claim 3, wherein the step of: inputting each input sequence corresponding to each sample into the primary mask model simultaneously, and outputting a vector at a mask position of the last layer corresponding to each input sequence after multi-layer Transformer block processing; inputting the vector at the mask position of the last layer corresponding to each input sequence into a classification layer, and outputting each entity distribution probability corresponding to each input sequence through softmax operation, comprises:performing the following operations on the input sequence corresponding to the sample A in the sample set:inputting the input sequence corresponding to the sample A into the primary mask model, in which the input sequence corresponding to the sample A is subjected to 12-layer Transformer block processing, a vector at the Emask position of the last layer is used as the expression of the entity to be predicted, and the expression is as follows:znUm = Transformer(znum_J, num = 1,2,...,127 _ Y input'out _ „12 mask ^maskwhere znum denotes the input of a num-th layer; zQ denotes the input of a first layer; zmask denotes the vector at the Emask position of the last layer; and E^sk denotes the value of the finally output Emask,using the classification layer as a feedforward neural network, wherein an input dimension of the classification layer is a dimension of E^sk, and an output dimension is the number of entities; inputting E^sk into the classification layer, and then obtaining the entity classification probability through softmax operation, in which the expression is as follows:y = softmax(MLP(E°dlatsk')')where y denotes an entity classification probability;inputting each input sequence corresponding to each sample in the sample set into the primary mask model simultaneously based on the above operations, and outputting each entity distribution probability corresponding to each input sequence.

5. The knowledge graph link prediction method according to claim 1, wherein constructing a loss function comprises:constructing a loss function based on each entity distribution probability corresponding to each input sequence; wherein the loss function is a cross entropy loss function, in which the expression is as follows:N Lce = -^yt log (yd i=lwhere y, denotes an actual label of the i-th entity, and y, takes the value of 0 or 1; when yt takes the value of 1, it indicates that the i-th entity is a label; when yt takes the value of 0, it indicates that the i-th entity is not a label; yt denotes the i-th probability corresponding to the i-th entity in the entity distribution probability; N denotes the number of all entities; and i denotes an index value.

6. The knowledge graph link prediction method according to claim 1, wherein after initializing a primary mask model, and extracting a sample from the target training set according to a preset batch size to obtain a sample set, the method further comprises:randomly initializing each entity structure embedding expression and each relation structure embedding expression of the embedding model within a preset range, and integrating each entity structure embedding expression and each relation structure embedding expression into an entity embedding matrix and a relation embedding matrix, respectively;containing a plurality of entity semantic expressions in the entity embedding vector obtained by the precoding model; and containing a plurality of relation semantic expressions in the relation 31embedding vector obtained by the precoding model;splicing the entity semantic expression and the relation semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and the relation structure embedding expression corresponding to the embedding model to obtain each splicing feature corresponding to each sample; inputting each splicing feature corresponding to each sample into a fusion module to obtain each fused head entity expression and each fused relation expression corresponding to each sample; constructing each fusion mask triplet corresponding to each sample according to the each fused entity expression and each fused relation expression corresponding to each sample; and adding a corresponding position vector to each element in each fusion mask triplet to obtain each fusion input sequence corresponding to each sample in combination with the start and end flag vectors of the primary mask model; andinputting each fusion input sequence corresponding to each sample into the primary mask model simultaneously.

7. The knowledge graph link prediction method according to claim 6, wherein the step of: splicing the entity semantic expression and the relation semantic expression corresponding to each sample in the sample set with the entity structure embedding expression and the relation structure embedding expression corresponding to the embedding model to obtain each splicing feature corresponding to each sample; and inputting each splicing feature corresponding to each sample into a fusion module to obtain each fused head entity expression and each fused relation expression corresponding to each sample, comprises:selecting any one sample B from the sample set, and defining the entity semantic expression and the relation semantic expression corresponding to the sample B as a description expression u rich in semantic information from the precoding model; defining the corresponding entity structure embedding expression and the relation structure embedding expression in the embedding model as an embedding expression v with structural information from the embedding model;splicing the descriptive expression u rich in semantic information from the precoding model and the embedding expression v with structural information from the embedding model to obtain the splicing feature c, in which the splicing feature is a splicing result of the two expressions, and the expression is as follows:c(u, v) = [u, v]inputting the splicing feature corresponding to the sample B into the fusion module, and extracting fusion information in a dimension reducing manner to obtain the fused head entity and relation expression, in which the expression is as follows:Eh = MLP (c(uh,vhy)E? = MLP(c(ur,vry)f fwhere denotes the fused head entity expression corresponding to the sample B; and Ef.denotes the fused relation expression corresponding to the sample B;splicing each sample in the sample set simultaneously based on the above operations to obtain each splicing feature corresponding to each sample, and obtain each fused head entity expression and each fused relation expression corresponding to each sample.

8. The knowledge graph link prediction method according to claim 7, wherein the step of: constructing each fusion mask triplet corresponding to each sample according to the each fused entity expression and each fused relation expression corresponding to each sample; and adding a corresponding position vector to each element in each fusion mask triplet to obtain each fusion input sequence corresponding to each sample in combination with the start and end flag vectors of the primary mask model, comprises:constructing a fusion mask triplet corresponding to the sample B based on the fused head entity expression and the fused relation expression corresponding to the sample B, and adding a corresponding position vector to each element in the fusion mask triplet corresponding to the sample B to obtain the fusion input sequence corresponding to the sample B in combination with the start and end flag vectors of the primary mask model, in which the expression is as follows: ^input_fusion _[p 4. pP0S pf 4. pPos pf 4. pP0S p , 4. pPos p 4. pP0Sl tEcLS I co ’ Ch ' C1 , T C2 , E-mask >^SEP ' C4 Jwhere xinput^uslon denotes a fusion input sequence corresponding to the sample B; ECLSpdenotes a start symbol of the sequence; Esep denotes an end symbol of the sequence; E^ denotes fa fused head entity expression corresponding to the sample B; Er denotes a fused relation expression corresponding to the sample B, Emask denotes a mask vector; and Efos denotes position encoding corresponding to an s-th position of the sequence;constructing each fusion mask triplet corresponding to each sample simultaneously through each fused head entity expression and each fused relation expression corresponding to each sample based on the above operations, and obtaining each fusion input sequence corresponding to each sample.

9. The knowledge graph link prediction method according to claim 6, wherein the embedding model is a translation model.

10. The knowledge graph link prediction method according to claim 9, wherein constructing a loss function comprises:defining the sum ehr of embedding vectors of the head entity and the relation by using ascore function as a cosine similarity function based on the principle of the translation model, in which the expression is as follows:^hr I If^r,^ = Cosine(ehr,et-) =where eh denotes an embedding vector of the head entity in the translation model; er denotes an embedding vector of the relation in the translation model; et denotes an embedding vector of a tail entity in the translation model; || || denotes the two-norm calculation; and f(h, r, t) denotes a score function;in the training process, adding a negative sample to the loss function of the translation model for calculation as a contrast loss Ls, so that the translation model learns an embedding representation in which a positive proportional score is higher than a negative proportional score, in which the expression is as follows:ef(h,r,t)ef(h,r,t) + y eWtfc) k—1where K denotes an entity set of the negative sample; and tk denotes a k-th entity in the entity set of the negative sample;using a cross entropy loss function as a classification loss LCE, in which the expression is as follows:N^ce =y< l°9(ydi=lwhere yt denotes an actual label of the i-th entity, and y; takes the value of 0 or I; when yi takes the value of 1, it indicates that the i-th entity is a label; when yt takes the value of 0, it indicates that the i-th entity is not a label; yt denotes the i-th probability corresponding to the i-th entity in the entity distribution probability; N denotes the number of all entities; and i denotes an index value;optimizing, by the primary mask model, a total loss function L as a weighted sum of the classification loss and the contrast loss, in which the expression is as follows:= ^ce + aLswhere LCE denotes a classification loss; a denotes a weight of the contrast loss; and Ls denotes a contrast loss.PCT / CN2024 / 087157A. CLASSIFICATION OF SUBJECT MATTERG06N 5 / 022(2023.01)iAccording to International Patent Classification (IPC) or to both national classification and IPCB. FIELDS SEARCHEDMinimum documentation searched (classification system followed by classification symbols)IPC:G06N 5 / -, G06N 3 / -, G06F 18 / -, G06F 40 / -Documentation searched other than minimum documentation to the extent that such documents are included in the fields searchedElectronic data base consulted during the international search (name of data base and, where practicable, search terms used) CNTXT, CNABS, DWPI, VEN, ENTXT, ENTXTC, CNKI: MWa, Kft, WM AS, Mft, RA, IMS, 'flfiAM, ffiAifiSl, AAIKA, Adamttltif, Transformer, softmax, knowledge graph, knowledge map,link, predict, entity, relation, index, pooling, embed, master mask model, position vector, loss function, evaluation function, classification probability, Adam optimizerC. DOCUMENTS CONSIDERED TO BE RELEVANTCategory* Citation of document, with indication, where appropriate, of the relevant passages Relevant to claim No. A CN 116127921 A (GUANGDONG UNIVERSITY OF TECHNOLOGY et al.) 16 May 2023 (2023-05-16) entire document 1-10 A CN 117708341 A (NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY OF PLA) 15 March 2024 (2024-03-15) entire document 1-10 A CN 113190691 A (QILU UNIVERSITY OF TECHNOLOGY) 30 July 2021 (2021-07-30) entire document 1-10 A CN 113535972 A (JILIN UNIVERSITY) 22 October 2021 (2021-10-22) entire document 1-10 A WO 2022234273 Al (HIVEMAP LTD.) 10 November 2022 (2022-11-10) entire document 1-10| | Further documents are listed in the continuation of Box C. | J | See patent family annex.* Special categories of cited documents: “T” later document published after the international filing date or priority “A” document defining the general state of the art which is not considered date and not in conflict with the application but cited to understand the to be of particular- relevance principle or theory underlying the invention “D” document cited by the applicant in the international application “X” document of particular- relevance; the claimed invention cannot be “E” earlier application orpatent but published on or after the international considered novel or cannot be considered to involve an inventive step filing date when the document is taken alone “L” document which may throw doubts on priority claim(s) or which is “Y” document of particular relevance; the claimed invention cannot be cited to establish the publication date of another citation or other considered to involve an inventive step when the document is special reason (as specified) combined with one or more other such documents, such combination “O” document referring to an oral disclosure, use, exhibition or other being obvious to a person skilled in the art means document member of the same patent family “P” document published prior to the international filing date but later than the priority date claimed Date of the actual completion of the international search 19 December 2024 Date of mailing of the international search report 25 December 2024 Name and mailing address of the ISA / CN China National Intellectual Property Administration (ISA / CN) China No. 6, Xitucheng Road, Jimenqiao, Haidian District, Beijing 100088 Authorized officer Telephone No.Form PCT / ISA / 210 (second sheet) (July 2022)INTERNATIONAL SEARCH REPORT Information on patent family membersInternational application No.PCT / CN2024 / 087157Patent document cited in search report Publication date (day / month / year) Patent family member, s) Publication date (day / month / year) CN 116127921 A 16 May 2023 None CN 117708341 A 15 March 2024 None CN 113190691 A 30 July 2021 CN 113190691 B 23 September 2022 CN 113535972 A 22 October 2021 CN 113535972 B 23 August 2022 WO 2022234273 Al 10 November 2022 None

Images