Link prediction model training method and device

Abstract

The invention relates to the field of link prediction, in particular to a training method and device of a link prediction model. The method mainly comprises the following steps: constructing data in a knowledge graph into an undirected graph, obtaining a closed sub-graph in the undirected graph, and obtaining a closed sub-graph representation matrix; gaussian noise is added to the closed subgraph representation matrix, and all types of noise are removed; obtaining a matrix sequence in a denoising process, taking each matrix in the matrix sequence as a state matrix, obtaining a vector matrix formed by embedding closed sub-graph nodes into vectors, pooling the state matrixes and the vector matrix, calculating a link probability of the link prediction model according to the pooled data, and obtaining a link prediction result of the link prediction model; and training the link prediction model based on the link probability. According to the method, noise information in the knowledge graph can be effectively filtered, node representation is enhanced, calculation overhead is reduced, and the accuracy of a link prediction task is improved.

Description

Technical Field

[0001] The present invention relates to the field of link prediction, and in particular to a training method and device for a link prediction model. Background Art

[0002] Graph link prediction is a fundamental problem in graph representation learning, which involves using existing factual knowledge in the knowledge graph to predict missing entities or relations in the graph, specifically including entity prediction and relationship prediction. Early link prediction solutions relied on hand-crafted, parameter-free heuristics, but these methods require extensive domain knowledge to be effective. Since structural information needs to be manually designed for each heuristic, in order to overcome this limitation, embedding-based methods have been proposed, which learn node embeddings based on the connections between nodes and use the embeddings to calculate similarity scores.

[0003] With the success of graph neural networks in learning graph representations, graph neural network methods have also emerged in link prediction tasks. The message passing neural network in the graph neural network can learn effective node representations through message passing, that is, predict the existence of a link by combining the node representations of two nodes in the link. After using graph neural networks to learn the graph representation of the knowledge graph, the scoring function proposed in the traditional link prediction method can be used as a prediction indicator, or existing models in other fields can be used as decoders. According to the knowledge vector representation learned in the encoding stage, the learning and prediction of the link prediction model can be evaluated.

[0004] Since the information in the knowledge graph is very sparse, it is impossible to fully utilize the information in the knowledge graph during prediction, and it may be affected by the noise or redundant information, resulting in biased results. Existing technologies use node labeling technology or node aggregation methods to enhance the feature information of nodes. However, since the size of subgraphs grows exponentially, the computational overhead will increase in denser graphs and deeper subgraphs.

[0005] In view of this, how to overcome the defects of the prior art and solve the phenomenon of result deviation caused by noise in the prior art is a problem to be solved in this technical field. Summary of the invention

[0006] In view of the above defects or improvement needs of the prior art, the present invention solves the problem of result deviation caused by noise.

[0007] The embodiment of the present invention adopts the following technical solution:

[0008] In the first aspect, the present invention provides a training method for a link prediction model, specifically: constructing the data in the knowledge graph as an undirected graph, obtaining a closed subgraph in the undirected graph, and obtaining a closed subgraph representation matrix; adding Gaussian noise to the closed subgraph representation matrix, and removing all types of noise; obtaining a matrix sequence in the denoising process, using each matrix in the matrix sequence as a state matrix, and obtaining a vector matrix composed of closed subgraph node embedding vectors, pooling the state matrix and the vector matrix, calculating the link probability of the link prediction model according to the pooled data, and training the link prediction model based on the link probability.

[0009] Preferably, the data in the knowledge graph is constructed as an undirected graph, a closed subgraph in the undirected graph is obtained, and a closed subgraph representation matrix is ​​obtained, specifically including: for each specified target node pair in the undirected graph, a neighbor node set of a specified number of hops for the target node pair is obtained, as well as an edge set from the node in the target node pair to each node in the neighbor node set, and the neighbor node set and the edge set are combined to obtain a closed subgraph of the corresponding number of hops; the adjacency matrix of the closed subgraph is calculated, and a vector matrix composed of embedded vectors of the closed subgraph nodes is calculated, and the node representation matrix is ​​calculated based on the adjacency matrix and the vector matrix.

[0010] Preferably, the adding of Gaussian noise to the closed subgraph representation matrix and removing all types of noise specifically includes: in each time step, adding Gaussian noise to the node representation matrix to obtain a state matrix; and removing the added Gaussian noise from the state matrix according to the mean and covariance of the Gaussian distribution until the denoising of the state matrix is ​​completed.

[0011] Preferably, the adding of Gaussian noise to the node representation matrix to obtain the state matrix specifically includes: the process of obtaining the state matrix from the node representation matrix is ​​described as follows:

[0012]

[0013] Among them, Z0 is the node representation matrix, Z t is the state matrix of the current time step t, I is the identity matrix, t is the current time step, is a hyperparameter, and N represents Gaussian distribution.

[0014] Preferably, removing the added Gaussian noise from the state matrix according to the mean and covariance of the Gaussian distribution specifically includes: the process of removing the added Gaussian noise is described as follows:

[0015] p θ (Z t-1 |Z t )=Ν(Z t-1 ;μ θ (Z t ,t),Σ θ(Z t ,t));

[0016] Among them, Z t-1 is the state matrix after denoising, Z t is the state matrix of the current time step t, θ is a hyperparameter, μ θ (Z t ,t) is the mean of the Gaussian distribution, Σ θ (Z t ,t) is the covariance of the Gaussian distribution.

[0017] Preferably, the sampled data is pooled, and the link probability of the link prediction model is calculated based on the pooled data, specifically including: obtaining a target representation of a closed subgraph based on a sequence of a state matrix and a vector matrix; classifying the target representation of the closed subgraph through a multi-layer perceptron, and calculating the link probability of each link predicted by the link prediction model based on the classification results.

[0018] Preferably, the target representation of the closed subgraph is obtained according to the sequence of state matrices and the vector matrix, specifically including: the pooling process is described as follows:

[0019] q = pool(σ(W1Z′+W2X));

[0020] Where q is the target representation of the closed subgraph, pool() is the graph pooling function, and Z′=[Z′ T ; …; Z′0] is a combination of state matrix sequences, X is a vector matrix composed of closed subgraph node embedding vectors, W1, W2 are learnable weight matrices in the pooling process, and σ() is a nonlinear function used for data dimensionality reduction.

[0021] Preferably, the training of the link prediction model based on the link probability specifically includes: calculating the binary cross entropy of the link prediction model according to the link probability of the link prediction model and the true probability of the link to represent the difference between the link probability and the true probability; using the binary cross entropy as a loss function to train the link prediction model until the difference between the link probability and the true probability is minimized.

[0022] In a second aspect, the present invention provides a training device for a link prediction model, specifically comprising: at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the training method for the link prediction model in the first aspect.

[0023] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method provided in the first aspect.

[0024] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method provided in the first aspect.

[0025] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method provided in the first aspect.

[0026] Compared with the prior art, the beneficial effect of the present invention is that: in the training process of the link prediction model, the node representation of the closed subgraph is obtained, and random noise is introduced to gradually damage it. The incremental damage process accumulates noise in multiple iterations, and then the node representation is restored by iteratively restoring the noise, and only the feature information obtained at a certain moment in the restoration process is taken. The link prediction model trained by this method can effectively filter the noise information in the knowledge graph and enhance the node representation, reduce the computational overhead, and improve the accuracy of the link prediction task. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 A flow chart of a link prediction model training method provided by an embodiment of the present invention;

[0029] Figure 2 A flow chart of another link prediction model training method provided by an embodiment of the present invention;

[0030] Figure 3 A flow chart of another link prediction model training method provided by an embodiment of the present invention;

[0031] Figure 4 A flow chart of another link prediction model training method provided by an embodiment of the present invention;

[0032] Figure 5 A schematic diagram of the model structure of a multi-layer perceptron in the method provided in an embodiment of the present invention;

[0033] Figure 6A schematic diagram of the structure of a training device for a link prediction model provided by an embodiment of the present invention;

[0034] The reference numerals are as follows:

[0035] 11: processor; 12: memory. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0038] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0039] Embodiment 1:

[0040] In order to train the link prediction model, it is necessary to collect entities and relationships in the requirements according to specific task requirements, build a text knowledge graph, and divide the training set, validation set, and test set into training, validation, and testing sets, which are used for link prediction model training, training result verification, and training result testing, respectively. In the specific implementation, the division ratio is 0.85:0.05:0.1, which can be distinguished by adding training (train), validation (val), and test (test) labels according to the ratio in the text file corresponding to the text knowledge graph. After the division is completed, load the training set obtained by the division to train the link prediction model.

[0041] like Figure 1 As shown, the specific steps of the link prediction model training method provided by the embodiment of the present invention are as follows:

[0042] Step 101: Construct the data in the knowledge graph into an undirected graph, obtain the closed subgraph in the undirected graph, and obtain the closed subgraph representation matrix.

[0043] In this embodiment, a closed subgraph is used to obtain a node representation matrix of the knowledge graph. In a specific implementation, the NetworkX library can be used to construct the data in the training set into an undirected graph. In the constructed undirected graph, any two nodes are selected to obtain the closed subgraph of the node pair. According to the adjacency matrix of the closed subgraph and the node embedding of the closed subgraph, the closed subgraph representation matrix is ​​obtained. In the above process, the product of the closed subgraph adjacency matrix obtained directly from the data in the training set and the matrix composed of the embedding vectors of the nodes in the closed subgraph is called the closed subgraph representation matrix.

[0044] In the above undirected graph, if there is an edge directly connected between any two selected nodes, the node pair is a positive sample, otherwise the node pair is a negative sample. The above method is used to calibrate positive and negative samples in training (train), validation (val) and test (test) data.

[0045] Step 102: Add Gaussian noise to the closed subgraph representation matrix and remove all types of noise.

[0046] In actual implementation, a closed subgraph representation with richer semantics can be obtained by adding Gaussian noise to the closed subgraph representation matrix and then removing it. For example, the closed subgraph representation matrix can be fed into a diffusion model, and Gaussian noise can be added to the closed subgraph representation matrix and denoised by the diffusion model. In the forward process of the diffusion model, the closed subgraph representation matrix is ​​obtained, and random noise is introduced to gradually perform incremental damage to gradually accumulate noise in multiple iterations. In the reverse process of the diffusion model, the noise accumulated in the forward process is iteratively restored to restore the original node representation, thereby completing denoising.

[0047] Step 103: Obtain a matrix sequence in the denoising process, use each matrix in the matrix sequence as a state matrix, and obtain a vector matrix composed of closed subgraph node embedding vectors, pool the state matrix and the vector matrix, calculate the link probability of the link prediction model based on the pooled data, and train the link prediction model based on the link probability.

[0048] In the denoising process, the noise in the data of each denoising step will gradually decrease, and the semantic information that helps link prediction will gradually increase. Therefore, the matrix sequence generated in the denoising process (the reverse process of the diffusion model) and the vector matrix composed of the closed subgraph node embedding vectors are used as the data used for training to reduce the noise in the data. Compared with the original closed subgraph representation matrix, the state matrix contains richer semantic information and can be better used for link prediction.

[0049] On the other hand, in order to iteratively optimize the link prediction model, it is also necessary to pool the sampled data, calculate the link probability of the link prediction model prediction results based on the pooled data, compare the link probability of the prediction results with the true probability, and train and optimize the link prediction model based on the comparison results to obtain the link prediction model with the best prediction performance.

[0050] After steps 101 to 103 provided in this embodiment, the data in the training set can be denoised by the diffusion model, thereby training a link prediction model with better prediction performance.

[0051] like Figure 2 As shown, the closed subgraph representation matrix can be obtained in the following way.

[0052] Step 201: For each designated target node pair in the undirected graph, obtain the neighbor node set of the designated number of hops of the target node pair and the edge set from the nodes in the target node pair to each node in the neighbor node set, and combine the neighbor node set and the edge set to obtain a closed subgraph of the corresponding number of hops.

[0053] Given two nodes u, v∈V in the training set V as the target node pair, obtain the k-hop neighbor node set V of the target node pair u,v ={i|d(u,i)≤k∩d(i,v)≤k}, where: i represents the node i that can be reached within k hops from node u and node v in the training set V, d(u,i) represents the distance from node u to node i; d(i,v) represents the distance from node v to node i, V u,v Represents the set of node i. And based on V u,v Get edge set E u,v , E u,v represents the edge from node u and node v to each node i. At this point, the k-hop closed subgraph of the target node pair can be obtained In general scenarios, k does not exceed 3 to avoid introducing too much meaningless information.

[0054] Step 202: Calculate the adjacency matrix of the closed subgraph, calculate the vector matrix of the nodes in the closed subgraph, and calculate the node representation matrix according to the adjacency matrix and the vector matrix.

[0055] k-hop closed subgraph through target node pairs Get the adjacency matrix A of the closed subgraph. The size of the adjacency matrix A is based on V u,v Determine that the edges between nodes in the adjacency matrix A are based on E u,v Sure.

[0056] In the specific implementation, closed subgraphs can be obtained through algorithms such as TransE, HouseE, node2vec, etc. The vector matrix X consisting of the node's embedding vectors.

[0057] The adjacency matrix A of the closed subgraph and the matrix X composed of the embedding vectors of the nodes in the closed subgraph are obtained, and the product of the matrix A and X is calculated to obtain the closed subgraph representation matrix Z=AX.

[0058] After step 201 and step 202 provided in this embodiment, the node representation matrix Z of the closed subgraph can be obtained.

[0059] After obtaining the node representation matrix, the node representation matrix can be denoised through the diffusion model. The diffusion model includes two processes: forward and reverse.

[0060] like Figure 3 As shown, Gaussian noise can be added to the closed subgraph representation matrix and all types of noise can be removed in the forward process in the following way.

[0061] Step 301: At each time step, add Gaussian noise to the node representation matrix to obtain a state matrix.

[0062] In the forward process, standard Gaussian noise (Gaussian noise for short) is added to the node representation matrix to destroy the node representation matrix Z0 = Z. In a specific implementation, Z can be constructed by gradually adding Gaussian noise to the node representation matrix Z0 in T steps in a Markov chain. 1:T , to get the state matrix Z of the last time step T The steps are as follows:

[0063]

[0064] Among them, Z t-1 is the closed subgraph representation matrix of the previous time step, Z t is the representation matrix of the closed subgraph at the current time step t, t∈{1,…,T} represents the number of diffusions, and the scale of adding noise at each step is determined by the parameter β t ∈(0,1) control. The forward process q(Z t |Z t-1 ) means: In each forward step, by t-1 Add Gaussian noise to get Z t When T→∞, the state I of each node i converges to the standard Gaussian distribution.

[0065] In this embodiment, by using the re-parameterization technique and the additivity of two independent Gaussian noises, the state matrix Z of the current time step can be derived from the node representation matrix Z0. t .

[0066] The process of obtaining the state matrix from the node representation matrix is ​​described as:

[0067]

[0068] Among them, Z0 is the node representation matrix, Z t is the state matrix of the current time step t, I is the identity matrix, t is the current time step, is a hyperparameter, and N represents Gaussian distribution.

[0069] In the above process, Z t It can be reparameterized as follows:

[0070]

[0071] In order to adjust the addition of noise, a linear noise scheduler is adopted, which uses three hyperparameters: s, α up and α low To achieve The linear noise scheduler is defined as follows:

[0072]

[0073] The linear noise scheduler uses three hyperparameters: s∈[0,1] controls the noise scale, and α low <α up ∈(0,1) sets the upper and lower limits for adding Gaussian noise.

[0074] Step 302: According to the mean and covariance of the Gaussian distribution, the added Gaussian noise is removed from the state matrix until the denoising of the state matrix is ​​completed.

[0075] In the inverse process of the diffusion model, the closed subgraph matrix Z is represented at each time step t Remove the added noise from the forward process to restore the closed subgraph representation matrix Z before adding noise at each time step t-1 The last closed subgraph from the forward process represents the matrix Z T At the beginning, the reverse process gradually recovers the noise-free node representation through the denoising transition step. Assume that the closed subgraph representation matrix sequence generated by the denoising process is Z′ T:0 . Where Z′ T =Z T The closed subgraph representation matrix Z′ for time step t is t , whose probability distribution is determined, so the inverse process can represent the matrix Z′ through the closed subgraph in time step t t The probability distribution of t-1 is used to derive the closed subgraph representation matrix Z′ t-1 The probability distribution of .

[0076] The process of removing the added Gaussian noise is described as follows:

[0077] p θ (Z′ t-1 |Z′ t ) = N(Z′ t-1 ;μ θ (Z′ t ,t),Σ θ (Z′ t ,t));

[0078] Among them, Z′ t-1 is the closed subgraph representation matrix after denoising, Z′ t is the closed subgraph representation matrix of the current time step t, θ is a hyperparameter, μ θ (Z′ t ,t) is the mean of the Gaussian distribution, Σ θ (Z′ t ,t) is the covariance of Gaussian distribution, and N represents Gaussian distribution.

[0079] In the reverse process, a neural network parameterized by θ is used to generate the mean of the Gaussian distribution. and covariance It can be seen that the reverse process is only related to the state matrix Z at the current time step t t and Gaussian noise ∈ t About,∈ t ~Ν(0,Ι).

[0080] In order to ensure that the denoised data is not distorted or deviates from the original node representation matrix Z0, it is also necessary to consider the deviation loss in the denoising process. The loss function is defined as follows:

[0081]

[0082] In the loss function L, by minimizing Complete the denoising process.

[0083] In the specific implementation process, during the denoising process, the link prediction model is trained for 5 rounds, each round has 25 epochs of training, and the batch size is 1024.

[0084] After step 301 and step 302 provided in this embodiment, denoising can be performed using the diffusion model.

[0085] like Figure 4 As shown, the sampled data can be pooled and the link probability of the link prediction model can be calculated based on the pooled data in the following manner.

[0086] Step 401: Obtain a target representation of a closed subgraph according to the sequence of state matrices and the vector matrix.

[0087] The closed subgraph generated by the combined denoising process represents the matrix sequence, that is, the sequence of state matrices Z′ T:1 , we get the combination of state matrix sequence Z′=[Z′ T ; …; Z′0]. The matrix sequence generated during the denoising process contains the information of the noise removal process, which can make the semantic information more prominent. Combining the matrix sequence facilitates subsequent link prediction.

[0088] The combination Z′ of the state matrix sequence and the vector matrix X are sent to the pooling layer, and further downsampled by graph pooling to form the target representation of the closed subgraph. The pooling process is described as follows:

[0089] q = pool(σ(W1Z′+W2X));

[0090] Where q is the target representation of the closed subgraph, pool() is the graph pooling function, and Z′=[Z′ T ; …; Z′0] is a combination of state matrix sequences, X is a vector matrix composed of closed subgraph node embedding vectors, W1, W2 are learnable weight matrices in the pooling process, and σ() is a nonlinear function used for data dimensionality reduction.

[0091] Pooling the combination of state matrix sequences and the vector matrix composed of closed subgraph node embedding vectors can better preserve the topological information of the closed subgraph, facilitate the comprehensive use of the semantic information and topological information contained in the subgraph in subsequent link prediction, and improve the accuracy of link prediction.

[0092] Graph pooling can reduce the number of parameters of the link prediction model, improve generalization, extract the global representation of the graph, and facilitate the subsequent link probability calculation.

[0093] In the above pooling process, the image pooling method used is center pooling: pool C (Z) = z u ⊙z v . Where ⊙ is the Hadamard product, z u 、z v It is the learned representation of node u and node v in the combination of matrix sequence Z, that is, the target node pair node u and node v in the matrix sequence Z.

[0094] Step 402: Classify the target representation of the closed subgraph through a multi-layer perceptron, and calculate the link probability of each link predicted by the link prediction model according to the classification result.

[0095] After pooling is completed, the pooled data is sent to the Multilayer Perceptron (MLP). Figure 5As shown in the figure, MLP is a deep learning model based on a feedforward neural network, which consists of multiple neurons, where each neuron layer is fully connected to the previous layer; the input layer receives the target representation q of the closed subgraph, the middle hidden layer is used to extract features and linear transformation, and the output layer gives the final classification result: whether there is a link between node u and node v.

[0096] The formula for calculating link probability is as follows:

[0097] P = MLP(q);

[0098] Wherein, MLP represents a multi-layer perceptron, and the link probability of whether the link exists is calculated according to the target representation q obtained in step 401.

[0099] After step 401 and step 402 provided in this embodiment, the link probability of the prediction result can be obtained.

[0100] After obtaining the link probability, the binary cross entropy of the link prediction model is calculated according to the link probability of the link prediction model and the true probability of the link to represent the difference between the link probability and the true probability. The binary cross entropy is used as the loss function to train the link prediction model until the difference between the link probability and the true probability is minimized.

[0101] In this embodiment, binary cross entropy loss is used as the loss function when training the link prediction model. Binary cross entropy loss is a commonly used loss function, which is particularly suitable for binary classification problems and is used to measure the difference between the predicted probability distribution of the link prediction model and the true probability distribution.

[0102] The binary cross entropy loss is calculated as:

[0103] H(Q,P)=-(Qlog(P)+(1-Q)log(1-P));

[0104] Wherein, P is the link probability calculated in step 402, and Q is the true probability in the actual data, that is, for positive samples, Q=1, and for negative samples, Q=0.

[0105] The training process according to the loss function is carried out for 10 rounds, each round has 50 epochs of training, the batch size is 32, and the dropout is set to 0.5.

[0106] After completing the training according to the above process, the validation set and the training set can be further loaded into the trained link prediction model, the link prediction probability can be obtained through the link prediction model, and then the accuracy of the link prediction probability can be tested, and the link prediction model can be further optimized according to the test results.

[0107] In actual implementation, the area under the receiver operating characteristic curve (AUC) score can be used to represent the accuracy of the link prediction model. The AUC range is between 0.5 and 1.0. The closer to 1.0, the higher the accuracy, and vice versa.

[0108] The calculation method of AUC is based on the Receiver Operating Characteristic (ROC) curve. The probability predicted by the link prediction model for each sample is sorted from high to low. For the sorted prediction probability, each prediction probability value is used as the classification threshold from high to low, and the prediction probability is converted into a binary category according to this threshold; in general scenarios, the prediction probability above the threshold is judged as the positive class, and the prediction probability below or equal to the threshold is judged as the negative class. For each threshold, the true positive rate (TPR) and false positive rate (FPR) are calculated; TPR refers to the proportion of samples that are correctly identified among truly positive samples; FPR refers to the proportion of samples that are misclassified as positive among truly negative samples.

[0109] The specific calculation method is:

[0110]

[0111] Among them, TP is a true positive, which refers to the number of samples whose true category is c and whose predicted category is also c; FN is a false negative, which refers to the number of samples whose true category is c and whose predicted category is not c; FP is a false positive, which refers to the number of samples whose true category is not c and whose predicted category is c; FN is a false negative, which refers to the number of samples whose true category is other categories and whose predicted category is also other categories.

[0112] The (FPR, TPR) corresponding to each threshold is used as the coordinate point, and these points are connected to form the ROC curve. The calculation of AUC can be regarded as the area below the ROC curve and extending to the area enclosed by the coordinate axis.

[0113] The training method of the link prediction model provided in this embodiment adds a diffusion model to the training process of the connection prediction model, directly performs diffusion denoising on the node representation, obtains the node representation of the closed subgraph, and introduces random noise to gradually damage it. The incremental damage process accumulates noise in multiple iterations, and then restores the original node representation by iteratively restoring the noise, and only takes the feature information obtained at a certain moment in the recovery process. The link prediction model trained by this method can effectively filter the noise information in the knowledge graph and enhance the node representation, reduce the computational overhead, and improve the accuracy of the link prediction task.

[0114] Embodiment 2:

[0115] Based on the link prediction model training method provided in the above embodiment 1, the present invention also provides a link prediction model training device that can be used to implement the above method, such as Figure 6 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The training device of the link prediction model of this embodiment includes one or more processors 11 and a memory 12. Figure 6 A processor 11 is taken as an example.

[0116] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0117] The memory 12 is a non-volatile computer-readable storage medium for a link prediction model training method, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the link prediction model training method in Example 1. The processor 11 executes various functional applications and data processing of the link prediction model training device by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, implements the link prediction model training method in Example 1.

[0118] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0119] The program instructions / modules are stored in the memory 12, and when executed by one or more processors 11, the training method of the link prediction model in the above embodiment 1 is executed, for example, the above described Figure 1 , Figures 1 to 4 The steps shown.

[0120] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 6 A processor 11 in the embodiment may enable the one or more processors to execute the link prediction model training method in embodiment 1, for example, to execute the above-described Figures 1 to 3 The steps shown can also be implemented Figure 4 The various modules and units described.

[0121] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0122] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A training method for a link prediction model, characterized in that: include: Construct the data in the knowledge graph into an undirected graph, obtain the closed subgraph in the undirected graph, and obtain the closed subgraph representation matrix; Add Gaussian noise to the closed subgraph representation matrix and remove all types of noise; Get the matrix sequence in the denoising process, take each matrix in the matrix sequence as the state matrix, and get the vector matrix composed of the closed subgraph node embedding vectors, pool the state matrix and the vector matrix, calculate the link probability of the link prediction model according to the pooled data, and train the link prediction model based on the link probability.

2. The link prediction model training method according to claim 1, characterized in that: The data in the knowledge graph is constructed as an undirected graph, a closed subgraph in the undirected graph is obtained, and a closed subgraph representation matrix is ​​obtained, which specifically includes: For each specified target node pair in the undirected graph, obtain the neighbor node set of the target node pair with a specified number of hops, and the edge set from the node in the target node pair to each node in the neighbor node set, and obtain a closed subgraph with the corresponding number of hops from the neighbor node set and the edge set; Calculate the adjacency matrix of the closed subgraph, and calculate the vector matrix composed of the embedded vectors of the nodes of the closed subgraph. Calculate the node representation matrix based on the adjacency matrix and the vector matrix.

3. The link prediction model training method according to claim 1, characterized in that: The method of adding Gaussian noise to the closed subgraph representation matrix and removing all types of noise specifically includes: At each time step, Gaussian noise is added to the node representation matrix to obtain the state matrix; According to the mean and covariance of the Gaussian distribution, the added Gaussian noise is removed from the state matrix until the denoising of the state matrix is ​​completed.

4. The link prediction model training method according to claim 3, characterized in that: The step of adding Gaussian noise to the node representation matrix to obtain a state matrix specifically includes: The process of obtaining the state matrix from the node representation matrix is ​​described as: Among them, Z0 is the node representation matrix, Z t is the state matrix of the current time step t, I is the identity matrix, t is the current time step, is a hyperparameter, and N represents Gaussian distribution.

5. The link prediction model training method according to claim 3, characterized in that: The step of removing the added Gaussian noise from the state matrix according to the mean and covariance of the Gaussian distribution specifically includes: The process of removing the added Gaussian noise is described as follows: p θ (Z′ t-1 |Z′ t )=N(Z′ t-1 ;μ θ (Z′ t ,t),∑ θ (Z′ t ,t)); Among them, Z′ t-1 is the closed subgraph representation matrix after denoising, Z′ t is the closed subgraph representation matrix of the current time step t, θ is a hyperparameter, μ θ (Z′ t , t) is the mean of the Gaussian distribution, ∑ θ (Z′ t , t) is the covariance of Gaussian distribution, and N represents Gaussian distribution.

6. The link prediction model training method according to claim 1, characterized in that: The step of pooling the sampled data and calculating the link probability of the link prediction model according to the pooled data specifically includes: According to the sequence of state matrices and the vector matrix composed of the embedding vectors of the closed subgraph nodes, the target representation of the closed subgraph is obtained; The target representation of the closed subgraph is classified by a multi-layer perceptron, and the link probability of each link predicted by the link prediction model is calculated based on the classification results.

7. The link prediction model training method according to claim 6, characterized in that: The target representation of the closed subgraph is obtained according to the sequence of state matrices and the vector matrix, specifically including: The pooling process is described as follows: q = pool(σ(W1Z′+W2X)); Where q is the target representation of the closed subgraph, pool() is the graph pooling function, and Z′=[Z′ T ; ...; Z′0] is a combination of state matrix sequences, X is a vector matrix composed of closed subgraph node embedding vectors, W1, W2 are learnable weight matrices in the pooling process, and σ() is a nonlinear function used for data dimensionality reduction.

8. The link prediction model training method according to claim 1, characterized in that: The training of the link prediction model based on the link probability specifically includes: According to the link probability of the link prediction model and the true probability of the link, the binary cross entropy of the link prediction model is calculated to represent the difference between the link probability and the true probability; Using binary cross entropy as the loss function, the link prediction model is trained until the difference between the link probability and the true probability is minimized.

9. A training device for a link prediction model, characterized in that: It includes at least one processor and a memory, the at least one processor and the memory are connected via a data bus, the memory stores instructions that can be executed by the at least one processor, and the instructions, after being executed by the processor, are used to complete the training method of the link prediction model described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the link prediction model training method as described in any one of claims 1-8.

Images