A collaborator prediction method and prediction system based on temporal collaborator knowledge graph
By constructing a temporal collaborator knowledge graph and using the encoder-decoder deep learning model to capture entity semantics and tense interactions, the problem of existing models failing to effectively utilize time information and multivariate relationships is solved, and the accuracy of collaborator predictions is improved.
Patent Information
- Application Number
- CN202110606169.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-31
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2041-05-31
AI Technical Summary
Existing collaborator prediction models fail to effectively utilize time information and diverse relationships and cannot accurately predict future collaborative authors.
The encoder-decoder deep learning model is adopted to capture entity semantic information through semantic encoder, and the temporal graph attention mechanism models the temporal interaction of the central entity neighborhood, and triple-scale convolution kernel strategy is used to perform triple-scale scoring to construct a temporal collaborator knowledge graph.
It improves the accuracy of collaborators' predictions, can better utilize time information and diverse relationships, enhances information utilization, and is suitable for other types of knowledge graphs.
Smart Images
Figure CN115481215B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of social network analysis, and relates to a method for predicting links between nodes in a graph. More specifically, it relates to a collaborator prediction method and prediction system based on a temporal collaborator knowledge graph. Background Art
[0002] In recent decades, the rapid development of the modern internet and its related technologies has led to increasingly frequent interactions between people of all kinds, fueling researchers' interest and enthusiasm in the vast online social networks. Social network analysis has a wide range of applications, such as public opinion analysis and control within public opinion networks, instant user recommendations within online social systems, and influence analysis. Collaborator networks, as a key subnetwork of social networks, have also garnered significant attention in recent years. Collaborator network prediction, a key task within collaborator networks, can provide powerful insights for academic researchers collaborating on projects and papers, recommending suitable collaborators and fostering the exchange of ideas between researchers.
[0003] Traditional work on co-author prediction often relies on homogeneous networks, i.e., networks with only one type of node and relationship, such as collaborator networks and friendship networks. Traditional co-author prediction models often ignore or underutilize information about various relationships, such as conferences, topics, and institutional affiliations. This information is crucial for co-author prediction. For example, authors with similar organizational structures are more likely to collaborate in the future.
[0004] After Google proposed multi-relational directed graphs (KGs) in 2012 and applied them to semantic search, knowledge graphs once again attracted the attention of scholars. Integrating these knowledge graphs, which contain a wealth of human prior knowledge, with deep learning has become a key approach to further improving the effectiveness of artificial intelligence. Knowledge graphs have once again gained prominence and have achieved tremendous success in fields such as semantic analysis, question-answering systems, and recommendation systems. Knowledge graphs are composed of a large number of triplets of the form (head entity, relationship, tail entity), where entities are nodes and relationships are directed edges between nodes. Collaborator prediction problems are represented on knowledge graphs as binary classification of triplets, i.e., determining whether the triple is true or not. Specifically, the head and tail entities must be fixed to the author type, and the relationship to the collaboration relationship.
[0005] Knowledge Graphs, as a multi-relational graph, can obtain a powerful representation of author nodes through KG embedding learning. KG embedding refers to mapping entities and relationships in a knowledge graph into a continuous embedding space. Existing KG embedding learning models are mainly divided into those for static KGs, such as the translation-based model TransE, the convolutional neural network-based ConvE, and the graph neural network-based KBGAT, but these models do not consider time information; and those for temporal KGs, such as TAE, which uses time order as a constraint to improve the quality of embedding representation, and HyTE, which associates each timestamp with a corresponding hyperplane. However, these models still consider each triple independently and cannot model the temporal interactions between entities. The so-called temporal interaction means that since the neighbor nodes and the central entity have temporal attributes, there is an interaction in temporal attributes between the neighbor nodes of each central entity. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and propose a collaborator prediction method based on a temporal collaborator knowledge graph. Due to the complexity of the problem, a deep learning model with an encoder-decoder is proposed. The encoder is divided into two parts: a semantic encoder that captures entity semantic information; an embedding encoder that models the temporal interactions between the central entity and the neighborhood and aggregates spatiotemporal neighborhood information through a temporal graph attention mechanism; and a decoder based on multi-scale convolutional kernels for triple scoring. The specific steps include the following:
[0007] The model training of the method of the present invention comprises the following steps:
[0008] Step (1): Constructing a temporal collaborator knowledge graph: Clean the collected academic relationships, create a temporal collaborator knowledge graph consisting of temporal triples in the form of head entity, relationship, and tail entity with time attributes, and store it in the Neo4j graph database;
[0009] Step (2): Train a semantic encoder based on a pre-trained language model to mine the semantic information of entities in the temporal collaborator knowledge graph as its initial representation;
[0010] Step (3): If the number of semantic encoder training rounds in step (2) does not reach the set value or the training loss does not reach the early stopping condition, then the normalized attention coefficient of the temporal triples in the neighborhood of each entity for the central entity is calculated according to the temporal graph attention mechanism, and the neighborhood information is weighted and aggregated according to the coefficient and a jump connection is added to update the entity embedding representation. The process is a single temporal graph attention layer (i.e., the entire process of calculating triple embedding, calculating normalized attention coefficient, and weighted aggregation of neighborhood information and adding a jump connection to update the entity embedding representation). The embedding representation of the entity is enhanced by training an embedding encoder stacked with at least one temporal graph attention layer. One training round refers to completing the step of updating the entity embedding representation once with complete data and updating the encoder parameters.
[0011] Step (4): After the semantic encoder training rounds reach the set value or the training loss reaches the early stopping condition in step (2), given a triple, its head and tail entities and relations are combined into a triple embedding matrix, and the decoder based on the multi-scale convolution kernel strategy is trained to decode the embedding matrix to obtain the score of the triple as its confidence;
[0012] The model test of the method of the present invention includes:
[0013] Step (5): After the decoder training round reaches the set value or the training loss reaches the early stopping condition in step (4), two authors who are already in the temporal collaborator knowledge graph but have never collaborated are given. The embedded representations of the two author entities and the collaborative relationship are obtained based on the trained embedding encoder and input into the trained decoder. The decoder scores the embedded identifiers of the entities and relationships and obtains their confidence. If it is greater than the set threshold, it is predicted that there will be collaboration between them. Completing the scoring of all triples in the dataset and updating the decoder parameters is a training round.
[0014] In this invention, the collaborator prediction method based on the temporal collaborator knowledge graph refers to predicting future collaborators using the temporal knowledge graph as an information carrier. The temporal collaborator knowledge graph is a multi-relational directed graph composed of triplets of the form (head entity, relationship, tail entity) with time attributes, where entities are nodes in the knowledge graph and relationships are directed edges between entities in the knowledge graph. Future collaborators are two authors who have never collaborated before and will publish the same paper together in the future.
[0015] In the present invention, the specific steps of step (1) include:
[0016] Step (1.1): The academic relationships are collected from public data on paper websites. The academic relationships refer to a collection of multiple relationships, such as collaborations between authors and the conferences and journals in which the papers are published. Data cleaning involves removing authors with too few appearances. The threshold for the number of appearances can be freely selected, resulting in a set of entities and relationships. Triples are constructed based on the multiple relationships contained in the academic relationships, such as collaborations, affiliated institutions, and paper topics. The triples have a time attribute, and the time of their creation, such as the time of paper publication, is appended to the triples.
[0017] Step (1.2): In order to enable each entity to obtain information about more distant neighbors (i.e., the shortest reachable path length between two entities is longer), auxiliary directed edges are artificially constructed between each entity and its multi-hop neighbors, which facilitates the flow of knowledge in the temporal collaborator knowledge graph.
[0018] In the present invention, the semantic encoder in step (2) refers to a semantic encoder that captures entity semantic information.
[0019] The specific steps include:
[0020] Step (2.1): Each entity e in the temporal collaborator knowledge graph described in step (1) is composed of a word sequence s, that is, e=s=[w1,…w l ,…w L ], where w l Refers to the lth word in the word sequence, and L refers to the total number of words in the word sequence. The word sequence s is fed into the pre-trained language model BERT to obtain the word embedding [w1,…w l ,…w L ]=BERT([w1,…w l ,…w L ]),in Represents word w l word embedding, d BERT Represents the dimension of word embedding.
[0021] Step (2.2): For each word embedding sequence obtained for entity e, an average strategy is adopted to convert the word embedding sequence into entity embedding, i.e. A fully connected layer is used to reduce the dimension of the entity embedding e obtained, that is, e init =FC s (e), where FC s represents a fully connected layer, e init This is the initial embedding of the entity.
[0022] In the present invention, the embedding encoder in step (3) refers to an embedding encoder that models the temporal interaction of the neighborhood of the central entity and aggregates the spatiotemporal neighborhood information through the temporal graph attention mechanism.
[0023] The specific steps include:
[0024] Step (3.1): Calculate the normalized attention coefficient of the neighbors of each central entity t through the temporal graph attention layer. For each central entity t, the triples with the central entity t as the tail entity are represented as the set V = [v1,…,v y ,…,v Y ], where v y =[h i ,r j ,t] and has the time attribute time a , where h i Represents the head entity, r j represents the relationship and t represents the tail entity, time a Represents the time attribute, Y represents the number of triples in the set V, and these triples are grouped according to the time attribute time a Sort in ascending order. For each triple v y , applying a linear transformation Where || represents the splicing operation, W1 is the linear transformation weight matrix, and e t Represents the head entity h i , relation r j and the embedding representation of the tail entity t, v y That is the triple v y The corresponding embedding representation. The obtained embedding set is recorded as V = [v1,…,v y ,…,v Y ].
[0025] The normalized attention coefficient of the triple formed by each neighbor is calculated through the bidirectional LSTM network Bi-LSTM. Bi-LSTM contains two LSTM networks, namely forward-layer and backward-layer. Specifically, and Represents the hidden layer state of the forward-layer and backward-layer at time step y-1, then the hidden layer state of the forward-layer and backward-layer at time step y is calculated as follows, and v y Refers to the embedding representation of the triple, then the hidden layer state of Bi-LSTM at time step y is Then through a fully connected layer FC a ()Get triple v y The attention coefficient, The weight matrix W2 and bias b are trainable parameters, and σ is the activation function LeakyReLU. The normalized attention coefficient is calculated by the softmax function.
[0026] Step (3.2): Update the central entity representation by weighted aggregation triples. The central entity t obtains a new embedding representation by weighted aggregation triples in the set V. σ is the activation function. The training process of the attention mechanism is stabilized by the multi-head attention mechanism. in and It is calculated by the kth independent attention head, where K represents the number of attention heads. If the current temporal attention layer is the last layer, the entity embeddings output by the multi-head attention mechanism will not be concatenated but averaged. In order to prevent the problem of gradient dispersion and the disappearance of entity semantic information due to the deepening of the network, a skip connection is adopted in each temporal graph attention layer, which is expressed as: Where W3 is the trainable weight matrix, That is the entity embedding output by the attention layer of the graph, e init is the initial embedding of the entity output by the semantic encoder in step 2, e' t is the output of the above multi-head attention mechanism.
[0027] Step (3.3): Update the embedding representation of the relationship. Learn the new embedding representation of the relationship through linear transformation, r'=W4r, the dimension of r' is the same as The dimensions are the same, W4 is the linear transformation weight matrix, r is the embedding representation before the relationship is updated, Entity embedding output by the graph attention layer.
[0028] Step (3.4): Train the encoder. Jointly train the semantic encoder and embedding encoder, and the loss function of the two is L 编码器 =∑ (h,r,t)∈Δ ∑ (h',r,r')∈Δ' [d (h,r,t) -d (h',r,t') +γ] + , where Δ represents the positive sample set, Δ' represents the negative sample set, γ represents the safety margin distance, [x] + =Max[x,0], e h ,e r ,e tThey represent the embedded representations of h, r, and t output by the embedding encoder, respectively, and l1 refers to l1 regularization.
[0029] Step (3.5): Calculate the encoder loss function value and update the encoder parameters.
[0030] In the present invention, the decoder in step (4) is used for triple scoring based on a multi-scale convolution kernel.
[0031] The specific steps include:
[0032] Step (4.1): Convolution is performed using a set of multi-scale convolution kernels.
[0033] Given a triple (h, r, t), e h ,e r ,e t Represent the embedded representations of h, r, and t output by the embedding encoder respectively. The head and tail entities and relations are represented as a triplet as an embedding matrix. Where d represents the dimension of entity embedding and relation embedding.
[0034] Use three convolution kernels of different sizes Convolution is performed on the triple embedding matrix to obtain the feature matrix τ1, τ2, τ3, where Represented by the convolution kernel ω i The resulting feature matrix, c i is the feature matrix τ i The generated feature matrices are concatenated. c=c1+c2+c3, where c is the dimension of p.
[0035] Step (4.2): Use multiple (at least one) multi-scale convolution kernel sets to perform convolution to stabilize the feature extraction process. The M multi-scale convolution kernel sets are denoted as Ω' = [Ω1,…,Ω M ], the output M feature matrices are spliced, M c is the dimension of the concatenated vector P.
[0036] Step (4.3): Score the triples according to P. Use a weight matrix w d Perform matrix dot multiplication, f(h,r,t)=P·w d The final decoder scoring function is Where * represents the convolution operation, b represents the bias, and Ω m is the m-th multi-scale convolution kernel set, and A is the embedding matrix of the triple (h, r, t).
[0037] Step (4.4): Train the decoder. The loss function of the decoder training is, in is the weight vector w d L2 regularization, Δ represents the positive sample set, and Δ' represents the negative sample set.
[0038] In the present invention, the specific steps of the model test in step (5) include:
[0039] Step (5.1): Select two author entities that have never collaborated. Use the trained embedding encoder to obtain their corresponding embeddings and the embedding of the collaboration relationship. This matrix is then fed into the trained decoder to obtain the score. The threshold is used to determine whether the score is true or not.
[0040] Based on the above method, the present invention also proposes a collaborator prediction system based on a temporal collaborator knowledge graph, comprising: a memory and a processor; a computer program is stored on the memory, and when the computer program is executed by the processor, the above prediction method is implemented.
[0041] The beneficial effects of the present invention compared with the existing technology include: proposing a collaborator prediction method based on the temporal collaborator knowledge graph, which can greatly improve the prediction performance by considering time information and effective semantic capture; modeling the temporal interaction between the neighbors of the central entity through the temporal graph attention mechanism and aggregating multi-hop spatiotemporal neighborhood information, which can maximize the information utilization, and this method can also be easily applied to other types of knowledge graphs.
[0042] The method of the present invention first takes into account multi-dimensional relationships, and secondly takes into account and models semantic information and time information. Finally, this method can also be easily applied to other types of temporal knowledge graphs. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 This is a flowchart of the collaborator prediction method based on the temporal collaborator knowledge graph of the present invention.
[0044] Figure 2 An example of a collaborator knowledge graph provided for an embodiment of this invention.
[0045] Figure 3 The temporal attention mechanism in the collaborator prediction method based on the temporal collaborator knowledge graph of the present invention is Figure 2 Model diagram under the embodiment.
[0046] Figure 4 The encoder in the collaborator prediction method based on the temporal collaborator knowledge graph of the present invention is Figure 2 Model diagram under the embodiment.
[0047] Figure 5 The decoder in the collaborator prediction method based on the temporal collaborator knowledge graph of the present invention is Figure 2 Model diagram under the embodiment.
[0048] Figure 6 This is the operating result of the present invention in the real-world temporal collaborator knowledge graph. DETAILED DESCRIPTION
[0049] The invention will be further described in detail with reference to the following specific embodiments and accompanying drawings so that those skilled in the art can better understand the invention. It should be understood that the specific embodiments described herein are only used to explain the invention and are not intended to limit the invention.
[0050] like Figure 1 As shown, the collaborator prediction method based on the temporal collaborator knowledge graph proposed in the present invention includes the following steps:
[0051] (1) Construction of the temporal collaborator knowledge graph: data cleaning of the collected academic relationships is performed, and then a temporal collaborator knowledge graph consisting of triples with time attributes in the form of head entity, relationship, and tail entity is created and stored in the Neo4j graph database;
[0052] (2) Train a semantic encoder based on a pre-trained language model to mine the semantic information of entities in the temporal collaborator knowledge graph as its initial representation;
[0053] (3) For each entity’s neighborhood, the normalized attention coefficient of the temporal triples for the central entity is calculated according to the temporal graph attention mechanism, and the neighborhood information is weighted and aggregated according to the coefficient to update the entity embedding representation. This process is a single temporal graph attention layer, and the embedding representation of the entity is enhanced by training the embedding encoder stacked with multiple temporal graph attention layers.
[0054] (4) Given a triple, its head and tail entities and relations are combined into a triple embedding matrix, and a decoder based on a multi-scale convolution kernel strategy is trained to decode the triple on the embedding matrix to obtain the score of the triple as its confidence;
[0055] (5) Given two authors who are already in the temporal collaborator knowledge graph but have never collaborated, their embedding representations are obtained based on the trained embedding encoder and input into the trained decoder to obtain their confidence. If it is greater than the set threshold, it is predicted that there will be collaboration between them.
[0056] Example
[0057] See also Figure 2 , assuming Figure 2The following is a temporal collaborator knowledge graph with six author entities A1,…A6, two paper entities P1 and P2, one author institution entity C1, and one conference entity K1. Solid edges represent actual relationships, while dashed edges are artificially added auxiliary edges. Assuming A1 is the central entity, A1 had a closer relationship with A5 in the past and a closer relationship with A4 later in time, because A1 and A4 had more frequent and later collaborations. With A1 as the central entity, the temporal interaction between A1 and A4 influences the relationship between A1 and A5, while A1's collaboration with A3 also facilitates collaboration between A1 and A4. Therefore, temporal interactions exist between A1's neighbors, and new collaborations between A1 and other authors depend on their neighborhood information. Collaborator prediction methods based on the temporal collaborator knowledge graph can effectively model the temporal interactions between A1's neighbors through the temporal graph attention mechanism, while the embedding encoder effectively aggregates A1's neighborhood information.
[0058] The specific steps of step (1) include:
[0059] In step (1.1), we clean the data based on the academic relationships obtained from the public data of the paper website, remove authors with too few appearances (this filtering threshold is optional), and filter out the entity set and relationship set. We construct triples based on the multiple relationships contained in the academic relationship, such as collaboration (Co-Author), affiliation (ResearchOf), and conference (PublishPaperOn). We also append the time of creation to the triples, such as the time of paper publication.
[0060] Step (1.2) artificially constructs an auxiliary edge between the central entity A1 and its multi-hop neighbor A6.
[0061] like Figure 3 As shown in the figure, there are 7 triplets with the central entity A1 as the tail entity. They are sorted according to the time attribute and sent into the bidirectional LSTM network. The hidden layer state of the bidirectional LSTM network at each step is sent to a fully connected layer to obtain the attention coefficient of each triplet. Then, the softmax function is used to obtain the normalized attention coefficients corresponding to the 7 triplets.
[0062] like Figure 4As shown in the figure, the triples with the central entity A1 as the tail entity are fed into the encoder. First, each triple is fed into the embedding encoder for embedding initialization. For model visualization, the dimension is assumed to be 2, which can actually be set to a value such as 50. Then, the initial embeddings of the head and tail entities of the triple and the relationship are spliced together and fed into two different attention heads (which can actually be set to a value such as 8) to calculate the new representation of the central entity A1. The output embeddings are spliced and jump-connected with the initial embedding of A1 to obtain the new representation of A1. The relationship is learned through linear transformation to obtain a representation with the same dimension as the new representation of A1. This is the first layer of temporal attention layer. The second layer of temporal attention layer is then calculated to obtain a further new representation of A1. Finally, the loss is calculated using the triple labels and the new representations of the head and tail entities and relationships to train the encoder.
[0063] The specific steps of step (3) include:
[0064] Step (3.1) calculates the normalized attention coefficient of the neighbors of each central entity through the temporal graph attention layer. For each central entity t, the triples with the central entity t as the tail entity are represented as the set V = [v1,…,v y ,…,v Y ], where v y =[h i ,r j ,t] and has the time attribute time a , where h i Represents the head entity, r j represents the relationship and t represents the tail entity, time a Represents the time attribute, Y represents the number of triples in the set V, and these triples are grouped according to the attribute time a Sort in ascending order. For each triple v y , applying a linear transformation Where || represents the splicing operation, W1 is the linear transformation weight matrix, and e t Represents the head entity h i , relation r j and the embedding representation of the tail entity t, v y That is the triple v y The corresponding embedding. The resulting embedding set is denoted as V = [v1,…,v y ,…,v Y ]. The normalized attention coefficient of the triple formed by each neighbor is calculated through the bidirectional LSTM network Bi-LSTM. Bi-LSTM contains two LSTM networks, namely forward-layer and backward-layer. Specifically, and Represents the hidden layer state of the forward-layer and backward-layer at time step y-1, then the hidden layer state of the forward-layer and backward-layer at time step y is calculated as follows, and Then the hidden layer state of Bi-LSTM at time step y is Then a fully connected layer is used to obtain the triplet v y The attention coefficient, The weight matrix W2 and bias b are trainable parameters, and σ is the activation function LeakyReLU. The normalized attention coefficient is calculated by the softmax function.
[0065] Step (3.2) weighted aggregation triples update the central entity representation. The central entity t obtains a new embedding representation by weighted aggregation triples in the set V. Stabilize the training process of the attention mechanism through the multi-head attention mechanism, in and It is calculated by the kth independent attention head, where K represents the number of attention heads. If the current temporal attention layer is the last layer, the entity embeddings output by the multi-head attention mechanism will not be concatenated but averaged. In order to prevent the problem of gradient dispersion and the disappearance of entity semantic information due to the deepening of the network, a skip connection is adopted in each temporal graph attention layer, which is expressed as: Where W3 is the trainable weight matrix, That is the entity embedding output by the attention layer of the graph, e init is the initial embedding of the entity output by the semantic encoder in step 2, e' t is the output of the above multi-head attention mechanism.
[0066] Step (3.3) updates the embedding representation of the relation. The new embedding representation of the relation is learned by linear transformation, r'=W4r, where the dimension of r' is the same as The dimensions are the same, W4 is the linear transformation weight matrix, r is the embedding representation before the relationship is updated, Entity embedding output by the graph attention layer.
[0067] Step (3.4) trains the loss function of the encoder. Jointly train the semantic encoder and embedding encoder, and the loss function of the two is L 编码器 =∑ (h,r,t)∈Δ ∑ (h',r,t')∈Δ' [d (h,r,t) -d (h',r,t') +γ]+ , where Δ represents the positive sample set, Δ' represents the negative sample set, γ represents the safety margin distance, [x] + =Max[x,0], e h ,e r ,e t They represent the embedded representations of h, r, and t output by the embedding encoder, respectively, and l1 refers to l1 regularization.
[0068] Example: Figure 5 As shown, assuming that the current triple is (A5, Co-Author, A1, 2013), the dimension of the embedding of entities and relations output by the embedding encoder is 7. Assuming that there is only one multi-scale convolution kernel set, after three convolution kernels of different sizes, that is, Get three sizes of The feature matrix of , where ReLU is the activation function, b1, b2, b3 are different biases, and after splicing, a matrix of size The vector, after the weight matrix w d Perform the dot product to get the score of the triple, which is the confidence.
[0069] In the present invention, the specific steps of step (4) include:
[0070] Step (4.1) uses a multi-scale convolution kernel set for convolution. Given a triple (h, r, t), e h ,e r ,e t Represent the embedded representation of h, r, t output by the embedding encoder respectively. The triple is represented as an embedding matrix, Where d represents the dimension of entity embedding and relation embedding. Three convolution kernels of different sizes are used Convolution is performed on the triple embedding matrix to obtain the feature matrix τ1, τ2, τ3, where Represented by the convolution kernel ω i The resulting feature matrix, c i Represents the feature matrix τ i Dimension. The generated feature matrix is spliced, c=c1+c2+c3, where c is the dimension of p.
[0071] Step (4.2) uses multiple (at least one) multi-scale convolution kernel sets to perform convolution to stabilize the feature extraction process. Multiple multi-scale convolution kernel sets are denoted as Ω' = [Ω1,…,Ω M ], the output M feature matrices are spliced, Mc is the dimension of the concatenated vector P.
[0072] Step (4.3) scores the triples according to P. A weight matrix w is used d Perform matrix dot multiplication, f(h,r,t)=P·w d The final decoder scoring function is Where * represents the convolution operation, b represents the bias, and Ω m is the m-th multi-scale convolution kernel set, and A is the embedding matrix of the triple (h, r, t).
[0073] Step (4.4) trains the decoder. The loss function for decoder training is, in is the weight vector w d L2 regularization, Δ represents the positive sample set, and Δ' represents the negative sample set.
[0074] like Figure 6 As shown in Figure 3, the results of this method on real-world academic relationship networks are excellent.
[0075] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the appended claims.
Claims
1. A collaborator prediction method based on temporal collaborator knowledge graph, characterized in that: The following steps are involved: Step (1): Construction of the temporal collaborator knowledge graph. Data cleaning of the collected academic relationships is performed. Then, a temporal collaborator knowledge graph consisting of temporal triples with time attributes in the form of head entity, relationship, and tail entity is created and stored in the Neo4j graph database. Step (2): Train a semantic encoder based on a pre-trained language model to mine the semantic information of entities in the temporal collaborator knowledge graph as its initial representation; Step (3): If the number of semantic encoder training rounds in step (2) does not reach the set value or the training loss does not reach the early stopping condition, then the normalized attention coefficient of the temporal triples in the neighborhood of each entity for the central entity is calculated according to the temporal graph attention mechanism, and the neighborhood information is weighted and aggregated according to the coefficient to update the entity embedding representation; the entire process of calculating triple embedding, calculating normalized attention coefficient, and weighted aggregation of neighborhood information and adding skip connections to update the entity embedding representation is a single temporal graph attention layer, and the embedding representation of the entity is enhanced by training the embedding encoder stacked with at least one temporal graph attention layer; Step (4): After the semantic encoder training rounds reach the set value or the training loss reaches the early stopping condition in step (2), given a triple, its head and tail entities and relations are combined into a triple embedding matrix, and the decoder based on the multi-scale convolution kernel strategy is trained to decode the embedding matrix to obtain the score of the triple as its confidence; Step (5): After the decoder training rounds reach the set value or the training loss reaches the early stopping condition in step (4), two authors who are already in the temporal collaborator knowledge graph but have never collaborated are given. Their embedding representations are obtained based on the trained embedding encoder and input into the trained decoder to obtain their confidence. If it is greater than the set threshold, it is predicted that they will collaborate.
2. The collaborator prediction method based on temporal collaborator knowledge graph according to claim 1 is characterized in that: Future collaborators are predicted using a temporal knowledge graph as an information carrier. The temporal collaborator knowledge graph is a multi-relationship directed graph consisting of triplets in the form of a head entity, a relationship, and a tail entity with a time attribute. The entities are nodes in the temporal collaborator knowledge graph, and the relationships are directed edges between the entities in the temporal collaborator knowledge graph. The future collaborators are two authors who have never collaborated before and will jointly publish the same paper in the future.
3. The collaborator prediction method based on temporal collaborator knowledge graph according to claim 1 is characterized in that: The specific steps of step (1) include: Step (1.1): The academic relationship refers to a set of multivariate relationships including the cooperative relationship between authors and the conference journals in which the paper is published, which is obtained through the public data of the paper website; the data cleaning refers to removing authors with too few appearances to filter out the entity set and relationship set; the filtering threshold of the number of appearances can be freely selected; triples are constructed based on the multivariate relationships of cooperative relationships, affiliated institutions, and paper topics contained in the academic relationship, and the time of creation of the triple is attached to the triple, which includes the time of paper publication; Step (1.2): In order to enable each entity to obtain information about its more distant neighbors, auxiliary directed edges are artificially constructed between each entity and its multi-hop neighbors to enable the flow of knowledge in the temporal collaborator knowledge graph; the more distant refers to a length longer than the shortest reachable path between two entities.
4. The collaborator prediction method based on temporal collaborator knowledge graph according to claim 1 is characterized in that: The specific steps of the semantic encoder in step (2) include: Step (2.1): Each entity e in the temporal collaborator knowledge graph is composed of a word sequence s, that is, e = s = [w1, ...w l ,...w L ], where w l Refers to the lth word in the word sequence, L refers to the total number of words in the word sequence; the word sequence s is fed into the pre-trained language model BERT to obtain the word embedding of each word [w1,...w l ,...w L ]=BERT([w1,...w l ,...w L ]),in Represents word w l word embedding, d BERT Represents the dimension of word embedding; Step (2.2): For each word embedding sequence obtained for entity e, an average strategy is adopted to convert the word embedding sequence into entity embedding, i.e. A fully connected layer is used to reduce the dimension of the entity embedding e obtained, that is, e init =FC s (e), where FC s represents a fully connected layer, e init This is the initial embedding of the entity.
5. The collaborator prediction method based on temporal collaborator knowledge graph according to claim 1 is characterized in that: The specific steps of embedding the encoder in step (3) include: Step (3.1): Calculate the normalized attention coefficient of the neighbors of each central entity t through the temporal graph attention layer; for each central entity t, the triple with the central entity t as the tail entity is represented as a set V = [v1, ..., v y ,...,v Y ], where v y =[h i , r j , t] and has the time attribute time a , where h i Represents the head entity, r j represents the relationship and t represents the tail entity, time a Represents the time attribute, Y represents the number of triples in the set V, and these triples are grouped according to the time attribute time a Sort in ascending order; for each triple v y , applying a linear transformation Where || represents the splicing operation, W1 is the linear transformation weight matrix, and et represent the head entity h i , relation r j and the embedding representation of the tail entity t, v y is the embedding representation of the triple; the obtained embedding set is recorded as V = [v1, ..., v y ,...,v Y ]; The normalized attention coefficient of the triple formed by each neighbor is calculated through the bidirectional LSTM network Bi-LSTM; Bi-LSTM contains two LSTM networks, namely forward-layer and backward-layer. Specifically and Represents the hidden layer state of the forward-layer and backward-layer at time step y-1, then the hidden layer state of the forward-layer and backward-layer at time step y is calculated as follows, and v y is the embedding representation of the triple, then the hidden layer state of Bi-LSTM at time step y is Then through a fully connected layer FC a ()Get triple v y The attention coefficient, σ(W2m y +b), where the weight matrix W2 and bias b are trainable parameters, σ is the activation function LeakyReLU; the normalized attention coefficient is calculated by the softmax function, Step (3.2): Update the central entity representation by weighted aggregation triples; the central entity t obtains a new embedding representation by weighted aggregation triples in the set V. σ is the activation function; the training process of the attention mechanism is stabilized by the multi-head attention mechanism, in and It is calculated by the kth independent attention head, where K represents the number of attention heads. If the current temporal attention layer is the last layer, the entity embeddings output by the multi-head attention mechanism will not be concatenated but averaged. In order to prevent the problem of gradient dispersion and the disappearance of entity semantic information due to the deepening of the network, a skip connection is adopted in each temporal graph attention layer, which is expressed as: Where W3 is the trainable weight matrix, That is the entity embedding output by the attention layer of the graph, e init is the initial embedding of the entity output by the semantic encoder in step 2, e′ t is the output of the above multi-head attention mechanism; Step (3.3): Update the embedding representation of the relationship; learn the new embedding representation of the relationship through linear transformation, r′=W4r, the dimension of r′ is the same as The dimensions are the same, W4 is the linear transformation weight matrix, r is the embedding representation before the relationship is updated, Entity embedding output by the graph attention layer; Step (3.4): Jointly train the semantic encoder and embedding encoder, and the loss function of the two is L 编码器 =∑ (h，r，t)∈Δ ∑ (h′，r，t′)∈Δ′ [d (h，r，t) -d (h′，r，t′) +γ] + , where Δ represents the positive sample set, Δ′ represents the negative sample set, and γ represents the safety margin distance. e h , e r , e t They represent the embedded representations of h, r, and t output by the embedding encoder respectively; l1 refers to l1 regularization; Step (3.5): Calculate the encoder loss function value and update the encoder parameters.
6. The collaborator prediction method based on temporal collaborator knowledge graph according to claim 1 is characterized in that: The specific steps of the decoder in step (4) include: Step (4.1): Convolution is performed using a multi-scale convolution kernel set; given a triplet (h, r, t), e h , e r , e t Represent the embedded representations of h, r, and t output by the embedding encoder respectively; the head and tail entities and relations are represented as a triplet as an embedding matrix, Where d represents the dimension of entity embedding and relation embedding; three convolution kernels of different sizes are used Convolution is performed on the triple embedding matrix to obtain the feature matrix τ1, τ2, τ3, where Represented by the convolution kernel ω i The resulting feature matrix, c i Represents the feature matrix τ i The dimension of the generated feature matrix is concatenated. c=c1+c2+c3, where c is the dimension of p; Step (4.2): Use at least one multi-scale convolution kernel set to perform convolution to stabilize the feature extraction process; the M multi-scale convolution kernel sets are denoted as Ω′=[Ω1, ..., Ω M ], the output M feature matrices are spliced, Mc is the dimension of the concatenated vector P; Step (4.3): Score the triples according to P; use a weight matrix w d Perform matrix dot multiplication, f(h, r, t) = P w d ; The final decoder scoring function is Where * represents the convolution operation, b represents the bias, and Ω m is the m-th multi-scale convolution kernel set, and A is the embedding matrix of the triple (h, r, t); Step (4.4): Train the decoder; the loss function of the decoder training is, in is the weight vector w d L2 regularization, Δ represents the positive sample set, and Δ′ represents the negative sample set.
7. The collaborator prediction method based on temporal collaborator knowledge graph according to claim 1 is characterized in that: The specific steps of the model test in step (5) include: Step (5.1): Select two author entities that have never collaborated, obtain their corresponding embeddings and the embedding of the collaboration relationship based on the trained embedding encoder, form an embedding matrix and input it into the trained decoder to obtain its score; determine whether it is true based on the set threshold.
8. A collaborator prediction system based on temporal collaborator knowledge graph, characterized in that: include: memory and processor; The memory stores a computer program, and when the processor executes the computer program, the prediction method according to any one of claims 1 to 7 is implemented.