A cross-domain paper recommendation method based on heterogeneous data embedding

Abstract

The application relates to a cross-domain paper recommendation method based on heterogeneous data embedding and belongs to the technical field of big data mining application and information processing. First, a field is divided for a data set, a directed acyclic graph is constructed for each subject, a latent Dirichlet distribution model is used to extract field semantics, and cross-domain correlation is learned through a maximum information coefficient. Then, papers and users are respectively expressed in the form of vectors through heterogeneous data embedding, and the mapping relationship between cross-domain papers is trained through the citation and non-citation relationship between the papers. If a keyword search is provided by a user, the field of the user is divided according to the keyword, and the interest list of the user is the cited literature. Finally, a cross-domain paper recommendation model is used to recommend papers for the user. The application can automatically evaluate the cross-domain correlation of papers, effectively overcome the technical defects that a traditional method only takes the content similarity of papers as the recommendation basis, and greatly improve the recommendation precision and efficiency.

Description

Technical Field

[0001] This invention relates to a cross-domain paper recommendation method, specifically a cross-domain paper recommendation method based on heterogeneous data embedding, belonging to the field of big data mining applications and information processing technology. Background Technology

[0002] Academic papers / documents, as a common learning resource, while promoting scientific research and innovation, have also caused a serious problem of information overload. Paper recommendation technology can automatically analyze and recommend massive amounts of papers, thereby helping researchers better utilize literature resources, quickly acquire knowledge, and effectively improve the usability of academic literature.

[0003] In recent years, knowledge dissemination has led to a rapid increase in the number of interdisciplinary papers published and the number of cross-disciplinary citations. In 2019, Gates et al., in a paper published in *Nature*, pointed out that the number of cross-disciplinary paper citations and the diversity of disciplinary sources cited have been continuously increasing in recent years. In 2021, Xie et al., in a paper published in *SIGIR*, analyzed the publication volume of five authoritative interdisciplinary journals over the past decade, showing that the overall number of cross-disciplinary research papers published has shown a year-on-year growth trend. Interdisciplinary papers / documents have become a crucial component of paper / document recommendations.

[0004] However, existing paper recommendation methods only suggest papers similar to researchers' research content, without considering the importance of cross-disciplinary papers. Therefore, it is of great significance and practical value to utilize big data mining and information processing technologies to accurately help researchers find the latest research results in relevant fields from massive academic resources. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings and defects of existing technologies, and to solve the technical problem of how to provide users with better cross-domain paper recommendation services in the context of the continuous development of interdisciplinary fields, especially how to improve the accuracy and efficiency of push notifications. This invention creatively proposes a cross-domain paper recommendation method based on heterogeneous data embedding.

[0006] The innovation of this method lies in combining heterogeneous data embedding and cross-domain relevance, and proposing a method to measure cross-domain paper recommendation, which can accurately recommend cross-domain papers to users engaged in interdisciplinary research.

[0007] The present invention is achieved using the following technical solution.

[0008] A cross-domain paper recommendation method based on heterogeneous data embedding includes the following steps:

[0009] Step 1: Divide the dataset into domains. First, construct a Directed Acyclic Graph (DAG) for each subject. Then, use a Latent Dirichlet Distribution Model to extract domain semantics. Finally, learn cross-domain associations through the Maximum Information Coefficient (MIC).

[0010] Step 2: Represent papers and users as vectors using heterogeneous data embedding. First, vectorize the entities and relationships in the literature graph. Then, train the mapping relationship between cross-domain papers using citation and non-citation relationships. Specifically, if a user provides keyword search, the user's domain is segmented based on the keywords, and the user's interest list becomes the cited literature.

[0011] Step 3: Utilize a cross-domain paper recommendation model to recommend papers to users. First, merge the different vector representations of papers and users. Then, train a three-layer feedforward neural network model to recommend cross-domain papers to users.

[0012] Beneficial effects

[0013] Compared with existing technologies, the method of this invention can automatically evaluate the cross-domain relevance of papers, effectively overcoming the technical shortcomings of traditional paper recommendation methods that rely solely on the similarity of paper content as the basis for recommendation, and significantly improving recommendation accuracy and efficiency. Attached Figure Description

[0014] Figure 1 This is the overall architecture diagram of the present invention.

[0015] Figure 2 This is the overall flowchart of the present invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. Figure 1 This is the overall structure diagram of this method.

[0017] like Figure 2 As shown, a cross-domain paper recommendation method based on heterogeneous data embedding includes the following steps:

[0018] Step 1: Divide the data into domains for the paper dataset.

[0019] Specifically, step 1 includes the following steps:

[0020] Step 1.1: Construct a Directed Acyclic Graph (DAG) for each subject.

[0021] A discipline consists of a set of concepts and theories. A discipline usually contains several categories, and each category corresponds to a research direction of the discipline, which is called a field.

[0022] Let G represent a DAG of subject categories, where each node n and its child nodes represent an academic field, n∈G.

[0023] Let r denote the root node, and r ∈ G. Each node, except the root node r, has a parent node, denoted as pa(n), where pa(n) ∈ G. For a given corpus of academic papers P, the vocabulary is represented by Z.

[0024] The paper's metadata includes the title, author, abstract, keywords, and one or more category labels, corresponding to nodes in G.

[0025] For any paper p with labels n∈G, p∈P, let the variable y n,p =1 to mark, otherwise y n,p =0. Based on the labeling constraint of the directed acyclic graph, if p has a label n, i.e. y n,p If y = 1, then p will be marked by all its parent nodes, i.e., y pa(n),p =1; If p is not marked by n, then p will not be marked by n's child nodes either, i.e., y ch(n),p =0.

[0026] Step 1.2: Use the Latent Dirichlet Distribution Model to extract domain semantics.

[0027] The way words are used is open and constantly evolving in the field of scientific research. Therefore, this invention uses the latent Dirichlet distribution model to extract domain semantics.

[0028] Specifically, for any paper p∈P, its topic probability distribution p(l|p) is learned through the Latent Dirichlet Allocation (LDA) model, thereby obtaining the probability distributions p(w|l) and p(n|l) of the word w of p and any node n∈G on l. Here, l represents the topic.

[0029] Step 1.3: Learn cross-domain associations using the Maximum Information Coefficient (MIC).

[0030] Specifically, for any paper p, the discipline to which p belongs is determined by the correlation between the probability distribution p(w|l) of p and the probability distribution p(n|l) of any node. In this invention, MIC is used to measure the correlation between p(w|l) and p(n|l).

[0031] The MIC and related definitions are as follows:

[0032] Let the binary variable data set D = {(f, c)}, where f and c represent a set of binary variables. Using the binning method, D is divided into grids in a two-dimensional coordinate system so that the data points in D are all distributed within the grids. If the number of divisions of the grid on the longitudinal x-axis is a and the number of divisions on the transverse y-axis is b, this divided grid is called an a-by-b grid. Given a grid G, let D| G denote the distribution of D within the grid G. For a given data set D, different grid divisions will result in different D| G .

[0033] Definition 1: Sample mutual information. Given the binary variable data set D = {(f, c)}, and the number of divisions on the longitudinal x-axis is a and the number of divisions on the transverse y-axis is b, then the sample mutual information of D is shown in Equation 1:

[0034] I*(D, a, b) = max I(D|G) (1)

[0035] where I(D| G ) represents the mutual information calculated from the distribution of D within G, and I*(D, a, b) represents the maximum mutual information calculated within all possible a-by-b grids G.

[0036] Definition 2: Sample feature matrix. Given the binary variable data set D = {(f, c)}, and the number of divisions on the longitudinal x-axis is a and the number of divisions on the transverse y-axis is b, then the sample feature matrix is defined as shown in Equation 2:

[0037]

[0038] Definition 3: Maximal information coefficient. Given the binary variable data set D = {(f, c)}, and the number of divisions on the longitudinal x-axis is a and the number of divisions on the transverse y-axis is b, satisfying the upper limit condition ab < B(n), then the maximal information coefficient MIC of D is defined as shown in Equation 3:

[0039]

[0040] where B(n) is a function of the sample size n, B(n) = n e . Usually, the value of the parameter e is 0.6.

[0041] Step 2: Through heterogeneous data embedding, represent the paper in the form of a vector.

[0042] As scientific research continues to develop, academic resources are becoming increasingly multimodal and heterogeneous. Based on the divide-and-conquer approach, these data can be used to construct features using different methods depending on their different data formats.

[0043] In this invention, heterogeneous data is divided into two categories: graph-based relational data and cross-domain association-based probabilistic data. Different embedding methods are used for different data types, and the feature representations of each type are concatenated to represent the vectorized representation of the final entity.

[0044] Includes the following steps:

[0045] Step 2.1: Relational data processing.

[0046] An academic graph typically consists of five entities and five relationships. Entities include papers, publications, publication years, authors, and research institutions. Relationships include the writing relationship between authors and research papers, the affiliation relationship between authors and research institutions, the publication relationship between papers and publications, the temporal relationship between papers and publication years, and the citation relationship between papers. These entities and relationships are selected because they exist in major literature databases, thus ensuring relatively complete data consistency and low acquisition costs.

[0047] In this invention, the TransD algorithm is specifically used to vectorize entities and relations in a bibliographic graph. The TransD algorithm takes into account the diversity of entity and relation representations. Compared with other graph embedding methods, the TransD algorithm has fewer training parameters and does not involve matrix-vector multiplication operations, which enables it to perform calculations on large-scale graph data.

[0048] In the TransD algorithm, each named entity / relation is assigned two vectors. The first vector captures the meaning of the entity / relation, and the second is used to construct a mapping matrix. Let the constructed graph g contain entities h, t, and relations r, where t ∈ N and r ∈ R, where N represents the set of entities (nodes) and R represents the set of relations. h, t, and r are used to represent the vectorized embedding results of the head node, tail node, and relation, respectively. Given a triple (h, t, r), its corresponding representation vectors are h, h, and r. p , t, t p ∈R n And r, r p ∈R m Where the suffix p denotes the mapping vector, h, h p , t, t p Let r and r' represent vectors. p R represents a relation vector. n R represents the set of vectors representing entities. mA set of vectors representing relations. Therefore, the mapping matrix M rh and M rt ∈R mxn Defined as:

[0049] M rh =r p h p T +I mxn (4)

[0050] M rt =r p t p T +I mxn (5)

[0051] Among them, I mxn T represents the identity matrix, and T represents the matrix transpose.

[0052] Based on this, the mapping of entities is further defined as:

[0053] h ⊥ =M rh h (6)

[0054] t ⊥ =M rt t (7)

[0055] Among them, h ⊥ Let t represent the mapping vector of h. ⊥ The mapping vector for t.

[0056] Use the scoring function f r (h,t) evaluates the vector results after embedding the triple (h,t,r) into the trained vector:

[0057] f r (h,t)=||h ⊥ +rt ⊥ || 2 2 (8)

[0058] By giving the constraints ||h||2≤1, ||t||2≤1, ||r||2≤1, ||h ⊥ ||2≤1 and ||t ⊥ ||2≤1, the loss function for embedding training As shown in Equation 6:

[0059]

[0060] Where S represents a positive triple, h′ and t′ represent negative instances, and r′ represents the relationship between them; [x] +f represents the absolute value of x, γ represents the hyperparameter of edge distance, and its value is always greater than 0; r () denotes the integral function of r, f r′ () denotes the integral function of r′; the triplet existing in the academic knowledge graph is represented as (h,r,t)∈S, while the negative example triplet generated randomly during training is represented as S′. (h,r,t) .

[0061] Step 2.2: Text data processing.

[0062] Probabilistic data is used to measure cross-domain relationships between papers, while knowledge dissemination between academic fields manifests as cross-domain paper citations. Cross-domain citation behavior reflects authors' preferences for knowledge in another field and their consideration of its potential influence. Therefore, this invention trains a mapping relationship between cross-domain papers by analyzing citation and non-citation relationships between papers.

[0063] Specifically, let W and M be two different academic fields, where T w and T M These represent the subject areas to which the papers in W and M belong, respectively, and P W and P M It is a corpus of labeled academic papers from two different domains. Let E = {(p,q)|p∈P} W ,q∈P M ,p cite q} represents the set of positive samples, where the elements are paper pairs with a citation relationship from W to M; E′={(p,q)|p∈P W ,q∈P M ,p does not reference q} to represent a negative sample.

[0064] The objective of this invention is to learn an asymmetric mapping function π from W and M to the same plane Q based on E and E′. The optimized function π is shown in Equation 10:

[0065] L(θ)=∑ (p,q)∈E,(p,q′)∈E′ (MIC(π(x p ),π(y q ))-MIC(π(x p ),π(y q′ (10)

[0066] Where θ represents all the parameters that need to be learned, x p Let y represent the mapping vector of p in the neighborhood W. q Let y represent the mapping vector of q in the neighborhood M. q′ Let q′ be the mapping vector in the neighborhood M.

[0067] The asymmetric mapping function π is represented as a vector using a three-layer feedforward neural network, which has two exponential linear unit layers and one sigmoid layer. The neural network employs a semantic-based representation as presented in the paper. As input, the vector mapped to the Q-plane The output is shown in Equation 11.

[0068]

[0069] in,

[0070] To improve robustness, a hinge loss function is used, and a regularization term is added. The loss function is transformed into:

[0071]

[0072] Where ∈ represents the parameters of the hinge loss function. This indicates regularization.

[0073] Step 3: Use a cross-domain paper recommendation model to recommend papers to users.

[0074] Specifically, it may include the following steps:

[0075] Step 3.1: Merge the different vector representations of the paper and the user.

[0076] Users have two search methods: one is to provide several keywords to initiate a search request, and the other is to provide articles of interest to initiate a search request. The data is then represented in vector form according to the text data processing methods described above.

[0077] Based on the user's published papers, publications, publication year, authors, research institutions, and the relationships between five entities, a knowledge graph is constructed and represented in vector form.

[0078] To preserve first-order and second-order similarity during the embedding process for each user or paper, this invention trains LINE(1st) and LINE(2nd) models respectively, and concatenates the representation vectors learned by the two models to obtain a long vector.

[0079] Step 3.2: Train a three-layer feedforward neural network model to recommend cross-domain papers to users.

[0080] Specifically, targeting user U's search information, similar to the process of text data processing, if the user provides keywords for the search, the list of papers that the user is interested in will be used as positive examples, and papers that do not exist in the list of interests will be randomly selected as negative examples; if the user provides papers for the search, the division of positive and negative examples is exactly the same as for text data.

[0081] Therefore, a three-dimensional training set is constructed for any user U.<U,p+,p-> We learn the parameters of the paper embedding model, where p+ is a positive example corresponding to tp and p- is a negative example corresponding to U. We train the paper embedding model using the loss function L(θ) of Equation 12, so that the model obtains the MIC between U and p+, and also obtains the MIC between U and p-.

Claims

1.A method for cross-domain paper recommendation based on heterogeneous data embedding, characterized in that, The method comprises the following steps: Step 1: Divide the field for the paper data set; first, construct a directed acyclic graph (DAG) for each discipline, then extract the field semantics using a latent Dirichlet distribution model, and finally learn the cross-domain association through the maximum information coefficient (MIC); Step 2: Embed the heterogeneous data to represent the papers and users in the form of vectors; First, the entities and relationships in the literature graph are represented in the form of vectors, and then the mapping relationship between cross-domain papers is trained through the citation and uncitation relationships between papers; If the user provides a keyword search, the user's field is divided according to the keywords, and the user's interest list is the cited literature; The mapping relationship between cross-domain papers is trained through the citation and uncitation relationships between papers, and the method is as follows: Let W and M be two different academic fields, where Tw and TM represent the subject fields to which the papers in W and M belong, and PW and PM be the labeled paper corpora of the two fields; let E = {(p, q) | p ∈ PW, q ∈ PM, p cites q} represent the positive sample set, and the elements in the set be the paper pairs with citation relationships from W to M; E' = {(p, q) | p ∈ PW, q ∈ PM, p does not cite q} represent the negative sample; Based on E and E', learn the asymmetric mapping function π from W and M to the same plane Q, and the optimization function π is shown in formula 10: L(θ) =∑ (p,q)∈W,(p,q')∈W' (MIC(π(x p ),π(y q ))-MIC(π(x p ),π(y q' )) (10) where θ represents all the parameters to be learned, x p represents the mapping vector of p in the domain W, y q represents the mapping vector of q in the domain M, y q' represents the mapping vector of q' in the domain M; The asymmetric mapping function π is obtained by a three-layer feedforward neural network with two exponential linear unit layers and one sigmoid layer, which employs the semantic-based paper representation As input, the vector mapped to the Q-plane represents the output, as shown in Equation 11; Wherein, To improve robustness, the hinge loss function method is adopted, and a regularization term is added, and the loss function is converted to: where ∈ denotes the parameter of the hinge loss function, denotes the regularization; Step 3: Use the cross-domain paper recommendation model to recommend papers for users; First, combine the different vector representations of papers and users, and then train a three-layer feedforward neural network model to recommend cross-domain papers for users. 2.The cross-domain paper recommendation method based on heterogeneous data embedding of claim 1, wherein, In step 1, the method for constructing a directed acyclic graph (DAG) for each discipline is as follows: Let G represent a DAG of a discipline classification, where each node n and its child nodes represent an academic field, n ∈ G; Let r represent the root node, and r ∈ G; each node except the root node r has a parent node, denoted as pa(n), pa(n) ∈ G; For a given paper corpus P, the vocabulary in P is denoted as Z; The metadata of the paper includes the title, author, abstract, keywords, and one or more classification labels, which correspond to the nodes in G; For any paper p with label n e G, p e P, use variable y n,p = 1 to mark, otherwise y n,p = 0; based on the DAG marking constraint, if p is marked with n, i.e., y n,p = 1, then p will be marked by all its parents, i.e., y pa(n),p = 1; if p is not marked by n, then p will not be marked by n's children, i.e., y ch(n),p = 0; The method for extracting field semantics using a latent Dirichlet distribution model is as follows: For any paper p ∈ P, learn its topic probability distribution p(l|p) through the latent Dirichlet distribution model (LDA), and obtain the probability distribution p(w|l) and p(n|l) of the word w and any node n ∈ G on l; wherein, l represents the topic; The method for learning cross-domain association through the maximum information coefficient (MIC) is as follows: For any paper p, determine the discipline to which p belongs through the correlation between the distribution probability p(w|l) of p and the distribution probability p(n|l) of any node, and use MIC to measure the correlation between p(w|l) and p(n|l). 3.The cross-domain paper recommendation method based on heterogeneous data embedding of claim 1, wherein, In step 2, the heterogeneous data is divided into two categories: graph-based relational data and probability data based on cross-domain association. Different embedding methods are used for different data forms, and the feature representations of each type are concatenated to represent the final entity vector representation. First, the TransD algorithm is used to vectorize the entities and relationships in the literature graph, Let the constructed graph g exist entity h, entity t and the relationship r between entities, t∈N, r∈R, N represents the entity set, R represents the relationship set; h, t, r are used to represent the vectorization embedding results of the head node, the tail node and the relationship between entities respectively; given a triple (h, t, r), the corresponding representation vector is h, h p , t, t p ∈R n and r, r p ∈R m , wherein the suffix p represents a mapping vector, h, h p , t, t p represents a vector, r, r p represents a relationship vector, R n represents a vector set of entities, R m represents a vector set of relationships; therefore, the mapping matrix M rh and M rt ∈R mxn are defined as: M rh = r p h p Τ + I mxn (4) M rt = r p t p Τ + I mxn (5) where I mxn denotes the identity matrix, and T denotes the matrix transpose; Based on this, the mapping of entities is further defined as: h ⊥ = M rh h (6) t ⊥ = M rt t (7) where h ⊥ denotes the mapping vector of h, t ⊥ denotes the mapping vector of t; Using a scoring function f r (h, t) pair of triplets (h, t, r) embedding trained vector results are evaluated: f r (h,t) = ||h ⊥ +r-t ⊥ || 2 2 (8) By giving the constraints ||h||2≤1, ||t||2≤1, ||r||2≤1, ||h ⊥ ||2≤1 and ||t ⊥ ||2≤1, the loss function for embedding training As shown in Equation 6: Wherein, S represents positive example triplets, h' and t' represent negative examples, and r' represents the relationship between them;[x] + represents the absolute value of x, and γ represents the hyperparameter of edge distance, which is always greater than 0;f r () represents the integral function of r, and f r’ () represents the integral function of r'; the triplets existing in the academic knowledge graph are represented as (h, r, t) ∈ S, and the negative example triplets generated by random extraction in training are represented as S' ( h,r,t). 4.The method of claim 1, wherein, In step 3, a three-layer feedforward neural network model is trained to recommend cross-domain papers to users as follows: Taking the search information of user U as the target, if the user provides keyword search, the list of papers of interest to the user is taken as the positive example, and a paper that does not exist in the list of interest is randomly selected as the negative example; if the user provides paper search, the positive and negative examples are divided, and the text data is exactly the same; For any user U, a three-dimensional training set is constructed to learn the parameters of the paper embedding model, where p+ is a positive example corresponding to tp, and p- is a negative example corresponding to U. The loss function L(θ) of formula 12 is used to train the paper embedding model, so that the model obtains MIC between U and p+, and MIC between U and p-.

Images