Author name disambiguation method based on contrastive learning and heterogeneous graph attention network
By employing contrastive learning and heterogeneous graph attention network methods, the problems of over-merging and over-splitting of papers with the same author in academic databases were solved, achieving high-purity paper clustering and accurate disambiguation results, thus improving the stability of academic databases.
Patent Information
- Application Number
- CN202211151607.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-21
- Publication Date
- 2026-01-20
- Estimated Expiration
- 2042-09-21
AI Technical Summary
In existing technologies, academic databases suffer from the problem of excessive merging and splitting of papers by authors with the same name, which affects the stable execution of subsequent algorithms. In particular, these errors are widespread and difficult to resolve during the operation of the AND algorithm.
We employ a method based on contrastive learning and heterogeneous graph attention networks. Through steps such as data preprocessing, paper representation learning, preliminary clustering, and paper cluster alignment, we utilize the BERT model and heterogeneous graph neural network to solve the disambiguation problem of authors with the same name and alleviate the phenomena of excessive merging and splitting of papers.
It effectively disambiguated authors with the same name, improved the purity and accuracy of paper clustering, ensured the quality of paper disambiguation results, and reduced misassignments in paper clusters.
Smart Images
Figure CN115481247B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of entity disambiguation technology in knowledge graph construction, and in particular to an author name disambiguation method based on contrastive learning and heterogeneous graph attention networks. Background Technology
[0002] Whether in today's big data era or the recently popular metaverse, eliminating ambiguity regarding entities with the same name is a crucial and challenging issue in the process of knowledge informatization. This problem is prevalent and of significant research importance in fields such as academic database construction, information retrieval, automated question answering, and recommender systems. Author name disambiguation has significant research value in academic database construction, and many scholars have participated in related research in recent years. Its application in academic database construction mainly focuses on authors with the same name. Currently, many papers in the system suffer from misassignment, with the ambiguity of Chinese scholars' English names being particularly severe. Many of these errors are historical mistakes that occurred during the operation of the author name disambiguation system, and these errors tend to increase as the number of papers in the system grows.
[0003] During the investigation of academic database construction, historical errors were further divided into two sub-scenarios: over-merging of papers and over-split of papers. Over-merging refers to the presence of papers by other experts in a particular expert's database, while over-split refers to the splitting of papers by the same expert into several clusters. Currently, these two phenomena are widely observed in the operation of the AND algorithm, and if these errors are not given sufficient attention and addressed, they will seriously affect the stable execution of subsequent algorithms. This problem is a major challenge in current AND research. Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide an author name disambiguation method based on contrastive learning and heterogeneous graph attention network. By using heterogeneous graph neural network, clustering, contrastive learning and other techniques, the disambiguation problem is transformed into an alignment problem by performing preliminary clustering on the papers, which better realizes the disambiguation problem of same-name authors and solves the problems of excessive paper merging and excessive paper splitting to a certain extent.
[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0006] An author name disambiguation method based on contrastive learning and heterogeneous graph attention networks includes the following steps:
[0007] S1, Data Preprocessing: Use MongoDB to access paper titles, authors, and institution information; use Python's character processing library to clean the data, remove noise, and obtain more standardized text, making it suitable for subsequent steps.
[0008] S2, Paper Representation Learning: Using contrastive learning to learn the representation of a paper and obtain the embedding of a uniform code for the paper;
[0009] S3, Preliminary paper clustering: The papers are clustered based on the principle of purity priority to alleviate the problem of excessive paper merging and obtain paper clusters;
[0010] S4, Alignment of paper clusters: Align the paper clusters obtained in the previous step using a heterogeneous graph attention network;
[0011] S5, Obtaining Disambiguation Results for Papers: Proposing over-splitting detection and over-splitting alignment algorithms to ensure the quality of paper disambiguation.
[0012] A further improvement to the technical solution of the present invention is that S2 specifically includes:
[0013] S21, using the language pre-trained model BERT to obtain the paper representation, this process is described as follows:
[0014]
[0015] In the formula, This is author a's i-th paper. It is a thesis The corresponding representation vector;
[0016] S22, Constructing positive pairs Construct negative pairs And combine positive and negative examples;
[0017] S23, introduce the training objective function h = f(bert(x)), and the training objective loss l i Described as:
[0018]
[0019] In the formula, N is the minimum batch size, τ is the temperature hyperparameter, and sim(h1, h2) is the cosine similarity.
[0020] S24, after training, the final representation vector v of the paper is obtained. i .
[0021] A further improvement to the technical solution of the present invention is that S3 specifically includes:
[0022] S31, by using the clustering process as an intermediate disambiguation process, the papers are divided into more clusters according to the rules, reducing the occurrence of different authors in the same cluster;
[0023] S32 uses LightGBN and hierarchical clustering models for clustering, and uses the negative gradient of the loss function as the residual approximation of the current decision tree to fit a new decision tree.
[0024] S33 introduces the indicator Recall. over-merge To describe the over-merging phenomenon in clustering results, this metric is defined as follows:
[0025]
[0026] In the formula, P represents the number of cases where two papers by the same author are in the same cluster; FN represents the number of cases where two papers by the same author are in two different clusters; M is the ideal clustering result, and N is the actual clustering result; Recall over-merge The higher the value, the lower the degree of over-splitting caused by clustering.
[0027] A further improvement to the technical solution of the present invention is that S4 specifically includes:
[0028] S41, generate candidate pairs for author entities with the same name;
[0029] S42. For each author entity, construct a heterogeneous graph. If candidate pairs have the same institution or co-author names or similar papers, connect them to obtain a heterogeneous graph G(V,E).
[0030] S43, using a heterogeneous graph attention network to determine author matching.
[0031] A further improvement to the technical solution of the present invention is that S43 specifically includes:
[0032] S431: The semantic embedding of each entity in the paper is obtained through the representation learning model of S2. The structural embedding of each entity is obtained by training the heterogeneous graph constructed in S42 through the LINE model.
[0033] S432, the two embeddings are combined as input features f, and the importance between different author entities e is calculated through self-attention. This process is described as follows:
[0034] t ij = self-attention(Wf) i Wf j )
[0035]
[0036] In the formula, W is the shared weight matrix, for each It refers to e i All neighboring nodes.
[0037] A further improvement to the technical solution of the present invention is that S5 specifically includes:
[0038] S51, generate pairs without repetition according to the rules of permutation and combination.<name:cid1,name:cid2> Construct a heterogeneous graph;
[0039] S52 uses a pre-trained HGAT to detect whether a pair belongs to a single author;
[0040] S53, aligning paper clusters by providing alignment rules;
[0041] S54, the process needs to be performed multiple times, the number of times is defined as loops, and the final cluster_pubs is the final disambiguation result.
[0042] A further improvement to the technical solution of the present invention is that, in S53, specifically, it includes:
[0043] S531, calculate the adjacent nodes of each node and connect the pairs of edges with the highest similarity scores between the nodes.
[0044] S532, after determining all nodes, uses DFS to implement the connected subgraph algorithm, derives the alignment rules, and then merges them.
[0045] The technological advancements achieved by this invention due to the adoption of the above technical solutions are as follows:
[0046] 1. This invention uses contrastive learning technology to fine-tune BERT-based paper representations, making the learned paper representations more suitable for the task of author name disambiguation.
[0047] 2. Based on the paper representations obtained in the previous step, this invention calculates the similarity between each paper to perform preliminary clustering of the papers and obtain fine-grained paper clusters. This process transforms the disambiguation problem into an alignment problem and makes full use of the textual semantic information of the papers for clustering, generating fine-grained paper clusters with high purity.
[0048] 3. In order to obtain the final disambiguation result, this invention needs to align fine-grained paper clusters. This process first uses the various attributes of the paper clusters to construct a heterogeneous graph network, then uses the heterogeneous graph neural network to learn the representation of the paper clusters, and finally calculates the similarity between each pair of paper clusters, and takes the most similar pairs for alignment. This process takes into account the structural information in the papers and learns through the heterogeneous graph neural network, thereby obtaining the final paper disambiguation result. Attached Figure Description
[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 This is an algorithm flowchart of the author name disambiguation method based on contrastive learning and heterogeneous graph attention network provided by the present invention;
[0051] Figure 2 This is a schematic diagram of the algorithmic model of the author name disambiguation method based on contrastive learning and heterogeneous graph attention network provided by this invention. Detailed Implementation
[0052] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.
[0053] This application provides an author name disambiguation method based on contrastive learning and heterogeneous graph attention networks, which solves the problem in the prior art of "excessive merging and splitting of papers during the operation of the AND algorithm, which seriously affects the stable execution of subsequent algorithms". This application focuses on two error scenarios that may occur during the AND process, and proposes an AND algorithm for such problems and how it can be applied in big data scenarios.
[0054] Explanation of some technical terms:
[0055] Author Name Disambiguation: This function correctly matches authors with the same name in academic databases, eliminating ambiguity caused by authors sharing the same name.
[0056] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments:
[0057] like Figure 1 , 2 As shown, an author name disambiguation method based on contrastive learning and heterogeneous graph attention networks includes the following steps:
[0058] S1, Data Preprocessing;
[0059] MongoDB is used to store information such as paper titles, authors, and institutions. Python's character processing library is used to clean the data, remove noise, and obtain more standardized text, making it suitable for subsequent steps.
[0060] S2, Paper Representation Learning;
[0061] Contrastive learning is used to learn the representation of the paper and obtain the embedding of the paper's unified code.
[0062] S3, Preliminary clustering of papers;
[0063] Clustering papers based on the principle of purity priority can alleviate the problem of excessive paper merging.
[0064] S4, alignment of paper clusters;
[0065] The paper clusters obtained in the previous step are aligned using a heterogeneous graph attention network;
[0066] S5, obtain the disambiguation results of the paper;
[0067] We propose over-splitting detection and over-splitting alignment algorithms to ensure the quality of paper disambiguation;
[0068] Specific implementation process;
[0069] S1. To address the data noise problem in the dataset and the factors that may affect the disambiguation quality, the dataset is first preprocessed, including data cleaning and analysis, including cleaning outlier data and analyzing samples from different feature perspectives. Then, feature engineering is performed on the dataset, and the processed data is used as input for subsequent model training.
[0070] S2, in paper representation learning, firstly, a language pre-trained model is used to initially obtain the paper representation, then citation contrastive learning is used to construct positive and negative example pairs, and after training, the final representation vector of the paper can be obtained; specifically, the following steps are included:
[0071] S21, firstly, the preliminary representation of the paper is obtained through the language pre-trained model BERT. This process can be described as follows:
[0072]
[0073] In the formula, This is author a's i-th paper. It is a thesis The corresponding representation vector;
[0074] S22 uses the SimCSE (Simultaneous Contrastive Learning) method to group papers with similarity together and separate papers with low similarity, constructing positive and negative examples and combining them; specifically including:
[0075] S221, Positive Example Construction: For a paper with author name 'a', use the BERT Encoder twice to obtain... and The vectors generated in each BERT process are not exactly the same, but the semantics of each pair are completely identical, thus forming a positive example pair. Furthermore, to better approximate the vector spaces obtained from papers by the same author, different papers by the same author... They are also considered positive samples, thus forming a positive pair.
[0076] S222, Negative Example Construction: To make the distance between papers by different authors with the same title greater, they are treated as negative samples. This leads to the negative example pair.
[0077] S23, p pos and p neg Combine to form Where x i It is the benchmark. is a positive example, This is a negative example. To establish the implicit relationship between training functions, a training objective function h = f(bert(x)) is introduced after the BERT-Encoder, where f is a linear layer function. The training objective loss is l. i As shown in the following formula:
[0078]
[0079] In the formula, N is the minimum batch size, τ is the (temperature) hyperparameter, and sim(h1, h2) is the cosine similarity.
[0080] After training, the representation vector v of the paper can be obtained. i .
[0081] S3, in the initial clustering of paper clusters, firstly, clustering is performed using a clustering model according to the principle of purity priority, generating as many appropriate clusters as possible. Then, the clustering is adjusted reasonably based on the over-merging index; specifically, the following steps are included:
[0082] S31. To address the issue of over-merging of papers, clustering is used as an intermediate disambiguation process. During clustering, papers are divided into as many clusters as possible according to certain rules, effectively reducing the occurrence of papers by different authors within the same cluster.
[0083] S32 introduces a metric called Recall.over-merge To describe the over-merging phenomenon in clustering results, this indicator is defined as shown in the formula:
[0084]
[0085] In the formula, TP represents the number of cases where two papers by the same author are in the same cluster; FN represents the number of cases where two papers by the same author are in two different clusters; M is the ideal clustering result, and N is the actual clustering result; Recall over-merge The higher the value, the lower the degree of over-splitting caused by clustering.
[0086] S4, in the process of aligning paper clusters, the author entities are first connected to obtain a heterogeneous graph. Then, a heterogeneous graph attention network is used to determine author matching; specifically, the following steps are included:
[0087] S41, generate candidate pairs for author entities (clusters) with the same name;
[0088] S42. For each author entity, construct a heterogeneous graph. If candidate pairs have the same institution or co-author names or similar papers, connect them to obtain a heterogeneous graph G(V,E).
[0089] S43, using heterogeneous graph attention network to determine author matching;
[0090] S431: The semantic embedding of each entity in the paper is obtained through the representation learning model of S2. The structural embedding of each entity is obtained by training the heterogeneous graph constructed in S42 through the LINE model.
[0091] S432, the two embeddings are combined as input features f. This process can be described as using self-attention between different author entities e to obtain node e. i For e j Importance ij The formula is as follows:
[0092] t ij = self-attention(Wf) i Wf j )
[0093] In the formula, W is the shared weight matrix, for each The e refers to i All neighboring nodes; where the normalized attention coefficients are as follows:
[0094]
[0095] S5, and finally, after multiple alignment rules, the paper clusters are aligned to obtain the final disambiguation result;
[0096] Includes the following steps:
[0097] S51. Generate pairs without repetition according to the rules of permutation and combination.<name:cid1,name:cid2> Construct a heterogeneous graph;
[0098] S52. Use pre-trained HGAT to detect whether a pair belongs to the same author;
[0099] S53. Align the paper clusters by providing alignment rules;
[0100] S531. Calculate the adjacent nodes of each node and connect the pairs of edges with the highest similarity scores between them.
[0101] S532. After determining all nodes, use DFS to implement the connected subgraph algorithm, derive the alignment rules, and then merge them.
[0102] S54. The process needs to be repeated multiple times, and the number of times is defined as loops; the final cluster_pubs is the final disambiguation result.
[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An author name disambiguation method based on contrastive learning and heterogeneous graph attention networks, characterized in that: Includes the following steps: S1, Data Preprocessing: Use MongoDB to access paper titles, authors, and institution information; use Python's character processing library to clean the data, remove noise, and obtain more standardized text, making it suitable for subsequent steps. S2, Paper Representation Learning: Using contrastive learning to learn the representation of a paper and obtain the embedding of a uniform code for the paper; Specifically, it includes: S21, using the language pre-trained model BERT to obtain the paper representation, this process is described as follows: ； In the formula, This is author a's i-th paper. It is a thesis The corresponding representation vector; S22, Constructing positive pairs Construct negative pairs And combine positive and negative examples; S23, Introduce the objective function for training. Training target loss Described as: ； In the formula, N is the minimum batch size. It is a temperature hyperparameter. Cosine similarity ; S24, after training, the final representation vector v of the paper is obtained. i S3, Preliminary paper clustering: The papers are clustered based on the principle of purity priority to alleviate the problem of excessive paper merging and obtain paper clusters; S4, Alignment of paper clusters: Align the paper clusters obtained in the previous step using a heterogeneous graph attention network; S5, Obtaining Disambiguation Results for Papers: Proposing over-splitting detection and over-splitting alignment algorithms to ensure the quality of paper disambiguation.
2. The author name disambiguation method based on contrastive learning and heterogeneous graph attention network according to claim 1, characterized in that: S3 specifically includes: S31, by using the clustering process as an intermediate disambiguation process, the papers are divided into more clusters according to the rules, reducing the occurrence of different authors in the same cluster; S32 uses LightGBN and hierarchical clustering models for clustering, and uses the negative gradient of the loss function as the residual approximation of the current decision tree to fit a new decision tree. S33, proposes indicators To describe the over-merging phenomenon in clustering results, this metric is defined as follows: ； In the formula, P represents the number of cases where two papers by the same author are in the same cluster; FN represents the number of cases where two papers by the same author are in two different clusters; M is the number of ideal clustering results, and N is the number of actual clustering results; The higher the value, the lower the degree of over-splitting caused by clustering.
3. The author name disambiguation method based on contrastive learning and heterogeneous graph attention networks according to claim 1, characterized in that: S4 specifically includes: S41, generate candidate pairs for author entities with the same name; S42. For each author entity, construct a heterogeneous graph. If candidate pairs have the same institution or co-author names or similar papers, connect them to obtain a heterogeneous graph G(V, E). S43, using a heterogeneous graph attention network to determine author matching.
4. The author name disambiguation method based on contrastive learning and heterogeneous graph attention network according to claim 3, characterized in that: S43 specifically includes: S431: The semantic embedding of each entity in the paper is obtained through the representation learning model of S2. The structural embedding of each entity is obtained by training the heterogeneous graph constructed in S42 through the LINE model. S432, the two embeddings are combined as input features f, and the importance between different author entities e is calculated through self-attention. This process is described as follows: ； ； In the formula, For a shared weight matrix, for each e j , It refers to e i All neighboring nodes.
5. The author name disambiguation method based on contrastive learning and heterogeneous graph attention network according to claim 1, characterized in that: S5 specifically includes: S51, generate pairs without repetition according to the rules of permutation and combination.<name:cid1, name:cid2> Construct a heterogeneous graph; S52 uses a pre-trained HGAT to detect whether a pair belongs to a single author; S53, aligning paper clusters by providing alignment rules; S54, the process needs to be performed multiple times, the number of times is defined as loops, and the final cluster_pubs is the final disambiguation result.
6. The author name disambiguation method based on contrastive learning and heterogeneous graph attention network according to claim 5, characterized in that: Specifically, S53 includes: S531, calculate the adjacent nodes of each node and connect the pairs of edges with the highest similarity scores between the nodes. S532, after determining all nodes, uses DFS to implement the connected subgraph algorithm, derives the alignment rules, and then merges them.