A disambiguation method based on decoupled representation and heterogeneous graph soft-hard attention network

Abstract

The application relates to a disambiguation method based on decoupling representation and heterogeneous graph soft and hard attention network, belongs to the technical field of entity disambiguation of academic database construction, and comprises the following steps: S1, data cleaning and preprocessing; S2, paper entity representation learning, using a language pre-training model OAG_BERT to obtain preliminary representation of the paper; S3, decoupling representation, using a variational auto-encoding model with a regularization term to decouple the features of the paper representation; S4, paper initialization clustering; S5, heterogeneous graph entity alignment, constructing a heterogeneous graph of paper and author information, and adopting a soft and hard attention network to align the entities of the heterogeneous graph; and S6, obtaining a paper disambiguation result. The application solves the error allocation problem generated in the actual application process of the traditional disambiguation method, improves the entity representation capability, and improves the performance of the disambiguation algorithm.

Description

TECHNICAL FIELD

[0001] The application relates to a disambiguation method based on decoupling representation and a heterogeneous graph soft and hard attention network, and belongs to the technical field of entity disambiguation of academic database construction. BACKGROUND

[0002] Nowadays, there are massive amounts of information generated every moment, and under the background of big data, an important topic of data mining is how to eliminate the ambiguity of homonym entities. This topic is widespread in the fields of academic knowledge base construction, information retrieval, automatic question answering, recommendation system and the like and has important research significance. In recent years, a large number of scholars have conducted in-depth research on the topic. However, in the current academic search system, there are a large number of paper and real author matching errors, many of which are historical errors generated when the author name disambiguation system runs, and these errors will gradually expand with the increase of the number of papers in the system.

[0003] In further investigation of this phenomenon, there are mainly two situations, namely, paper over-merging and paper over-splitting. Over-merging refers to the allocation of papers of multiple homonym authors to a certain person, and over-splitting refers to the splitting of papers belonging to an author into multiple authors. At present, the two phenomena widely exist in the running process of the author name disambiguation (AND) algorithm, and if these errors are not paid attention to and solved, they will seriously affect the stable execution of subsequent algorithms, and this problem is a big challenge in the current AND research. SUMMARY

[0004] The application aims to provide a disambiguation method based on decoupling representation and a heterogeneous graph soft and hard attention network, and solve the defects in the prior art.

[0005] In order to achieve the above-mentioned purpose, the technical scheme adopted by the application is:

[0006] A disambiguation method based on decoupling representation and a heterogeneous graph soft and hard attention network comprises the following steps:

[0007] S1, data cleaning and preprocessing, storing paper data into an unstructured database MongoDB, deleting repeated values in the data, completing missing values according to certain rules, and performing consistent processing on inconsistent attributes;

[0008] S2, paper entity representation learning, using a language pre-training model OAG_BERT to obtain the preliminary representation of the paper;

[0009] S3, decoupling representation, using a variational auto-encoding model with a regularization term to decouple the features of the paper representation;

[0010] S4, initialize clustering of the papers according to accuracy;

[0011] S5, heterogeneous graph entity alignment, construct a heterogeneous graph of papers and authors, and use a soft and hard attention network to perform entity alignment on the heterogeneous graph;

[0012] S6, obtain the paper disambiguation result.

[0013] Further improvement of the technical scheme of the application is that the S2 comprises:

[0014] S21, using a language pre-training model OAG_BERT to obtain a paper representation:

[0015]

[0016] wherein, represents a paper i with a paper author name a, is a vector representation corresponding to the paper;

[0017] S22, constructing a positive example pair constructing a negative example pair and combining the positive example pair and the negative example pair;

[0018] S23, introducing a trained target function h=f(bert(x)), and a trained target loss is:

[0019]

[0020] wherein, N is a minimum batch size, tau is a hyperparameter, and sim(h1, h2) is a cosine similarity;

[0021] S24, obtaining a paper author name representation vector after training

[0022] Further improvement of the technical scheme of the application is that the S3 comprises:

[0023] S31, inputting the paper author name representation vector into a variational auto-encoding model, and obtaining a representation vector z in a variational auto-encoder VAE through training, wherein the loss function in this process can be described as:

[0024] F(theta, phi, beta; emb, z) >= L(theta, phi; emb, z, beta) = E qφ (z|emb)[logp θ (emb|z)]-betaD KL (q φ (z|emb)||p(z))

[0025] Wherein, θ, φ are model parameters, β is a hyperparameter for controlling the degree of decoupling, and z is a hidden vector;

[0026] S32, adjusting the hyperparameter β to obtain the best entity decoupling representation z, and the decoupling quantification index is specifically:

[0027]

[0028] Wherein, L is the number of samples, The absolute value between the lth sample and the sample after fixed single dimension transformation.

[0029] The further improvement of the technical scheme of the application is that the S4 comprises:

[0030] S41, the clustering process is taken as an initial process of disambiguation, and the papers are divided into more clusters according to the distance index according to the accuracy rate;

[0031] S42, vector clustering is performed through a LightGBN model to construct an author entity cluster candidate set;

[0032] S43, an index Recall over-merge is proposed to describe the over-merging phenomenon of the clustering result, and the index Recall over-merge is:

[0033]

[0034] Wherein, TP represents the number of cases that two papers of the same author are in the same cluster; FN represents the number of cases that two papers of the same author are in two clusters respectively; M is the ideal clustering result number, and N is the actual clustering result number; the higher the Recall over-merge value, the lower the over-splitting degree caused by clustering.

[0035] The further improvement of the technical scheme of the application is that the S5 comprises:

[0036] S51, a candidate pair is generated for an author entity with the same name;

[0037] S52, for each author entity, a heterogeneous graph is constructed, and if the names of the institutions and co-authors of the candidate pairs are the same or the papers are similar, the candidate pairs are connected to each other to obtain a heterogeneous graph G (V, E);

[0038] S53, an author entity matching is determined by using a heterogeneous graph soft and hard attention network;

[0039] S54, the final entity and the loss function are:

[0040] e final =(LeakRelu W(e HGAT ||ehard ))

[0041] where e HGAT is the representation of entity e aggregated by the heterogeneous graph self-attention network, e hard is the representation of entity e aggregated by the heterogeneous graph hard attention mechanism.

[0042]

[0043] where d is the Euclidean distance, and η is the margin value to ensure that the loss value is greater than 0.

[0044] The further improvement of the technical scheme of the application is that the S53 comprises:

[0045] S531, obtain the semantic embedding of each paper entity through S3, construct a heterogeneous graph for training, and obtain the structural embedding of each entity;

[0046] S532, merge the semantic embedding and the structural embedding together as input features f, and obtain the importance between different author entities e through self_attention, and the process is described as:

[0047] t ij = self_attention(Wf i , Wf j )

[0048] where W is a shared weight matrix, and for each is all neighbor nodes of e i ; wherein the normalized attention coefficient is as follows:

[0049]

[0050] is a trainable parameter;

[0051] S533, the final representation of the entity e is the result of aggregating the neighbor features, and the process is represented as:

[0052]

[0053] where σ is an activation function, is a neighbor node of e i ;

[0054] S534, in order to solve the non-zero problem of attention value, a method of combining hard attention is proposed to aggregate the features:

[0055]

[0056]

[0057]

[0058] wherein, is the number of neighbor nodes of entity e, |E1| is all entities in cluster 1, is the number of entities containing the edge (h i ,r i ).

[0059] The further improvement of the technical scheme of the present application is that the S6 comprises:

[0060] S61, generating non-repeated author entities Pairs<name:cid1, name:cid2> according to the rules of permutation and combination, and constructing a heterogeneous graph;

[0061] S62, using a pre-trained iHGAT framework to detect whether a group of pairs belongs to an author;

[0062] S63, aligning the paper cluster by giving alignment rules;

[0063] S631, calculating the adjacent nodes of each node, and connecting the highest similarity score of the edges between two nodes;

[0064] S632, after all nodes are determined, using dfs to realize the connected subgraph algorithm, derive the alignment rule, and merge;

[0065] S64, the process needs to be performed multiple times, the number of times is defined as loops, and the final derived cluster set is the final disambiguation result.

[0066] Due to the adoption of the above technical scheme, the present application has the following technical effects:

[0067] The present application applies a pre-trained model OAG_BERT, contrastive learning and decoupling representation method in the representation learning stage to improve the representation ability, and further improve the performance of the downstream task; in the initialization clustering stage, the occurrence of paper over-merging phenomenon is greatly reduced in the manner of accuracy principle; in the heterogeneous graph entity alignment stage, the hard attention mechanism is adopted to solve the problem of non-zero attention value, so that more attention is paid to important neighbor node information.

[0068] The present application solves the error allocation problem generated in the actual application process of the traditional disambiguation method, improves the entity representation ability, and improves the performance of the disambiguation algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 is the algorithm flowchart of the present application;

[0070] Figure 2It is an algorithm model framework of the present application. DETAILED DESCRIPTION

[0071] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0072] A disambiguation method based on decoupling representation and heterogeneous graph soft and hard attention network, as shown in Figure 1 The method comprises the following steps:

[0073] S1, data cleaning and preprocessing

[0074] The paper data is stored in the unstructured database MongoDB, the repeated values in the data are deleted, the missing values are completed according to certain rules, and the inconsistent attributes are unified, and finally the data is processed into a format beneficial to downstream tasks.

[0075] S2, paper entity representation learning

[0076] The language pre-training model OAG_BERT is used to obtain the preliminary representation of the paper.

[0077] S3, using a variational auto-encoding model with a regularization term to decouple the features of the paper representation

[0078] S4, paper initialization clustering

[0079] The paper is initialized and clustered according to the accuracy principle.

[0080] S5, heterogeneous graph entity alignment

[0081] The paper and author information heterogeneous graph is constructed, and the soft and hard attention network is used for entity alignment of the heterogeneous graph;

[0082] S6, obtaining the paper disambiguation result

[0083] A disambiguation performance evaluation algorithm is proposed to ensure the quality of paper author name disambiguation;

[0084] The specific implementation process will be described in detail below:

[0085] A disambiguation method based on decoupling representation and heterogeneous graph soft and hard attention network, its algorithm model framework is as shown in Figure 2 :

[0086] S1, data cleaning and preprocessing

[0087] The paper data is stored in the unstructured database MongoDB, the repeated values in the data are deleted, the missing values are completed according to certain rules, and the inconsistent attributes are unified, and finally the data is processed into a format beneficial to downstream tasks.

[0088] S2, paper entity representation learning

[0089] S21, use language pre-training model OAG_BERT to obtain paper representation, which can be described as:

[0090]

[0091] Wherein represents the paper i whose paper author name is a, is the corresponding vector representation of the paper.

[0092] S22, construct positive example pair Construct negative example pair And combine the positive example pair and the negative example pair;

[0093] S23, introduce the training objective function h=f(bert(x)), the training objective loss Can be described as:

[0094]

[0095] Wherein, N is the minimum batch size, τ is the hyperparameter, sim(h1, h2) is the cosine similarity.

[0096] S24, get paper author name representation vector after training

[0097] S3, use variational auto-encoding model with regularization term to decouple the features of paper representation

[0098] S31, input the paper author name representation vector into the variational auto-encoding model, and obtain the representation vector z in the variational auto-encoder VAE through training, the loss function of this process can be described as:

[0099] F(θ,φ,β;emb,z)≥L(θ,φ;emb,z,β)=E qφ (z|emb)[logp θ (emb|z)]-βD KL (q φ (z|emb)||p(z))

[0100] Wherein, θ,φ are model parameters, β is a hyperparameter controlling the degree of decoupling, and z is a hidden vector.

[0101] S32, adjust the hyperparameter β to obtain the best entity decoupling representation z, and the decoupling quantification index can be described as:

[0102]

[0103] wherein L is the number of samples, is the absolute value between the lth sample and its transformed sample with fixed single dimension.

[0104] S4, paper initialization clustering

[0105] S41, take the clustering process as the initial process of disambiguation, divide the papers into more clusters according to the distance index according to the accuracy principle;

[0106] S42, perform vector clustering through the LightGBN model to construct an author entity cluster candidate set;

[0107] S43, propose the index Recall over-merge to describe the over-merging phenomenon of clustering results, the index definition can be described as:

[0108]

[0109] wherein TP represents the number of cases in which two papers of the same author are in the same cluster; FN represents the number of cases in which two papers of the same author are in two clusters; M is the ideal clustering result number, and N is the actual clustering result number. The higher the Recall over-merge value, the lower the over-splitting degree caused by clustering.

[0110] S5, heterogeneous graph entity alignment

[0111] S51, generate a candidate pair for an author entity (cluster) with the same name;

[0112] S52, for each author entity, construct a heterogeneous graph, and if the candidate pairs have the same institution, co-author name or similar papers, connect them to obtain a heterogeneous graph G(V, E);

[0113] S53, determine the author entity matching by using a heterogeneous graph soft and hard attention network;

[0114] S531, obtain the semantic embedding of each paper entity by S3, construct a heterogeneous graph for training, and obtain the structural embedding of each entity;

[0115] S532, merge the semantic embedding and the structural embedding together as input features f, and obtain the importance between different author entities e by self-attention, which can be described as:

[0116] t ij = self_attention(Wf i ,Wf j )

[0117] where W is the shared weight matrix, for each for all neighbors of e i where the normalized attention coefficient is as follows:

[0118]

[0119] are trainable parameters.

[0120] S533, where the final representation of entity e is the result of aggregating its neighbor features, this process can be represented as:

[0121]

[0122] where σ is the activation function, for all neighbors of e i

[0123] S534, to solve the problem of non-zero attention values, a way of combining hard attention is proposed to aggregate features, this process can be represented as:

[0124]

[0125]

[0126]

[0127] where, is the number of neighbor nodes of entity e, |E1| is all entities in cluster 1, is the number of entities containing the adjacent edge (h i , r i ).

[0128] S54, where the final entity representation and loss function of S5 can be represented as:

[0129] e final = (LeakRelu W (e HGAT ||e hard ))

[0130] where e HGAT is the representation of entity e aggregated by the heterogeneous graph self-attention network, e hard is the representation of entity e aggregated by the heterogeneous graph hard attention mechanism.

[0131]

[0132] where d is the Euclidean distance, η is the margin value to ensure that the loss value is greater than 0.

[0133] S6, obtain the paper disambiguation result​

[0134] S61、According to the rule of permutation and combination, generate non-repeated author entity Pairs<name:cid1, name:cid2>, and construct a heterogeneous graph;

[0135] S62, use the pre-trained iHGAT framework to detect whether a group of pairs belongs to an author;

[0136] S63, align the paper cluster by giving alignment rules, and the specific operation steps are:

[0137] S631, calculate the adjacent edge nodes of each node, and connect the highest similarity score of each pair of nodes.

[0138] S632, after all nodes are judged, use dfs to realize the connected subgraph algorithm, obtain the alignment rule, and merge.

[0139] S64, the process needs to be performed multiple times, the number of times is defined as loops, and the final cluster set is the final disambiguation result.

[0140] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A disambiguation method based on decoupled representation and heterogeneous graph soft-hard attention networks, characterized in that, Includes the following steps: S1. Data cleaning and preprocessing: The paper data is stored in the unstructured database MongoDB, duplicate values ​​are removed, missing values ​​are filled in according to certain rules, and inconsistent attributes are consistentized. S2. Entity representation learning for the paper: Using the pre-trained language model OAG_BERT to obtain the initial representation of the paper. S3. Decoupling representation: The paper's representation is decoupled using a variational autoencoder model with regularization terms. S4. Perform initial clustering on the papers based on accuracy. S5. Heterogeneous graph entity alignment: Construct a heterogeneous graph of paper and author information, and use soft and hard attention networks to align entities in the heterogeneous graph; S5 includes: S51. Generate candidate pairs for author entities with the same name; S52. For each author entity, construct a heterogeneous graph. If candidate pairs have the same institution or co-author names or similar papers, connect them to obtain a heterogeneous graph G(V,E). S53. Use heterogeneous graph soft and hard attention networks to determine author entity matching; S53 includes: S531. Obtain the semantic embedding of each entity in the paper through S3, construct a heterogeneous graph for training, and obtain the structural embedding of each entity. S532. The semantic embedding and structural embedding are combined as input features f, and the importance between different author entities e is calculated through self-attention. This process is described as follows: t ij =self_attention(Wf i ,Wf j ) Where W is the shared weight matrix, for each For e i All neighboring nodes; where the normalized attention coefficients are as follows: These are trainable parameters; S533, The final representation of entity e is the result of its aggregated neighbor features, and this process is expressed as: Where σ is the activation function, For e i The neighboring nodes; S534. To address the issue of non-zero attention values, a method combining hard attention is proposed to aggregate features: in, |E1| is the number of neighboring nodes of entity e, and |E1| is all entities in cluster 1. It contains adjacent edges (h) i ,r i The number of entities; S54. The final entity and loss function are: have been final =(LeakReluW(e HGAT ||e hard )) Among them, e HGAT Let e ​​be the representation of entity e aggregated by a heterogeneous graph self-attention network. hard The representation of entity e aggregated by the hard attention mechanism of heterogeneous graphs; Where d is the Euclidean distance and η is the margin value to ensure that the loss value is greater than 0; S6. Obtain the disambiguation results of the paper.

2. The disambiguation method based on decoupled representation and heterogeneous graph soft-hard attention network according to claim 1, characterized in that, S2 includes: S21. Obtain paper representations using the language pre-trained model OAG_BERT: in, Let i represent paper whose author's name is 'a'. This is the vector representation of the paper. S22, Constructing positive pairs Constructing counterexamples And combine positive and negative pairs; S23. Introduce the training objective function h = f(bert(x)), and the training objective loss l. i for: Where N is the minimum batch size, τ is the hyperparameter, and sim(h1, h2) is the cosine similarity. S24. Obtain the representation vector of the paper's author name after training.

3. The disambiguation method based on decoupled representation and heterogeneous graph soft-hard attention network according to claim 1, characterized in that, S3 includes: S31. Input the representation vector of the paper's author names into the variational autoencoder (VAE) model. The VAE model obtains the representation vector z through training, with the loss function being: F(θ,φ,β;emb,z)≥L(θ,φ;emb,z,β) =E qφ (z|emb)[logp θ (emb|z)]-βD KL (q φ (z|emb)||p(z)) Where θ and φ are model parameters, β is a hyperparameter controlling the degree of decoupling, and z is a latent vector; S32. Adjust the hyperparameter β to obtain the optimal entity decoupling representation z, wherein the decoupling quantification index is specifically: Where L is the number of samples, It is the absolute value between the l-th sample and its transformed sample with a fixed single dimension.

4. The disambiguation method based on decoupled representation and heterogeneous graph soft-hard attention network according to claim 1, characterized in that, S4 includes: S41. Using the clustering process as the initial disambiguation process, and based on the principle of accuracy, divide the paper into more clusters according to the distance index; S42. Construct a candidate set of author entity clusters by performing vector clustering using the LightGBN model; S43. Propose the indicator Recall over-merge To describe the over-merging phenomenon in clustering results, the metric Recall is used. over-merge for: Where TP represents the number of cases where two papers by the same author are in the same cluster; FN represents the number of cases where two papers by the same author are in two different clusters; M is the ideal clustering result, and N is the actual clustering result; Recall over-merge The higher the value, the lower the degree of over-splitting caused by clustering.

5. The disambiguation method based on decoupled representation and heterogeneous graph soft-hard attention networks according to claim 1, characterized in that, S6 includes: S61. Generate unique author entities Pairs according to the rules of permutation and combination.<name:cid1,name:cid2> Construct a heterogeneous graph; S62. Use the pre-trained iHGAT framework to detect whether a pair belongs to the same author; S63. Align the paper clusters by providing alignment rules; S631. Calculate the neighboring nodes of each node, and connect the pairs of edges with the highest similarity scores between the nodes. S632. After determining all nodes, use DFS to implement the connected subgraph algorithm, derive the alignment rules, and then merge them. S64. The process needs to be repeated multiple times, with the number of iterations defined as loops. The final set of clusters obtained is the final disambiguation result.

Images