A paper relevance analysis method based on graph convolutional neural network and knowledge base

By constructing a multi-dimensional paper knowledge base and combining Inception-GCN and NOCO models, the problems of insufficient extraction and overfitting between papers in the existing technology are solved, and a more comprehensive paper correlation analysis is achieved.

CN114741519BInactive Publication Date: 2024-12-03BEIJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210150878.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-18
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art only considers the citation relationship between papers in the analysis of paper correlation, resulting in the loss of correlation information between papers that are similar in the research field but have no direct or indirect citation relationships, and the existing models are prone to over-specified during training. Combined problem.

Method used

A paper knowledge base containing citation relationships, author relationships and professional term relationships between papers was constructed. Combined with graph convolutional neural network, an improved Inception-GCN model was proposed for paper category division, and a NOCO model was used for paper community discovery, through multi-dimensional analysis Get more comprehensive relevance information.

Benefits of technology

The nonlinear relationships between papers were effectively extracted, the accuracy of paper category division and community discovery was improved, the risk of overfitting was reduced, and more comprehensive paper correlation analysis results were obtained.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114741519B_ABST
    Figure CN114741519B_ABST
Patent Text Reader

Abstract

The present invention proposes a new paper relevance analysis method: extract key information from a collection of papers, construct a collection of papers knowledge base, combine graph convolutional neural networks, propose an improved Inception‑GCN model to complete paper category division, use the NOCO model to complete paper community discovery, and then complete the relevance analysis of papers in the collection of papers knowledge base. The present invention proposes a new graph node classification model: the Inception‑GCN model. Combining the Inception method originally used for the CNN model with the GCN model can enable the new model to effectively solve the overfitting and smoothing problems while enhancing the feature learning ability. Experiments have shown that using this model for paper node classification can achieve better results than the existing technology.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology processing, and in particular to a paper correlation analysis method based on graph convolutional neural network and knowledge base. Background Art

[0002] In the 21st century, the continuous emergence of academic research results reflects the progress of the times and the development of science and technology, but at the same time, the storage, analysis and management of a large number of results are also very laborious. In recent years, the number of papers in various fields has increased dramatically, and the innovation points are diverse; however, people's demand for reviewing and counting papers related to specific fields and tasks has become increasingly strong, which has brought considerable challenges to paper analysis technology.

[0003] When conducting correlation analysis on papers, the two most important subtasks are paper classification and paper community discovery. Paper classification is to divide the literature according to research fields and research tasks, and add category labels to the paper entities in the paper collection knowledge base. This division has predetermined categories, and there is no intersection between categories. Paper community discovery refers to dividing the literature into "communities" and adding community labels to the paper entities in the paper collection knowledge base. The so-called "community" is a collection of labeled papers, with close connections within the collection and sparse connections outside the collection. Unlike paper classification, community discovery does not have predetermined labels, and there may be overlaps between "communities".

[0004] Based on the above two subtasks, this paper proposes a new paper relevance analysis method: extracting key information from the paper collection, building a knowledge base, combining graph convolutional neural network, proposing an improved Inception-GCN model to complete paper category classification, using the NOCO model to complete paper community discovery, and then completing the relevance analysis of the papers in the paper collection.

[0005] Related key technologies

[0006] knowledge base:

[0007] A knowledge base is a collection of knowledge stored, organized, managed, and used in computers to describe concepts and their relationships in the physical world. Knowledge is expressed in the form of "entity-relationship-entity" or "entity-attribute-attribute value", and a knowledge base is a collection of such triplets. Since entities are connected to each other through relationships, a knowledge base is a complex network knowledge structure that can more appropriately describe, store, and manage complex knowledge systems to meet subsequent analysis needs.

[0008] Graph Convolutional Neural Networks:

[0009] Graph Convolutional Neural Network (GCN) is a scalable method for semi-supervised learning of graph data based on convolutional neural network variables. It is a deep learning graph embedding method without random walks. Graph data is different from traditional sequences and images. It is an infinite-dimensional non-Euclidean space data. There are unordered nodes of variable sizes on the graph, and each node has a different number of adjacent nodes. The complexity of graph data makes existing deep learning methods face huge challenges in processing. Graph Convolutional Neural Network extends the convolution operation from traditional data to graph data. In essence, it is a feature extractor like convolutional neural network. It is the basis of many complex graph neural network models. We can use the features extracted by GCN to perform downstream work such as node classification, graph classification, and link prediction on graph data.

[0010] Prior art related to the technical solution of the present invention

[0011] One of the prior arts related to the technical solution of the present invention, as shown in the attached Figure 1 What is shown is an academic big data analysis method based on the citation relationship between papers [Tan Zhaowei, Liu Changfeng, Zhou Jinguang, et al. Academic big data analysis method based on the citation relationship between papers:, CN105808729B[P]. 2019.]. The invention provides an academic big data analysis method based on the citation relationship between papers, and the implementation process includes the following three steps: (1) after correlation analysis and processing of the local paper data set, a paper citation network is constructed in the database; (2) according to the citation relationship in the paper citation network, an analysis algorithm is constructed, and the importance and mutual relationship of the nodes in the paper citation network are obtained through the analysis algorithm, and the importance of the paper relative to the central paper is obtained; (3) the one-to-one citation relationship of the paper is converted into a mapping set of the citation direction and a mapping set of the cited direction, and the development path between the specified papers is obtained in the paper citation network through the extraction algorithm, and the importance of the path is calculated according to the paper importance obtained in (2).

[0012] Disadvantages of the prior art 1

[0013] The citation relationship between papers does play a vital role in the correlation analysis of papers, but it is far from enough to only consider the citation relationship between papers. The scale of the collection of papers is very large, and there is no direct or indirect citation relationship between many papers with very similar research fields or research tasks. Only considering the citation relationship will lose the correlation information. The technical solution of the present invention not only takes into account the citation relationship, but also takes into account the author relationship between papers, the relationship of commonly existing professional terms, the paper category attributes, the paper community attributes, etc. It can retain paper information from multiple dimensions and analyze correlation.

[0014] Prior art related to the present invention

[0015] Technical solution of prior art 2

[0016] Another prior art related to the technical solution of the present invention is as follows Figure 2 What is shown is a method for constructing a paper classification model based on a gated graph attention network [Wang Meihong, Qiu Linling, Li Han, et al. Method and system for constructing a paper classification model based on a gated graph attention network:.]. The paper classification model proposed by the invention technology includes several layers connected in sequence. Among them, each layer includes a graph neural network structure and a classifier. In the first layer, the graph neural network structure and the classifier are directly connected. In the tth layer, the graph neural network structure and the classifier are connected through a gated structure, and t is an integer greater than 1; determine the feature matrix of each paper sample i in the sample data set, input the feature matrix of each paper sample into the classification model, and train the classification model with the type of each paper sample as a label.

[0017] Disadvantages of the second prior art

[0018] The paper classification model proposed in the technical solution of this invention introduces a gating mechanism based on the graph attention network to aggregate the information of distant nodes, which can improve the classification accuracy to a certain extent. However, since the number of parameters involved in the training of the model is very large, it has high requirements for the data set, making training difficult and prone to overfitting problems. Summary of the invention

[0019] The purpose of the invention is to provide a new paper relevance analysis method based on graph convolutional neural network and knowledge base. First, extract key information from the paper collection and build a paper knowledge base; then transform the classification problem of the paper collection into a node classification problem in the knowledge base, and propose an improved Inception-GCN model in combination with the graph convolutional neural network to complete the paper classification work on the constructed knowledge base, and add the obtained category attributes to the paper entity of the knowledge base; finally, use the NOCO model based on the graph convolutional neural network to complete the community discovery of the paper collection, and add the obtained community attributes to the paper entity of the knowledge base.

[0020] In order to achieve the purpose of the invention, the technical solution provided by the present invention is:

[0021] The beneficial effects of the present invention are:

[0022] The present invention effectively solves the following technical problems existing in the prior art:

[0023] (1) The nonlinear relationship extraction between papers is single.

[0024] The prior art generally only uses the citation relationship or author relationship between papers. Many papers with the same research field and similar research problems do not directly or indirectly have the above two relationships. Therefore, the prior art generally loses a lot of correlation information between papers. When constructing the paper knowledge base, the technical solution of the present invention not only uses the citation and author relationships between the above papers, but also adds "professional terminology" entities, and adds the non-linear relationship between the "paper" entity and the "professional terminology" entity to the knowledge base. Professional terms are obtained by partial manual annotation combined with named entity recognition methods, which can extract key content related to fields and technologies in papers to a great extent. These are all impossible to obtain in citation relationships and author relationships.

[0025] (2) Existing technologies generally only perform correlation analysis on a collection of papers by implementing the paper classification task. Paper classification is divided according to pre-set categories, and the category attribute information that can be added to the paper is limited. While implementing the paper category division, the present invention also implements the paper community discovery task. Paper community discovery can analyze a collection of papers without supervision, and the community attributes obtained cover a wider range and have richer content than category attributes. The two tasks complement each other and can obtain a more comprehensive correlation analysis result for the collection of papers.

[0026] (3) This paper proposes a new graph node classification model: Inception-GCN model. Combining the Inception method originally used for CNN model with GCN model can effectively solve the overfitting and smoothing problems while enhancing the feature learning ability of the new model. Experiments show that the model can achieve better results than the existing technology when used for paper node classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the overall framework of the prior art solution 1;

[0028] Figure 2 It is a schematic diagram of the overall framework of the second prior art solution;

[0029] Figure 3 Schematic diagram of the Inception network structure. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings.

[0031] The present invention provides a new paper correlation analysis method based on graph convolutional neural network and knowledge base, comprising the following steps:

[0032] Step 1), extract key information from the collection of papers and build a collection of papers knowledge base;

[0033] By analyzing the relationship between papers, the present invention selects the following three nonlinear relationships to construct a collection of papers knowledge base: the citation relationship between papers, the relationship between papers and authors, and the inclusion relationship between papers and professional terms. Among them, professional terms are not included in the paper data set, but are obtained by partial manual annotation combined with named entity recognition methods. After comparing commonly used named entity recognition methods such as SpaCy, NLTK, and Stanford NER, this embodiment preferentially selects SpaCy as the professional terminology named entity recognition method.

[0034] The entities and entity attributes of the final paper collection knowledge base are shown in Table 1, and the relationships between entities are shown in Table 2.

[0035]

[0036] Table 1 Entities and attributes

[0037] paper author Terminology Citing / cited relationship Authorship Containment relationship

[0038] Table 2 Relationships between entities

[0039] Step 2) Classify the collection of papers and divide the documents in the collection according to the content and the direction involved. This classification has predetermined categories, and there is no intersection between categories. The core idea is to transform the classification problem of the collection of papers into the node classification problem in the knowledge base. The detailed steps are as follows:

[0040] Step 2.1), use external knowledge to label some categories of the paper collection;

[0041] The SciKG large knowledge graph from Aminer contains a lot of content concept relationships in the computer field, from which we search for literature that intersects with the paper collection and label it by category.

[0042] Step 2.2), combining graph convolutional neural network to propose an improved Inception-GCN model semi-supervised classification algorithm to classify unlabeled papers;

[0043] Traditional graph convolutional neural network (GCN) formula:

[0044]

[0045] Problems: If you want to enhance the learning ability of GCN, you can use two methods: deepening the number of layers and increasing the features of each layer. Both methods can increase the perception domain of GCN. But at the same time, it will increase the difficulty of training and the risk of overfitting.

[0046] The present invention introduces the Inception network structure into GCN to solve the above problems. The Inception network structure module is as follows Figure 3 As shown, multiple convolutions with different receptive fields are performed in parallel, and the depth slices of different filters are concatenated into the same layer to merge the results.

[0047] Let the graph convolution network formed by simply concatenating the perceptual domain R be h R (X, A)

[0048]

[0049] Among them, the activation function of the first layer is ReLU, the activation function of the second layer is softmax, X is the feature matrix of the initial nodes on the graph, A is the adjacency matrix, W(l) is the weight matrix unique to each layer, which is the matrix to be trained, and (l) indicates which layer this is the matrix.

[0050] Inception-GCN after merging:

[0051]

[0052] where ∪ R=1,2,3 h R (..., A) represents R parallel branches receiving the same input, and the concatenation of the outputs of each branch is used as the overall output.

[0053] Step 2.3), complete the paper category division work on the constructed paper collection knowledge base, and add the obtained category attributes to the paper entity of the knowledge base.

[0054] Step 3) Use the NOCO model based on graph convolutional neural network to complete the community discovery of the paper collection, and add the obtained community attributes to the paper entity in the knowledge base.

[0055] This paper selects the NOCO model proposed by Shchur et al. [Shchur, Oleksandr, Günnemann, Stephan. Overlapping Community Detection with Graph Neural Networks [C]. The First International Workshop on Deep Learning on Graphs: Methods and Applications (DLG'19) 2019.] to complete the paper community detection task in the knowledge base created in this paper. This model has been proven on some datasets with correct community annotations that it can well restore the original community in an unsupervised situation.

[0056] The NOCO model consists of two parts: the Bernoulli-Poisson model and the graph convolutional neural network model. The Bernoulli-Poisson model is used to model the graph structure, setting the value on the node adjacency matrix as the result of a probability distribution, and using the community belonging vector of each node as the parameter of this probability distribution. The graph convolutional neural network model models the vector representing the node community affiliation and the adjacency matrix and attribute vector of the nodes on the graph to obtain the node community affiliation vector.

[0057] The specific steps of the NOCO model to complete community discovery and add community attributes to paper entities are as follows:

[0058] In step 3.1), the Bernoulli-Poisson model is used to model the graph structure, and the community belonging vector of each node is used as a parameter to generate a probability distribution as the value on the node adjacency matrix.

[0059] The generation method is that when the subordinate relationship is given When the adjacency matrix A uv For independent and identically distributed sampling according to this formula: Auv~Bernoulli(1-exp(-F u F v T ), the log-likelihood function of the parameter F on this distribution is

[0060]

[0061] After balancing the parameter weights, that is, weighting them according to the number of empty edges on the graph and the number of edges on the graph, the loss function used is obtained.

[0062]

[0063] Among them, F (l) is the row vector representing the community affiliation of node l, that is, the lth row of the matrix F.

[0064] Step 3.2), use the graph convolutional neural network model to model the vector representing the node community affiliation, the adjacency matrix of the nodes on the graph, and the attribute vector to generate a community affiliation matrix.

[0065] The model uses a two-layer graph convolutional neural network with the formula:

[0066]

[0067] Each layer uses ReLU as the activation function to reduce the amount of calculation. X represents the input, It represents the critical matrix of the graph with its own edges, W(l) is the weight matrix unique to each layer, which is also the matrix to be trained, and (l) indicates which layer the matrix is.

[0068] By finding the appropriate neural network parameters θ, the final subordinate matrix F is obtained:

[0069]

[0070] In step 3.3), the model outputs affiliation vectors for each node, adding community attributes to the paper entity.

[0071] Through the description of the above embodiments, the main technical key contributions of the present invention are:

[0072] (1) A new method for constructing a collection of papers knowledge base: It takes into account a variety of nonlinear relationships, includes rich entities such as "professional terms", and adds attributes such as paper categories and communities.

[0073] (2) The graph convolutional neural network was improved and the Inception-GCN model was proposed for the paper category classification task.

[0074] (3) Implement paper relevance analysis on two subtasks: paper category classification and paper community discovery, to optimize the analysis effect.

[0075] The described embodiments are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

Claims

1. A paper correlation analysis method based on graph convolutional neural network and knowledge base, characterized in that: The following steps are involved: Step 1), extract key information from the collection of papers and build a collection of papers knowledge base; Step 2), paper category division, divide the documents in the paper collection according to the content and the direction involved, and propose an improved Inception-GCN model based on the graph convolutional neural network to complete the paper category division work on the constructed paper collection knowledge base, and add the obtained category attributes to the paper entity of the paper collection knowledge base, including: Step 2.1), use external knowledge to label some categories of the paper collection; Step 2.2), combining graph convolutional neural network to propose an improved Inception-GCN model semi-supervised classification algorithm to classify unlabeled papers; In the step 2.2), the improved Inception-GCN model semi-supervised classification algorithm absorbs and performs multiple convolutions with different perception domains, and splices the depth slices of different filters into the same layer, thereby merging the results. The specific steps are: Let the graph convolution network formed by simply concatenating the perceptual domain R be h R (X, A) Among them, the activation function of the first layer is ReLU, the activation function of the second layer is softmax, X is the feature matrix of the initial nodes on the graph, A is the adjacency matrix, W(l) is the weight matrix unique to each layer, that is, the matrix to be trained, (l) indicates which layer this is the matrix; Inception-GCN after merging: where ∪ R=1,2,3 h R (…, A) represents R parallel branches receiving the same input, and the concatenation of the outputs of each branch is used as the overall output; Step 2.3), complete the paper category classification work on the constructed paper collection knowledge base, and add the obtained category attributes to the paper entity of the paper collection knowledge base; Step 3), use the NOCO model based on graph convolutional neural network to complete the community discovery of the collection of papers, and add the obtained community attributes to the paper entity of the collection of papers knowledge base.

2. According to claim 1, a method for analyzing paper relevance based on graph convolutional neural network and knowledge base, characterized in that: In the step 1), the construction of the paper collection knowledge base is to construct the paper collection knowledge base by using three non-linear relationships: the citation relationship between papers, the work relationship between papers and authors, and the inclusion relationship between papers and professional terms.

3. According to claim 2, a method for analyzing paper relevance based on graph convolutional neural network and knowledge base, characterized in that: The professional terms are obtained by partial manual annotation combined with named entity recognition method.

4. According to claim 3, a method for analyzing paper relevance based on graph convolutional neural network and knowledge base is characterized in that: The named entity recognition method is one of the SpaCy named entity recognition method, the NLTK named entity recognition method or the Stanford NER named entity recognition method.

5. According to claim 1, a method for analyzing paper relevance based on graph convolutional neural network and knowledge base, characterized in that: In step 3), the paper partitioning task in the paper collection is transformed into a community discovery task on the graph knowledge base, and the specific steps are as follows: Step 3.1), use the Bernoulli-Poisson model to model the graph structure, and use the community belonging vector of each node as a parameter to generate a probability distribution as the value on the node adjacency matrix; Step 3.2), using the graph convolutional neural network model to model the vector representing the node community affiliation, the adjacency matrix of the nodes on the graph, and the attribute vector to generate a community affiliation matrix; Step 3.3), according to the community affiliation matrix, output the affiliation vector for each node and add the community attribute to the paper entity.

6. The method for analyzing paper relevance based on graph convolutional neural network and knowledge base according to claim 5, characterized in that: In the step 3.1), the parameter generation probability distribution is generated in the following manner: When affiliation is given When the adjacency matrix A uv For independent and identically distributed sampling according to this formula: Auv~Bernoulli(1-exp(-F u F v T ), the log-likelihood function of the parameter F on this distribution is: After balancing the parameter weights, that is, weighted processing is performed according to whether the nodes on the graph are related, the loss function used is obtained; Among them, F (1) is the row vector representing the community affiliation of node l, that is, the lth row of the matrix F.

7. The method for analyzing paper relevance based on graph convolutional neural network and knowledge base according to claim 5, characterized in that: In the step 3.2), The graph convolutional neural network model uses a two-layer graph convolutional neural network, and the formula is: Each layer uses ReLU as the activation function to reduce the amount of calculation; X represents the input, represents the critical matrix of the graph with its own edges, W(l) is the weight matrix unique to each layer, which is also the matrix to be trained, and (l) indicates which layer the matrix is; By finding the appropriate neural network parameters θ, the final subordinate matrix F is obtained: