A topic classification method for scientific literature based on graph diffusion enhanced hyperedge

Abstract

The present invention relates to the field of graph deep learning, and specifically to a method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedges, comprising the steps of: collecting scientific and technological document data; extracting text features from scientific and technological documents using unique hot encoding to obtain a document feature matrix; constructing a graph data structure with the obtained scientific and technological document data; inputting the graph data structure into an original graph-hyperedge generator to alleviate the imbalance in the number of nodes; inputting the graph data structure into a dual-channel hyperedge enhancer based on graph diffusion to alleviate the imbalance in node topology; training an unbiased classifier based on an enhanced hypergraph and a hypergraph neural network, inputting the enhanced hypergraph into the unbiased classifier to obtain the final classification result, and subject-classifying scientific and technological documents. The present invention solves the technical problem that the method using graph neural networks in the prior art leads to inaccurate subject classification of scientific and technological documents due to the imbalanced distribution of subject categories of scientific and technological documents.

Description

Technical Field

[0001] The present invention relates to the field of graph deep learning, and in particular to a method for subject classification of scientific and technological literature based on graph diffusion enhanced hyperedges. Background Art

[0002] With the rapid expansion of scientific and technological resources, the number of scientific and technological literature has increased dramatically. More than 2 million scientific and technological literature are published worldwide each year, and the annual growth rate continues to grow at about 7%. These documents not only provide rich ideological resources for the scientific community, but also increase the knowledge burden of scientific and technological practitioners. Therefore, scientific and technological literature reviewers face huge challenges. They need to quickly and accurately divide scientific and technological literature into topics and fields, and distribute them to peers in professional fields for in-depth review. Although existing automated text classification tools have made some progress, the topics of scientific and technological literature show the problem of category imbalance, and the number of documents on popular topics far exceeds the number of documents on unpopular topics. According to the statistics of ICLR 2024 accepted papers, the conference accepted a total of 2,260 articles, of which about 470 involved the popular topic "Diffusion Model", and about 350 involved the popular topics "Big Model" and "Graph Neural Network", respectively. These three topics accounted for half of the total number of articles accepted. In contrast, the number of papers involving the unpopular topic "Data Statistics" was only 80, ranking tenth among all topics, indicating that the number of papers on more unpopular topics will be even smaller. It can be seen that there is an obvious imbalance in the distribution of scientific and technological literature, which greatly reduces the accuracy of literature subject classification and forces manual review to become an indispensable link, which not only increases resource consumption but also prolongs the review cycle. In addition, the inaccurate classification of scientific and technological literature obviously reduces the efficiency of literature retrieval by researchers.

[0003] In this context, it is particularly urgent and important to study effective subject classification methods for scientific and technological literature. From an academic perspective, studying and improving the performance of classification algorithms can improve the classification accuracy of scientific and technological literature, allowing researchers to more accurately retrieve literature in related fields. This not only helps to accelerate the progress of academic research, but also promotes interdisciplinary communication and cooperation. Secondly, from a practical application perspective, an efficient and balanced classification system can greatly improve the efficiency of document management and paper review. This is particularly important for application areas such as library information systems, paper review systems, and academic search engines. Improving classification algorithms can help these systems better serve users, provide more accurate search results, and optimize user experience. Finally, these improvements can not only meet the challenges in the field of scientific and technological literature, but may also provide new solutions to classification problems in other fields.

[0004] Current research and practice show that traditional scientific literature subject classification methods focus on using word embedding and traditional neural network (CNNs) algorithms in text classification, which have significant limitations in capturing the relationship between documents. Graph neural networks (GNNs) can model the relationship between documents well due to their advantages in modeling graph structure data. Therefore, the present invention creates a scientific literature citation network, solves the problem of scientific literature subject classification based on graph neural networks, and converts scientific literature subject classification into a node classification problem on a graph structure.

[0005] In fact, in the node classification task of graph structure, category imbalance is a common challenge, which provides a good idea for us to solve the problem of reduced classification accuracy caused by the distribution difference between popular and unpopular subject documents in the subject classification of scientific and technological literature. Category imbalance in graph structure mainly includes two situations: node number imbalance and node topology imbalance. The former refers to the significant difference in the number of nodes in different categories, and the latter refers to the asymmetric topological structure of the nodes on the decision boundary. These two imbalance problems may lead to the offset of the decision boundary, thereby reducing the performance of the classifier. In recent years, research usually adopts oversampling methods to synthesize new nodes by fusing node features and construct new topological structures. However, this method may introduce noise and destroy the topological characteristics of the original graph, thereby affecting the generalization ability of the model. In addition, most methods only consider the imbalance of the number of nodes and ignore the impact of node topological imbalance, resulting in low classifier accuracy, which in turn leads to inaccurate classification results of scientific and technological literature themes. Therefore, it is of great significance to deeply explore and develop scientific and technological literature thematic classification technology based on graph neural networks.

[0006] Existing methods for solving category imbalance can be mainly divided into two categories: generative methods and algorithmic methods.

[0007] The use of generative methods to solve the class imbalance problem of graphs mainly starts from the data and balances the class distribution by increasing the number of minority class nodes. The DR-GCN (double regularized graph convolutional network) method generates virtual secondary nodes through additional conditional GAN ​​(adversarial neural network) and adjusts the features of these virtual nodes to closely align with the features of adjacent nodes. The GraphSMOTE (a method for constructing an embedding space to encode the similarity between nodes) method applies SMOTE (synthetic minority class oversampling technique) to the graph, uses a feature extractor based on GNNs (graph neural network) to generate minority class nodes, trains an edge generator to generate topological structures for new nodes, thereby forming an enhanced graph, and trains an unbiased classifier on the enhanced graph. The GraphENS (a method for comprehensive classification of neighbor-aware ego networks based on class imbalance node classification) method uses the ego network of minority class nodes and the ego network of randomly selected nodes to synthesize a new ego network that retains the characteristics of minority class nodes as much as possible, and introduces a node mixing method based on saliency to generate the central node of the ego network. The GraphSR (a self-training strategy and data enhancement algorithm designed specifically for handling the problem of unbalanced node classification in graph data) method uses two components, namely the similarity-based selection module and the reinforcement learning module, to predict pseudo labels for the unlabeled node set, and obtain a complementary set of labeled node sets to balance the node distribution. The GraphSHA (a method for synthesizing hard samples for unbalanced node classification) method uses the probability vector generated by the graph neural network to calculate the confidence of the node, and uses the confidence to extract the anchor node and the auxiliary node, and performs feature synthesis and edge synthesis on the two nodes to expand the decision boundary of the minority class.

[0008] Algorithm-level methods aim to modify learning algorithms to solve class imbalance. The ReNode (a semi-supervised node classification method using class imbalance of the metric affecting conflict detection) method designs a metric Totoro that affects conflict detection to measure the degree of node topological imbalance. The smaller the Totoro (the metric affecting conflict detection), the closer the node is to the center of the topological class. The training weight of the labeled node close to the center of the topological class is increased to alleviate the topological imbalance. ReNode is often used with other data-level methods to solve the class imbalance problem of graphs. The TAM (a topology-aware loss function adjustment method) method finds that compensating for minority class nodes that deviate from the class connectivity pattern is likely to cause minority class nodes to be misclassified. It proposes to intelligently adjust the class margin according to the degree of deviation from the connectivity pattern, which can effectively improve class imbalance. The ImGCL (a graph contrastive learning framework for imbalanced node classification) method uses the k-means clustering algorithm to generate an initial pseudo-label distribution, uses PBS (a stepwise balanced sampling strategy) for downsampling, and proposes an adaptive downsampling scheme based on node centrality to balance the class distribution on all classes.

[0009] The imbalance problem of semi-supervised graph node classification tasks includes imbalance in the number of nodes and imbalance in node topology. Currently, most of the work focuses on generative methods, because generative methods not only focus on the semantic features of nodes, but also on the topological features of nodes. These focuses lead to models that show superior performance in solving category imbalance problems.

[0010] However, the new samples synthesized by the generative method cannot be guaranteed to belong to the minority class. These synthesized nodes may be noise, resulting in reduced generalization performance of the classifier. In addition, due to the connectivity of the graph structure, it is also necessary to generate a topological structure for the new samples, which not only makes the generated topological structure unreasonable, but also destroys the topological structure of the original graph.

[0011] A large part of the current research only considers the problem of unbalanced node quantity. Unbalanced node topology is also a problem worthy of attention. The current methods proposed to solve the problem of unbalanced node topology are only applicable to evenly connected graphs. If there are nodes on the decision boundary that are not connected to the topological center node or are too far away from the topological center node, then the method will play a small or no role. Summary of the invention

[0012] The purpose of the present invention is to provide a method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedges to solve the technical problem of inaccurate subject classification of scientific and technological documents using graph neural network methods in the prior art. The specific technical solution is as follows:

[0013] The present invention provides a method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedge, comprising the following steps:

[0014] S1. Collect scientific literature data;

[0015] S2, extract text features from scientific and technological literature using one-hot encoding to obtain a document feature matrix;

[0016] S3, constructing a graph data structure with the scientific literature data obtained in S1, in which a node represents a scientific literature, an edge represents the relationship between two scientific literatures that cite or are cited, and using the literature feature matrix obtained in S2 to form the original graph to define a scientific literature citation network;

[0017] S4, inputting the graph data structure into the original graph-hyperedge generator to obtain a similar hyperedge set to alleviate the imbalance of the number of nodes, specifically: the nodes connected by the edges of the original graph are respectively formed into adjacent hyperedge sets, and the similarity between the nodes is calculated by cosine similarity to obtain a similar hyperedge set;

[0018] S5. Input the graph data structure into a dual-channel hyperedge enhancer based on graph diffusion to obtain an enhanced hyperedge set to alleviate the imbalance of node categories. Specifically, the nodes of unpopular topic documents are randomly sampled, and global candidate nodes and local candidate node sets are identified to form an enhanced hyperedge set.

[0019] S6, de-duplicate and merge the adjacent hyperedge set and similar hyperedge set obtained in S4 and the enhanced hyperedge set obtained in S5 to obtain an enhanced hypergraph;

[0020] S7. Use the enhanced hypergraph and hypergraph neural network to train an unbiased classifier, input the enhanced hypergraph into the unbiased classifier to obtain the final classification result, and thus classify the topics of scientific and technological literature.

[0021] The further improvement of the scientific literature subject classification method based on graph diffusion enhanced hyperedge of the present invention is that step S5 includes: using graph diffusion convolution to model and obtain the connectivity characteristics of the nodes in the graph, and the graph diffusion convolution is defined by the diffusion matrix:

[0022] ;

[0023] in, represents the weight coefficient, T represents the transfer matrix, S represents the diffusion matrix, k Indicates the number of hops. , T The eigenvalue of ;

[0024] The diffusion matrix is ​​calculated, the unweighted rigid edges of the original graph are converted into weighted elastic edges, and the original graph is transformed into a weighted dense graph. k The weighted dense graph is sparsely constructed by the method and threshold method to obtain the weighted graph of the target.

[0025] The further improvement of the scientific literature subject classification method based on graph diffusion enhanced hyperedge of the present invention is that the graph diffusion convolution includes a PPR version, and the transfer matrix of the PPR version graph diffusion T and weight coefficient They are:

[0026] ;

[0027] ;

[0028] in, represents the adjacency matrix, represents the degree matrix, represents the transition probability.

[0029] The further improvement of the scientific literature subject classification method based on graph diffusion enhanced hyperedge of the present invention is that the graph diffusion convolution includes HK version, and the transfer matrix of HK version graph diffusion is T and weight coefficient They are:

[0030] ;

[0031] ;

[0032] in, represents the adjacency matrix, represents the degree matrix, e represents the natural base, represents the diffusion time, Indicates the factorial of the number of hops.

[0033] A further improvement of the scientific literature subject classification method based on graph diffusion enhanced hyperedge of the present invention is that step S5 further includes: calculating the unpopular subject document nodes Category For all nodes that do not belong to class The sum of the association strengths of the nodes, that is, the sum of the edge weights of the elastic edges between the nodes, and the smallest value is The global candidate nodes are based on the unpopular topic document nodes. The global candidate node of the hyperedge centered on is defined as:

[0034] ;

[0035] in, Indicates the unpopular subject document node, represents the set of scientific literature nodes, Indicates the maximum subject category number, Indicates the subject category to which the unpopular subject literature belongs. Indicates the subject category number, Indicates the subject category. Represents the unpopular topic document node in the weighted graph and Document Node The weight of the weighted edge between .

[0036] The further improvement of the subject classification method of scientific and technological documents based on graph diffusion enhanced hyperedge of the present invention is that step S5 also includes: selecting the edge weight ranking of the head node of the elastic edge with the unpopular subject document node as the top node. k The tail node is the node with the highest correlation strength, i.e. the node with the least popular subject literature ranks first. k Nodes of form a local candidate node set and participate in the enhanced hyperedge set Composition, unpopular subject literature nodes The local candidate node set Defined as:

[0037] ;

[0038] in: represents the local candidate node set, Represents a set of scientific and technological literature nodes;

[0039] Unpopular topic document nodes obtained using dual-channel hyperedge enhancer based on graph diffusion , local candidate node set and global candidate nodes Combine to get a new enhanced hyperedge set .

[0040] The further improvement of the subject classification method of scientific and technological literature based on graph diffusion enhanced hyperedge of the present invention is that step S6 comprises: enhancing the adjacency matrix of the hypergraph Defined as:

[0041] ;

[0042] in, , Represents a node set; , represents a hyperedge set; Representation Node Super edge , Representation Node Not a super edge .

[0043] The further improvement of the scientific literature subject classification method based on graph diffusion enhanced hyperedge of the present invention is that the degree of the enhanced hypergraph includes the degree of the node, the degree of the node is defined as:

[0044] ;

[0045] in, Represents the nodes in the hypergraph adjacency matrix and super edge connectivity, Represents a hyperedge The weight of .

[0046] The further improvement of the subject classification method of scientific and technological literature based on graph diffusion enhanced hyperedge of the present invention is that the degree of the enhanced hypergraph also includes the degree of the hyperedge, and the degree of the hyperedge Defined as:

[0047] ;

[0048] in, Represents the nodes in the hypergraph adjacency matrix and super edge connectivity, Represents a node set.

[0049] A further improvement of the scientific literature subject classification method based on graph diffusion enhanced hyperedge of the present invention is that step S6 further includes: establishing a hypergraph convolution layer:

[0050] ;

[0051] in: represents the enhanced hypergraph of the node degree matrix, represents the weight of the hyperedge, represents the hyperedge matrix of the enhanced hypergraph, represents the adjacency matrix of the enhanced hypergraph, Indicates The nodes of a layer represent a hypergraph of matrices, Indicates The nodes at the +1 level represent the hypergraph of the matrix, Indicates The document feature matrix of the layer, Indicates The parameters of the layer, represents the activation function;

[0052] A hypergraph neural network is generated using two layers of hypergraph convolution and a softmax function, and the parameters are updated using the cross entropy loss function:

[0053] ;

[0054] in: L represents the loss function, V represents the set of scientific literature nodes, c Indicates the subject category to which the document belongs. Indicates the maximum subject category number, Representation Node Tags, represents the indicator function, It means equal to, Indicates that the unbiased classifier classifies scientific literature nodes The probability of belonging to any of the topic categories.

[0055] The application of the technical solution of the present invention has the following beneficial effects:

[0056] The subject classification method of scientific and technological literature based on graph diffusion enhanced hyperedges is oriented to semi-supervised node classification tasks, analyzes the category imbalance problem, and divides the category imbalance problem into node quantity imbalance and node topology imbalance. To address the problem of unbalanced number of nodes, the present invention proposes an original graph-hyperedge converter in view of the shortcomings of existing methods in generating noise nodes and destroying the original topological structure, introduces the perspective of hypergraph structure conversion problem to alleviate the node imbalance problem, converts the original graph edges into adjacent hyperedge sets to solve the problem of generating hypergraphs, generates similar hyperedge sets for nodes based on node similarity, and uses hyperedges to directly construct high-order relationships of nodes, which preliminarily alleviates the problem of unbalanced number of nodes. It not only eliminates the hidden dangers of noise nodes, but also solves the problem of existing methods destroying the topological structure; to address the problem of node topological imbalance, existing methods rarely consider it. In view of the shortcomings of existing methods that cannot work on non-connected nodes or long-distance nodes, the present invention proposes a hyperedge enhancer based on graph diffusion, which generates enhanced hyperedges only for a few types of nodes, among which the global candidate nodes represent relative topological center nodes after graph diffusion and related calculations. The use of hyperedges cleverly solves the shortcomings of existing methods, and while alleviating node topological imbalance, the enhanced hyperedges further alleviate the imbalance in the number of nodes. The use of hypergraph neural networks is superior to traditional graph neural networks used in existing methods. It can comprehensively learn the new high-order relationships between hyperedges and nodes in the hypergraph constructed by the present invention, improve the accuracy of semi-supervised node classifiers under the problem of class imbalance, and solve the technical problem of inaccurate subject classification of scientific and technological literature using graph neural network methods in the existing technology, thereby improving research efficiency and helping researchers quickly locate relevant literature through precise classification, avoiding wasting time on a large amount of irrelevant information.

[0057] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0059] Figure 1 It is a flow chart of the subject classification method of scientific and technological literature based on graph diffusion enhanced hyperedge of the present invention;

[0060] Figure 2 It is a schematic diagram of the original graph of scientific and technological literature subject classification based on graph diffusion enhanced hyperedge of the present invention;

[0061] Figure 3 It is a schematic diagram of a hyperedge set of a scientific and technological literature subject classification method based on graph diffusion enhanced hyperedges of the present invention;

[0062] Figure 4 It is a schematic diagram of a hypergraph of a scientific and technological literature subject classification method based on graph diffusion enhanced hyperedges of the present invention;

[0063] Figure 5 It is a schematic diagram of a hyperedge generator of an original graph of a scientific and technological literature subject classification method based on graph diffusion enhanced hyperedges of the present invention;

[0064] Figure 6 It is a schematic diagram of a dual-channel hyperedge enhancer of the scientific and technological literature subject classification method based on graph diffusion enhanced hyperedge of the present invention. DETAILED DESCRIPTION

[0065] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0066] Due to the wide application of graphs in practical fields such as social networks and citation networks, graph neural networks have developed rapidly and performed well in processing graph downstream tasks. Generally, graph neural networks assume that data satisfies independent and identical distribution (iid) during training. However, graph data in real scenarios are difficult to satisfy independent and identical distribution. The laws of nature cause them to be inherently class-imbalanced. For example, when social networks identify fake users, the number of ordinary users far exceeds the number of fake users.

[0067] Class imbalance is a common challenge in real-world graph-based node classification scenarios, which manifests itself in the form of imbalanced node numbers and imbalanced node topology. Node imbalance manifests itself in the large difference in the number of nodes in different categories, which means that graph neural network learning is no longer fair and is more biased towards the majority class, resulting in overfitting of the minority class. Node topology imbalance manifests itself in the form of an asymmetric topological structure of nodes on the topology boundary. The labels of nodes are determined by nodes near the decision boundary, and the asymmetric topological structure may cause the decision boundary to shift.

[0068] Recent research can be divided into generative methods and algorithm-level methods. Generative methods generate new nodes and edges by sampling mixed node features to balance the distribution of node numbers. Algorithmic-level methods focus on minority class nodes by modifying the weights and parameters of the objective function. Currently, most of the work focuses on generative methods, because generative methods not only focus on the semantic features of nodes, but also on the topological features of nodes. These focuses lead to models that perform better in solving class imbalance problems. However, the new samples synthesized by generative methods and their topology are not representative enough, which may lead to the generation of noise and the destruction of the original topological structure. In addition, current generative methods are basically limited to alleviating the problem of imbalanced node numbers, while ignoring the problem of imbalanced node topology, which will cause the decision boundary to shift and reduce the performance of the classifier.

[0069] The present invention can address the problems that the existing recognition method used in the subject classification of scientific and technological documents generates noise, destroys the original topological structure, and does not consider the imbalance of node topology.

[0070] See also Figure 1 to Figure 6 As shown, a method for subject classification of scientific and technological literature based on graph diffusion enhanced hyperedges includes the following steps:

[0071] S1. Collection includes N Scientific and technological literature data of the articles;

[0072] S2. Use unique hot encoding to extract text features from the content of each scientific document to obtain the document feature matrix , Represents the feature dimension. The specific process is as follows:

[0073] S201, segmenting scientific and technological literature into words;

[0074] S202, deleting stop words and punctuation marks from the words segmented in S201;

[0075] S203, removing duplicates from the words obtained in S202 and adding them to the vocabulary;

[0076] S204, setting the index position of each word in the vocabulary to 1;

[0077] S3. Label each scientific document to obtain a label set , all tags belong to The subject categories are , which contains popular topic categories such as Figure 2 The majority class nodes in also include unpopular topic categories, such as Figure 2 The minority class nodes in

[0078] S301, construct a graph data structure with the obtained scientific literature data, where a node represents a scientific literature, and an edge represents the relationship between two scientific literatures that cite or are cited. To define a scientific literature citation, where represents the set of scientific literature nodes, represents the adjacency matrix of the graph, Represents the document feature matrix, each row of which represents the attribute of a node. Represents the dimension of the node attribute. Representation Node Neighbor nodes of

[0079] S4. Existing solutions are mainly generative methods, which balance the category distribution by generating new nodes, but these methods may introduce noise and destroy the original topological structure, thus affecting the generalization ability of the model. Hypergraph, as a new modeling method, provides a promising solution. The hyperedges in the hypergraph can associate multiple nodes. This structure can directly capture the high-order relationships between nodes without introducing new nodes or edges. Each new hyperedge actually represents a new node association. This method effectively avoids the noise that may be introduced by traditional generative methods and retains the topological structure of the original graph. Therefore, Figure 5 As shown, the present invention inputs the graph structure into an original graph-hyperedge generator to alleviate the quantity imbalance. The specific process is as follows:

[0080] S401: In order to maintain the original structure of the original graph, the nodes connected by the edges of the original graph are firstly formed into adjacent hyperedge sets ,Right now ,in, represents an adjacent hyperedge, and represents any two scientific literature nodes, Indicates the literature and literature Is there a reference association? If the value of is 1, it means there is a reference association, otherwise it does not exist. At this stage, the super edge set is no different from the original edge set, and only the original information can be learned, and the category imbalance problem still exists;

[0081] S402: In order to alleviate the imbalance of the number of nodes, the present invention introduces a similar hyperedge set , which is composed of anchor nodes , and Most similar k Nodes The hyperedges are composed of express The cosine similarity is used to calculate the similarity between two nodes. This method enriches the expression of minority nodes in the learning process by increasing the similarity relationship between nodes in the hypergraph, and preliminarily solves the problem of insufficient learning knowledge due to the insufficient number of minority nodes.

[0082] S5, such as Figure 6 As shown, for the problem of unbalanced node topology, the present invention inputs the graph structure into a dual-channel hyperedge enhancer based on graph diffusion to alleviate the node category imbalance. The specific process is as follows:

[0083] S501. The motivation of the present invention is to mine relatively balanced nodes (i.e. global candidate nodes) to form hyperedges with anchor nodes and enhance the topological balance of anchor nodes. The present invention uses graph diffusion convolution (GDC) to model the connectivity characteristics of nodes in the graph. This method converts the unweighted rigid edges of the original graph into weighted elastic edges, thereby more accurately reflecting the importance and connection strength between nodes. Graph diffusion is a process of transforming the unweighted rigid edges of the original graph into weighted elastic edges. Defined by:

[0084] ;

[0085] in, represents the weight coefficient, T represents the transfer matrix, S represents the diffusion matrix, Represents the number of hops. In order to ensure the convergence of graph diffusion, there are two restrictions: ① ,②Transfer matrix T The eigenvalue of There are two commonly used versions of graph diffusion convolution, namely personalized PageRank (PPR) version and heat kernel (HK) version.

[0086] Transfer matrix of PPR version graph diffusion and weight coefficient They are:

[0087] ;

[0088] ;

[0089] in, is the adjacency matrix, is the degree matrix, is the transition probability, Indicates the number of hops.

[0090] Transfer matrix of HK version graph diffusion and weight coefficient They are:

[0091] ;

[0092] ;

[0093] in, is the adjacency matrix, is the degree matrix, e represents the natural base, e is approximately equal to 2.07, is the diffusion time, Indicates the number of hops. Indicates the factorial of the number of hops.

[0094] After calculating the diffusion matrix, the unweighted rigid edges of the original graph are converted into weighted elastic edges, and the original graph is transformed into a weighted dense graph. k method (a data processing operation) and threshold method to perform sparseness and finally obtain the weighted graph of the target ,in, represents the set of scientific literature nodes, represents the weighted adjacency matrix, Represents the document feature matrix, each row of which represents the attribute of a node. Represents the dimension of node attributes. It is worth noting that the elastic edges of the weighted graph converted by the graph diffusion method are not necessarily exactly the same as the rigid edges of the original graph. This is because the graph diffusion process may reveal strong association information between nodes that are not directly connected in the original graph, and these relationships will be presented in the weighted graph in the form of elastic edges. At the same time, the association between directly connected nodes that originally existed in the original graph may be weak, and may be filtered out after the sparsification strategy, resulting in the deletion of these edges in the weighted graph. This conversion is not just a simple assignment of weights to the edges in the graph, but uses the graph diffusion method to discover and reconstruct the strength of the association between nodes.

[0095] S502, Random sampling of unpopular topic literature nodes , identify global candidate nodes and local candidate node set To form an enhanced hyperedge set .

[0096] S503: Considering the impact of topological imbalance, the node with the weakest association with other classes is selected as the global candidate node of the current class. Specifically, calculate the unpopular topic document nodes Category For all nodes that do not belong to class The sum of the association strengths of the nodes, that is, the sum of the edge weights of the elastic edges between the nodes, and the smallest value is The global candidate nodes are also based on the unpopular topic document nodes. The global candidate node of the hyperedge centered at is defined as:

[0097] ;

[0098] in, Indicates the unpopular subject document node, represents the set of scientific literature nodes, Indicates the maximum subject category number, Indicates the subject category to which the unpopular subject literature belongs. Indicates the subject category number, Indicates the subject category. Represents the unpopular topic document node in the weighted graph and Document Node The method uses the transformation thinking to solve the problem of unbalanced node topology, that is, the node with the weakest association with other classes is actually in a relatively central position in the topological structure. In hypergraph learning, the hyperedge adds the topologically relatively central node, which can also allow more information of the hyperedge to be transmitted to the hyperedge with similar nodes, thereby improving the accuracy of the classifier.

[0099] S504: In order to enhance the representation of unpopular subject document nodes, the present invention selects the unpopular subject document nodes as the head nodes of the elastic edges with the edge weight ranking at the top. k The tail node, that is, the node with the highest correlation strength among the unpopular subject documents k Nodes of form a local candidate node set and participate in the enhanced hyperedge set Composition, unpopular subject literature nodes The local candidate node set Defined as:

[0100] ;

[0101] in, Indicates the unpopular topic document node and Document Node The weight of the weighted edge between Indicates the number of hops. Represents a set of scientific literature nodes. The strength of the association between nodes is determined by the weight of the weighted graph. The larger the weight, the stronger the association between nodes, and vice versa.

[0102] S505: Use the unpopular topic document nodes obtained by the dual-channel hyperedge enhancer based on graph diffusion , local candidate node set and global candidate nodes Combine them to get a new enhanced hyperedge set .

[0103] S6, such as Figure 2 As shown, the adjacent hyperedge set generated by S4 is , similar hyperedge sets and the enhanced hyperedge set generated by S5 Remove duplicates and merge to get an enhanced hypergraph .in, Represents the node set and the node set of the original graph is the same, represents a hyperedge set, represents the hyperedge weight matrix.

[0104] In this enhanced hypergraph In the present invention, both the imbalance of node quantity and the imbalance of topology are improved to the maximum extent. , use the hypergraph neural network (a deep learning model for processing hypergraph data) to train an unbiased classifier. The specific process is as follows:

[0105] S601. Enhanced hypergraph adjacency matrix Defined as:

[0106] ;

[0107] in, , , Representation Node Super edge , Representation Node Not a super edge .

[0108] Unlike ordinary graphs, the degree of a hypergraph includes the degree of its nodes and the degree of its edges. Degree is defined as:

[0109] ;

[0110] in, Represents the nodes in the hypergraph adjacency matrix and super edge connectivity, Represents a hyperedge The weight of .

[0111] Super Edge Degree Defined as:

[0112] ;

[0113] in, Represents the nodes in the hypergraph adjacency matrix and super edge A hypergraph convolutional layer is constructed using the following formula:

[0114] ;

[0115] in, represents the enhanced hypergraph of the node degree matrix, represents the hyperedge matrix of the enhanced hypergraph, represents the adjacency matrix of the enhanced hypergraph, Indicates The nodes of a layer represent a hypergraph of matrices, and is the original document feature matrix Indicates the parameters, Represents the activation function.

[0116] S602, we use two layers of hypergraph convolution and a softmax function (normalized exponential function) to generate the final prediction target, and use the cross entropy loss function to update the parameters, that is,

[0117] ;

[0118] in, L represents the loss function, represents the set of scientific literature nodes, c Indicates the subject category to which the document belongs. Indicates the maximum subject category number, Representation Node Tags, represents the indicator function, It means equal to, Indicates that the classifier classifies scientific literature nodes The probability of being classified into any of the topic categories.

[0119] The reason why the hypergraph neural network can discover the high-order correlation between nodes in the graph is that, on the one hand, the hypergraph neural network can use the parameter matrix to extract node features, and on the other hand, based on the node features, the hypergraph neural network will also extract hyperedge features. Unlike ordinary GNNs, ordinary GNNs obtain output node features by aggregating node features, while hypergraph neural networks (HGNN) obtain output node features by aggregating hyperedge features, which is a node-edge-node process.

[0120] S7. After the hypergraph neural network, an unbiased classifier can be trained. The unbiased classifier is no longer a classifier that is biased towards popular subject documents, that is, the classification accuracy of popular subject documents is high, while the classification accuracy of unpopular subject documents is low. The enhanced hypergraph is input into the unbiased classifier to obtain the final classification result, and the scientific and technological documents are classified by subject.

[0121] Figure 1This is the overall architecture flow chart of the present invention. The scientific and technological literature data includes literature content and related information. The literature content includes information such as title and abstract. The related information includes information such as citation and citation. The literature content is encoded by one-hot encoding to obtain a feature matrix, and a label matrix is ​​obtained by manual annotation. The related information is extracted by relationship to obtain an adjacency matrix. A graph data structure is constructed using the feature matrix, the label matrix and the adjacency matrix. The obtained graph data structure is input into the original graph-hyperedge generator and the dual-channel hyperedge enhancer based on graph diffusion. The original graph-hyperedge generator obtains adjacent hyperedges using connectivity and similar hyperedges using similarity. The dual-channel hyperedge enhancer based on graph diffusion obtains enhanced hyperedges based on graph diffusion. Adjacent hyperedges, similar hyperedges and enhanced hyperedges are combined to obtain an enhanced hypergraph. The enhanced hypergraph is trained by a hypergraph neural network to obtain an unbiased classifier, thereby obtaining classification results of Topic 1, Topic 2 and Topic 3, thereby realizing the subject classification of scientific and technological literature.

[0122] Figure 2~Figure 4 The flowchart of the method of the present invention is as follows: first, the original graph structure is passed through (a) the original graph-hyperedge generator to generate adjacent hyperedge sets and similar hyperedge sets; then, it is passed through (b) the dual-channel hyperedge generator to generate enhanced hyperedge sets. Secondly, the generated hyperedge sets are merged and deduplicated to form an enhanced hypergraph. Finally, the hypergraph neural network is used to learn the high-order correlation knowledge of the hypergraph, thereby obtaining the classification results of the nodes.

[0123] Figure 5 The structure of the original graph-hyperedge generator is described in detail. The original graph is passed through two hyperedge identifiers to obtain the adjacent hyperedge set and the similar hyperedge set respectively. Figure 6 The detailed structure of the dual-channel hyperedge generator is shown. The original graph is formed into a weighted graph through graph diffusion, and then the local node set and global candidate nodes are obtained through two recognizers. The global candidate nodes, local node sets and anchor nodes together constitute the enhanced hyperedge set. Figure 2~Figure 6 middle represents a hyperedge set, Represents a scientific literature node, H L Represents the topic classification results, W 1 ~ W 12 Represents the edge weights of a weighted graph.

[0124] The present invention aims at the node topic classification task of the scientific literature citation network, proposes to introduce a hypergraph structure to alleviate the category imbalance problem, and uses hyperedges to directly construct high-order relationships between nodes, which neither generates noise nodes nor destroys the original topological structure. Regardless of whether the nodes have connectivity and distance on the original graph, the hyperedges can directly explain the high-order relationships between nodes.

[0125] The present invention proposes to use graph diffusion to obtain the degree of association between document nodes, convert it into a weighted graph, and give meaning to the original graph nodes. The weighted graph can be used to obtain the relative topological center nodes of unpopular subject categories and the important node set of unpopular subject document nodes, forming a hyperedge enhanced hypergraph, which alleviates the imbalance of node quantity and topological imbalance.

[0126] The present invention can accurately classify scientific and technological literature by subject, which helps to improve research efficiency. Through precise classification, it helps researchers quickly locate relevant literature and avoid wasting time on a large amount of irrelevant information. It optimizes the performance of the paper review system so that reviewers can accurately review papers that are in line with their own research fields, reducing additional expenses such as manual review.

[0127] This paper selects three standard citation network data sets to verify the effectiveness of the method. This paper selects the accuracy rate to evaluate the accuracy of the subject classification of scientific and technological literature, and the calculation method is:

[0128] ;

[0129] in, represents the true positive rate, represents the true negative rate, represents the false positive rate, Indicates the false negative rate.

[0130] The F1 score (F1) is used to evaluate the balance of the accuracy of subject classification of scientific literature under different subject categories. The calculation method is:

[0131] ;

[0132] ;

[0133] ;

[0134] in, represents the true positive rate, represents the true negative rate, Represents the false negative rate, recall represents the recall rate, and pre represents the precision rate.

[0135] The methods compared in this invention include Re-Weight (a method of assigning high loss weights to minority class samples), Over-Sampling (a method of balancing distribution by repeating minority class samples), SMOTE (an oversampling method of generating new samples by mixing the features of minority class nodes and their nearest neighbors), DR-GCN (a method of generating virtual child nodes through GAN and adjusting the features of these virtual nodes to closely align with the features of adjacent nodes), GraphSMOTE (an oversampling method of applying SMOTE to synthesize new nodes and training edge generators to generate topological structures), GraphENS (a method of combining the self-network of minority class nodes with randomly selected nodes to synthesize a new self-network), GraphSHA (an oversampling method involving hard samples) and GraphSR (a method of using two components to predict pseudo labels for unlabeled node sets and obtain a complementary set of labeled node sets to balance the distribution of nodes). The experimental results are shown in Table 1.

[0136] Table 1 Comparative test results of scientific literature subject classification methods based on graph diffusion hyperedge

[0137]

[0138] In the table, Cora represents a citation dataset containing 2708 papers and 5429 citation associations, CiteSeer represents a citation dataset containing 3312 papers and 4715 citation associations, PubMed represents a citation dataset containing 16452 papers and 39308 citation associations, and Ours represents the method proposed in the present invention.

[0139] It can be found from the table that the present invention is basically superior to other baselines in different data sets. The present invention has an average improvement of about 6.19% in accuracy compared to the best performing baseline, and an average improvement of about 3.01% in F1 score compared to the best performing baseline. These results show that the present invention can not only improve the overall classification accuracy of scientific and technological literature topics, but also improve the balance of classification accuracy, which shows that our method has made a certain contribution to improving the accuracy of topic classification of unpopular topic documents with high difficulty in topic classification.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for subject classification of scientific and technological literature based on graph diffusion enhanced hyperedges, characterized in that: The steps include: S1. Collect scientific literature data; S2, extract text features from scientific and technological literature using one-hot encoding to obtain a document feature matrix; S3, constructing a graph data structure with the scientific literature data obtained in S1, in which a node represents a scientific literature, an edge represents the relationship between two scientific literatures that cite or are cited, and using the literature feature matrix obtained in S2 to form the original graph to define a scientific literature citation network; S4, inputting the graph data structure into the original graph-hyperedge generator to obtain a similar hyperedge set to alleviate the imbalance of the number of nodes, specifically: the nodes connected by the edges of the original graph are respectively formed into adjacent hyperedge sets, and the similarity between the nodes is calculated by cosine similarity to obtain a similar hyperedge set; S5. Input the graph data structure into a dual-channel hyperedge enhancer based on graph diffusion to obtain an enhanced hyperedge set to alleviate the imbalance of node categories. Specifically, the nodes of unpopular topic documents are randomly sampled, and global candidate nodes and local candidate node sets are identified to form an enhanced hyperedge set. S6, de-duplicate and merge the adjacent hyperedge set and similar hyperedge set obtained in S4 and the enhanced hyperedge set obtained in S5 to obtain an enhanced hypergraph; S7. Use the enhanced hypergraph and hypergraph neural network to train an unbiased classifier, input the enhanced hypergraph into the unbiased classifier to obtain the final classification result, and thus classify the scientific literature by subject.

2. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedge according to claim 1, characterized in that: Step S5 includes: Graph diffusion convolution is used for modeling and the connectivity characteristics of nodes in the graph are obtained. Graph diffusion convolution is defined by the diffusion matrix: ； in, represents the weight coefficient, T represents the transfer matrix, S represents the diffusion matrix, k Indicates the number of hops. , T The eigenvalue of ; The diffusion matrix is ​​calculated, the unweighted rigid edges of the original graph are converted into weighted elastic edges, and the original graph is transformed into a weighted dense graph. k The weighted dense graph is sparsely constructed by the method and threshold method to obtain the weighted graph of the target.

3. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedge according to claim 2 is characterized in that: Graph diffusion convolution includes PPR version, the transfer matrix of PPR version graph diffusion T and weight coefficient They are: ； ； in, represents the adjacency matrix, represents the degree matrix, represents the transition probability.

4. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedge according to claim 2 is characterized in that: Graph diffusion convolution includes HK version, the transfer matrix of HK version graph diffusion T and weight coefficient They are: ； ； in, represents the adjacency matrix, represents the degree matrix, represents the natural base, represents the diffusion time, Indicates the factorial of the number of hops.

5. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedge according to claim 2, characterized in that: Step S5 also includes: Calculate unpopular subject document nodes Category For all nodes that do not belong to class The sum of the association strengths of the nodes, that is, the sum of the edge weights of the elastic edges between the nodes, and the one with the smallest value is taken as the class Global candidate nodes, unpopular topic literature nodes The global candidate node of the hyperedge with centered Defined as: ； in, Indicates the unpopular subject document node, represents the set of scientific literature nodes, Indicates the maximum subject category number, Indicates the subject category to which the unpopular subject literature belongs. Indicates the subject category number, Indicates the subject category. Represents the unpopular topic document node in the weighted graph and Document Node The weight of the weighted edge between .

6. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedges according to claim 5 is characterized in that: Step S5 also includes: Select the node with the unpopular topic document node as the head node of the elastic edge and rank first by edge weight k The tail node, that is, the node with the highest correlation strength among the unpopular subject documents k Nodes of form a local candidate node set and participate in the enhanced hyperedge set Composition, unpopular subject literature nodes The local candidate node set Defined as: ； in, represents the local candidate node set; Unpopular topic document nodes obtained using dual-channel hyperedge enhancer based on graph diffusion , local candidate node set and global candidate nodes Combine to get a new enhanced hyperedge set .

7. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedge according to claim 1, characterized in that: Step S6 includes: Enhanced hypergraph adjacency matrix Defined as: ； in, , Represents a node set; , represents a hyperedge set; Representation Node Super edge , Representation Node Not a super edge .

8. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedges according to claim 7 is characterized in that: The degree of the enhanced hypergraph includes the degree of the node and the degree of the node is defined as: ； in, Represents the nodes in the hypergraph adjacency matrix and super edge connectivity, Represents a hyperedge The weight of .

9. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedges according to claim 7, characterized in that: The degree of the enhanced hypergraph also includes the degree of the hyperedge. Defined as: ； in, Represents the nodes in the hypergraph adjacency matrix and super edge connectivity, Represents a node set.

10. The method for subject classification of scientific and technological documents based on graph diffusion enhanced hyperedges according to claim 9, characterized in that: Step S6 also includes: Create a hypergraph convolutional layer: ； in, represents the enhanced hypergraph of the node degree matrix, represents the weight of the hyperedge, represents the hyperedge matrix of the enhanced hypergraph, represents the adjacency matrix of the enhanced hypergraph, Indicates The nodes of a layer represent a hypergraph of matrices, Indicates The nodes at the +1 level represent the hypergraph of the matrix, Indicates The document feature matrix of the layer, Indicates The parameters of the layer, represents the activation function; A hypergraph neural network is generated using two layers of hypergraph convolution and a softmax function, and the parameters are updated using the cross entropy loss function: ； in, L represents the loss function, c Indicates the subject category to which the document belongs. Representation Node Tags, represents the indicator function, It means equal to, Indicates that the unbiased classifier classifies scientific literature nodes The probability of being classified into any of the topic categories.

Images