Paper Classification Method and Device Based on Local Enhancement and Bias Removal Contrast

Abstract

The present invention discloses a paper classification method and device based on local enhancement and unbiased contrast, which relates to the field of node classification and includes: constructing a graph dataset based on paper classification training data, and constructing local samples for each node in the graph dataset; using the local samples of each node and an enhancement strategy to obtain a first enhanced view and a second enhanced view and inputting them into a graph encoder and a linear projection layer with shared weights to obtain corresponding node representations; constructing an unbiased contrast loss function according to the corresponding node representations, and training the graph encoder based on the unbiased contrast loss function to obtain a graph encoder after graph contrast learning; obtaining a paper set to be classified and constructing a corresponding graph dataset, and inputting the feature matrix and adjacency matrix in the graph dataset corresponding to the paper set to be classified into the trained paper classification model, and passing through the graph encoder and classifier after graph contrast learning to obtain a classification result. The present invention solves problems such as sampling bias.

Description

Technical Field

[0001] The present invention relates to the field of node classification, and particularly to a paper classification method and device based on local enhancement and unbiased contrast. Background Art

[0002] In recent years, self-supervised learning has attracted great attention from researchers. It eliminates the need for expensive annotated data and achieves performance comparable to supervised methods in many tasks. Contrastive learning is one of the main paradigms of self-supervised learning. It constructs multiple augmented views of the input data through an augmentation strategy, and then maximizes the similarity between positive sample pairs while minimizing the similarity between negative sample pairs, thereby encouraging the model to learn meaningful embedding representations. Benefiting from some pioneering research in the fields of computer vision and natural language processing, contrastive learning has also made substantial progress in the application of graphs.

[0003] The performance of graph contrast models depends to a large extent on the choice of graph augmentation strategies. Although data augmentation is mature enough in other fields, due to the uniqueness of graph data, graph augmentation still needs further exploration. Most existing methods involve deleting existing edges and discarding node features according to a certain ratio. Researchers have also tried to combine expert knowledge or higher-order relationships (communities) to guide graph augmentation. However, the augmentations in these methods do not consider the distribution of nodes. Exploring the distribution of nodes, edges, and features helps generate diverse views without changing semantics as much as possible. However, it is a challenge to learn the node distribution without label information.

[0004] In addition, most of the sampling strategies adopted by existing node-level graph contrast models regard the embeddings of the same node in different views as positive sample pairs, and regard the embeddings of other nodes except the node itself in different views as negative sample pairs. However, this way may cause sampling bias, taking samples with the same node label as negative samples (false negative samples), resulting in nodes with the same label being pushed apart in the representation space, and ultimately affecting the performance of the model in downstream tasks. Summary of the Invention

[0005] The purpose of this application is to propose a paper classification method and device based on local enhancement and unbiased contrast for the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a paper classification method based on local enhancement and unbiased contrast, including the following steps:

[0007] Build a paper classification model and train it to obtain a trained paper classification model. The trained paper classification model includes a graph encoder and a classifier that have undergone graph contrastive learning. The graph encoder that has undergone graph contrastive learning is trained through the following process: Construct a graph dataset based on the paper classification training data; Construct local samples for each node in the graph dataset; Use the local samples of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively, obtaining an enhanced adjacency matrix and an enhanced feature matrix; Perform two samplings on the enhanced adjacency matrix and the enhanced feature matrix respectively to obtain a first enhanced view and a second enhanced view; Input the feature matrix and adjacency matrix of the first enhanced view and the feature matrix and adjacency matrix of the second enhanced view into two graph encoders with shared weights respectively, obtaining the embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view. The embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view are respectively input into two linear projection layers with shared weights, obtaining the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view; Construct a debiased contrastive loss function based on the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view, and train the graph encoder based on the debiased contrastive loss function to obtain a graph encoder that has undergone graph contrastive learning.

[0008] Obtain a paper set to be classified and construct a corresponding graph dataset. Among them, the nodes in the graph dataset represent papers or authors, and the edges between two nodes represent the citation relationship between two papers or the cooperation relationship between two authors. The feature vector of each node in the graph dataset corresponds to the elements of the bag-of-words representation in each paper. Input the feature matrix and adjacency matrix of the graph dataset corresponding to the paper set to be classified into the trained paper classification model, first pass through the graph encoder that has undergone graph contrastive learning to obtain the embedding corresponding to each node, and each node's embedding passes through the classifier to obtain the classification result. The classification result includes the academic theme of the paper or the research field of the author.

[0009] Preferably, the graph encoder adopts a graph convolutional structure, and its expression is as follows:

[0010] ;

[0011] Where, represents the feature matrix, represents the adjacency matrix, is the adjacency matrix with self-loops added, represents the identity matrix, is the degree matrix, represents the th adjacency matrix with self-loops added for the th node, and is a non-linear activation function. is the weight matrix in the graph convolution structure, represents the function corresponding to the graph encoder, represents the embedding of each node output by the graph encoder.

[0012] Preferably, local samples are constructed for each node in the graph dataset, specifically including:

[0013] Obtain the first-order neighbors of each node through the adjacency matrix;

[0014] Input the feature matrix and adjacency matrix of the graph dataset into the graph encoder to obtain the embedding of each node in the graph dataset;

[0015] Input the embedding of each node in the graph dataset into the k-NN algorithm to obtain the k nearest neighbors of each node;

[0016] The first-order neighbors and k nearest neighbors of each node constitute the local sample of each node.

[0017] Preferably, the local samples of each node and the multivariate Bernoulli distribution are used to enhance each node at the structural level and the feature level respectively, to obtain an enhanced adjacency matrix and an enhanced feature matrix, specifically including:

[0018] Construct first modeling data according to the row vector corresponding to each node in the adjacency matrix of each node and its local samples, calculate the parameters of the Bernoulli distribution corresponding to each node according to the first modeling data, and the likelihood function of the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data is shown as the following formula:

[0019] ;

[0020] where, represents the likelihood function of the parameters of the Bernoulli distribution corresponding to the th node, represents the th sample in the th node of the first modeling data, th dimension value, is the number of samples, is the total number of nodes, represents the th dimension parameter of the th node;

[0021] The parameters of the Bernoulli distribution corresponding to the th node calculated from the first modeling data are as follows:

[0022] ;

[0023] Among them, represents the parameters of the Bernoulli distribution corresponding to the -th node calculated from the first modeling data;

[0024] Sample the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data to obtain an enhanced adjacency matrix;

[0025] ;

[0026] Among them, represents the multinomial Bernoulli distribution, represents the -th enhanced adjacency matrix of the node, " " means subject to;

[0027] Construct the second modeling data according to the row vectors corresponding to each node in the feature matrix for each node and its local samples;

[0028] In response to the feature vector of the node being binary data, calculate the parameters of the Bernoulli distribution corresponding to each node according to the second modeling data, and sample the parameters of the Bernoulli distribution corresponding to each node calculated from the second modeling data to obtain an enhanced feature matrix, as shown in the following formula:

[0029] ;

[0030] Among them, represents the parameters of the Bernoulli distribution corresponding to the -th node calculated from the second modeling data, represents the -th enhanced feature matrix of the node;

[0031] In response to the feature vector of the node not being binary data, calculate the parameters of the standard Gaussian distribution corresponding to each node according to the second modeling data, and the likelihood function of the parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data is as shown in the following formula:

[0032] ;

[0033] Among them, represents the likelihood function of the parameters of the standard Gaussian distribution corresponding to the -th node calculated from the second modeling data, represents the second modeling data, represents the value of the j-th sample in the second modeling data, represents the -th mean vector of all sample values in the second modeling data, Represents the covariance matrix of the values of all samples in the th node of the second modeling data, represents the probability density function;

[0034] Sample the parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data to obtain an enhanced feature matrix, as shown in the following formula:

[0035] ;

[0036] where, represents the multivariate standard Gaussian distribution.

[0037] Preferably, the embeddings of each node in the first enhanced view and the embeddings of each node in the second enhanced view are respectively input into two weight-sharing linear projection layers to obtain the node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view, specifically including:

[0038] The node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view are calculated using the following formula:

[0039] ;

[0040] ;

[0041] where, represents the linear projection layer, represents the embedding of each node in the first enhanced view, represents the embedding of each node in the second enhanced view, represents the node representation of each node in the first enhanced view, represents the node representation of each node in the second enhanced view, represents the weight matrix in the linear projection layer.

[0042] Preferably, a debiased contrast loss function is constructed based on the node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view, specifically including:

[0043] Regard the nodes in the local samples of each node as false negative samples and penalize them by assigning weights, as shown in the following formula:

[0044] ;

[0045] where, represents the th local sample corresponding to the th node in the first enhanced view, Node representation of a node or the node representation of the th node in the second enhanced view represents the weight of the th node in the first enhanced view or the second enhanced view;

[0046] Regarding the node representation of the same node in the first enhanced view and the node representation in the second enhanced view as a positive sample pair, and regarding the node representations of other nodes except the node itself in the first enhanced view and the second enhanced view as negative sample pairs;

[0047] The debiased loss function for positive sample pairs is:

[0048] ;

[0049] where represents the node representation of the th node in the first enhanced view, represents the node representation of the th node in the second enhanced view, , , represents the node representation of the th node in the second enhanced view, represents the temperature coefficient, represents the cosine similarity function;

[0050] The debiased loss function for positive sample pairs is:

[0051] ;

[0052] where , , represents the node representation of the th node in the first enhanced view;

[0053] The debiased contrastive loss function[[ID=6'']] is expressed as:

[0054] ;

[0055] where represents the total number of nodes.

[0056] In a second aspect, the present invention provides a paper classification device based on local enhancement and debiased contrast, including:

[0057] The model construction module is configured to construct and train a paper classification model to obtain a trained paper classification model. The trained paper classification model includes a graph encoder and a classifier that have undergone graph contrastive learning. The graph encoder that has undergone graph contrastive learning is trained through the following process: constructing a graph dataset based on paper classification training data; constructing local samples for each node in the graph dataset; using the local samples of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively to obtain an enhanced adjacency matrix and an enhanced feature matrix; performing two samplings on the enhanced adjacency matrix and the enhanced feature matrix respectively to obtain a first enhanced view and a second enhanced view; inputting the feature matrix and adjacency matrix of the first enhanced view and the feature matrix and adjacency matrix of the second enhanced view into two graph encoders with shared weights respectively to obtain the embeddings of each node in the first enhanced view and the embeddings of each node in the second enhanced view. The embeddings of each node in the first enhanced view and the embeddings of each node in the second enhanced view are respectively input into two linear projection layers with shared weights to obtain the node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view; constructing a debiased contrastive loss function based on the node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view, and training the graph encoder based on the debiased contrastive loss function to obtain a graph encoder that has undergone graph contrastive learning.

[0058] The classification module is configured to obtain a set of papers to be classified and construct a corresponding graph dataset. Among them, the nodes in the graph dataset represent papers or authors, and the edges between two nodes represent the citation relationship between two papers or the cooperation relationship between two authors. The feature vector of each node in the graph dataset corresponds to the elements of the bag-of-words representation in each paper. Input the feature matrix and adjacency matrix of the graph dataset corresponding to the set of papers to be classified into the trained paper classification model. First, pass through the graph encoder that has undergone graph contrastive learning to obtain the embeddings corresponding to each node. The embeddings of each node pass through the classifier to obtain the classification results. The classification results include the academic topics of the papers or the research fields of the authors.

[0059] In a third aspect, the present invention provides an electronic device, including one or more processors; a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0060] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0061] Fifth aspect, the present invention provides a computer program product, including a computer program, which when executed by a processor implements the method described in any implementation manner of the first aspect.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] (1) The paper classification method based on local enhancement and unbiased contrast proposed by the present invention constructs local samples for each node in the graph dataset, uses statistical methods to estimate the distribution of local samples, samples from the obtained distribution to generate a first enhanced view and a second enhanced view, sends the first enhanced view and the second enhanced view into a graph encoder with shared weights to learn node representations under different views, generates positive and negative sample pairs, and uses the node used to construct the local sample as a false negative sample of its corresponding node for weighted penalty to construct an unbiased contrast loss function; continuously optimizes the model based on the unbiased contrast loss function to prompt the model to learn more effective representations.

[0064] (2) The paper classification method based on local enhancement and unbiased contrast proposed by the present invention adopts an enhancement strategy composed of structure-level enhancement and feature-level enhancement to provide high-quality views for contrast in the graph contrast learning process, and reduces sampling bias through false negative sample weighting, significantly improving the representation ability of graph contrast learning.

[0065] (3) The paper classification method based on local enhancement and unbiased contrast proposed by the present invention can design a new enhancement strategy using local samples and reduce sampling bias by penalizing false negative samples, improving the accuracy of the node classification task. Description of the Drawings

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for description in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0067] Figure 1 It is a schematic flowchart of the paper classification method based on local enhancement and unbiased contrast in the embodiments of this application;

[0068] Figure 2 It is a schematic framework diagram of the paper classification method based on local enhancement and unbiased contrast in the embodiments of this application;

[0069] Figure 3 It is a schematic diagram of the paper classification device based on local enhancement and unbiased contrast in the embodiments of this application;

[0070] Figure 4Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Detailed implementation manners

[0071] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0072] Figure 1 There is shown a paper classification method based on local enhancement and debiased contrast provided by an embodiment of the present application, including the following steps:

[0073] S1. Construct a paper classification model and train it to obtain a trained paper classification model. The trained paper classification model includes a graph encoder and a classifier that have undergone graph contrast learning. The graph encoder that has undergone graph contrast learning is trained through the following process: Construct a graph dataset based on paper classification training data; construct local samples for each node in the graph dataset; use the local samples of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively to obtain an enhanced adjacency matrix and an enhanced feature matrix; perform two samplings on the enhanced adjacency matrix and the enhanced feature matrix respectively to obtain a first enhanced view and a second enhanced view; input the feature matrix and adjacency matrix of the first enhanced view and the feature matrix and adjacency matrix of the second enhanced view into two graph encoders with shared weights respectively to obtain the embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view. The embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view are respectively input into two linear projection layers with shared weights to obtain the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view; construct a debiased contrast loss function according to the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view, and train the graph encoder based on the debiased contrast loss function to obtain a graph encoder that has undergone graph contrast learning.

[0074] In a specific embodiment, the graph encoder adopts a graph convolutional structure, and its expression is as follows:

[0075] ;

[0076] where represents the feature matrix, represents the adjacency matrix, is the adjacency matrix with self-loops added, represents the identity matrix, is the degree matrix, Denote the adjacency matrix with self-loops for the th node, is a non-linear activation function, is the weight matrix in the graph convolution structure, denotes the function corresponding to the graph encoder, denotes the embedding of each node output by the graph encoder.

[0077] Specifically, the graph encoder used in the embodiments of the present application adopts a graph convolution structure. The input of this graph convolution structure is the feature matrix and the adjacency matrix in the graph dataset, and the output is the embedding of each node in the graph dataset.

[0078] In a specific embodiment, local samples are constructed for each node in the graph dataset, specifically including:

[0079] Obtain the first-order neighbors of each node through the adjacency matrix;

[0080] Input the feature matrix and the adjacency matrix of the graph dataset into the graph encoder to obtain the embedding of each node in the graph dataset;

[0081] Input the embedding of each node in the graph dataset into the k-NN algorithm to obtain the k nearest neighbors of each node;

[0082] The first-order neighbors and the k nearest neighbors of each node constitute the local sample of each node.

[0083] Specifically, obtain the graph dataset. Define a graph dataset as , where denotes the feature matrix, denotes the adjacency matrix, denotes the nodes in the graph dataset. Each node has a feature vector , denotes the dimension of the feature vector, and the feature matrix is composed of the feature vectors of all nodes in the graph dataset. The adjacency matrix indicates whether there is an edge between nodes and . If there is an edge, the corresponding element in the adjacency matrix . During the training process of the graph encoder, the graph dataset is divided into a training set, a validation set, and a test set.

[0084] Construct local samples for each node in the graph dataset according to the original graph data. The local sample of a node consists of its first-order neighbors and k nearest neighbors. The first-order neighbors can be directly obtained from the neighbor matrix, and the k nearest neighbors are obtained by performing the k-nearest neighbor algorithm (k-NN) on the embedding obtained after inputting the feature matrix and the adjacency matrix of the graph dataset.

[0085] In a specific embodiment, the local samples of each node and the multivariate Bernoulli distribution are used to perform structure-level enhancement and feature-level enhancement on each node respectively, obtaining an enhanced adjacency matrix and an enhanced feature matrix, specifically including:

[0086] Construct first modeling data according to the row vector corresponding to each node in the adjacency matrix of each node and its local samples, calculate the parameters of the Bernoulli distribution corresponding to each node according to the first modeling data, and the likelihood function of the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data is shown as follows:

[0087] ;

[0088] Wherein, represents the likelihood function of the parameters of the Bernoulli distribution corresponding to the -th node, represents the -th sample in the -th node of the first modeling data, -th dimension value, is the number of samples, is the total number of nodes, represents the -th dimension parameter of the -th node;

[0089] The parameters of the Bernoulli distribution corresponding to the -th node calculated from the first modeling data are as follows:

[0090] ;

[0091] Wherein, represents the parameters of the Bernoulli distribution corresponding to the -th node calculated from the first modeling data;

[0092] Sample the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data to obtain an enhanced adjacency matrix;

[0093] ;

[0094] Wherein, represents the multivariate Bernoulli distribution, represents the enhanced adjacency matrix of the -th node, and " " means subject to;

[0095] Construct second modeling data according to the row vector corresponding to each node in the feature matrix of each node and its local samples;

[0096] In response to the feature vector of the node being binary data, the parameters of the Bernoulli distribution corresponding to each node are calculated according to the second modeling data, and the parameters of the Bernoulli distribution corresponding to each node calculated from the second modeling data are sampled to obtain an enhanced feature matrix, as shown in the following formula:

[0097] ;

[0098] where, represents the parameters of the Bernoulli distribution corresponding to the -th node calculated from the second modeling data, represents the -th enhanced feature matrix of the node;

[0099] In response to the feature vector of the node not being binary data, the parameters of the standard Gaussian distribution corresponding to each node are calculated according to the second modeling data, and the likelihood function of the parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data is as shown in the following formula:

[0100] ;

[0101] where, represents the likelihood function of the parameters of the standard Gaussian distribution corresponding to the -th node calculated from the second modeling data, represents the second modeling data, represents the value of the j-th sample in the second modeling data, represents the mean vector of all sample values in the -th node of the second modeling data, represents the covariance matrix of all sample values in the -th node of the second modeling data, represents the probability density function;

[0102] The parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data are sampled to obtain an enhanced feature matrix, as shown in the following formula:

[0103] ;

[0104] where, represents the multivariate standard Gaussian distribution.

[0105] Specifically, the performance of the graph encoder learned through graph contrastive learning depends to a large extent on the choice of graph augmentation strategy. Based on the unique status of local samples in the graph dataset, new graph augmentation strategies are designed using local samples. The graph augmentation strategy is as follows: Design augmentation methods at two levels, namely the structure level and the feature level, using local samples. Both the first augmented view ViewU and the second augmented view View V are jointly generated by the augmentation at the structure level and the feature level. The difference between the first augmented view View U and the second augmented view View V lies in that they are obtained by sampling twice respectively based on the augmentation results at the structure level and the feature level. The different results of the two samplings lead to the existence of two augmented views for graph contrastive learning.

[0106] (1) Augmentation at the structure level: The topological structure in the graph dataset is usually represented by an adjacency matrix, which only contains binary data. Therefore, for the augmentation at the structure level, the embodiments of the present application use the multivariate Bernoulli distribution to learn the data distribution.

[0107] Construct the first modeling data based on local samples . The first modeling data is composed of the row vectors corresponding to each node and each node in its corresponding local sample in the adjacency matrix . The dimension of these row vectors is , which is the same as the number of nodes in the graph dataset. From the first modeling data , the parameters of the multivariate Bernoulli distribution can be calculated. The parameters of the multivariate Bernoulli distribution are the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data. After obtaining the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data, directly sample to obtain the augmented adjacency matrix .

[0108] (2) Augmentation at the feature level: Construct the second modeling data based on local samples. The second modeling data is composed of the row vectors corresponding to each node and each node in its corresponding local sample in the feature matrix . The dimension of each row vector is d. For the case where the feature vector of the node is also binary data, the embodiments of the present application also use the multivariate Bernoulli distribution to learn the feature distribution. Similar to the augmentation at the structure level, the augmented feature matrix is calculated. For the case where the feature vector of the node is not binary data, the embodiments of the present application use the multivariate standard Gaussian distribution to learn the feature distribution. Calculate the likelihood function of the parameters of the standard Gaussian distribution corresponding to each node from the second modeling data, solve to obtain the parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data, and finally generate the augmented feature matrix 。

[0109] In a specific embodiment, the embeddings of each node in the first enhanced view and the embeddings of each node in the second enhanced view are respectively input into two weight-sharing linear projection layers to obtain the node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view, specifically including:

[0110] The node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view are calculated using the following formula:

[0111] ;

[0112] ;

[0113] where, represents the linear projection layer, represents the embeddings of each node in the first enhanced view, represents the embeddings of each node in the second enhanced view, represents the node representation of each node, represents the node representation of each node, represents the weight matrix in the linear projection layer.

[0114] Specifically, referring to Figure 2 , the generated first enhanced view View U and View V are fed into two weight-sharing graph encoders to learn the node representations under different views and . By passing and through a weight-sharing linear projection layer respectively, and are obtained respectively. According to and , a debiased loss function is constructed, and finally a debiased contrastive loss function is obtained.

[0115] In a specific embodiment, a debiased contrastive loss function is constructed according to the node representations of each node in the first enhanced view and the node representations of each node in the second enhanced view, specifically including:

[0116] Regarding the nodes in the local samples of each node as false negative samples and punishing them by assigning weights, as shown in the following formula:

[0117] ;

[0118] where, represents the local sample corresponding to the th node, represents the Node representation of nodes or the second enhanced view Node representation of nodes , Indicates the first enhanced view or the second enhanced view The weight of each node;

[0119] The node representation of the same node in the first enhanced view and the node representation of the second enhanced view are regarded as positive sample pairs, while the node representation of other nodes in the first enhanced view and the node representation of the second enhanced view except the node itself are regarded as negative sample pairs;

[0120] The debiasing loss function for the positive sample pair is:

[0121] ;

[0122] in, Indicates the first enhanced view The node representation of the nodes, Indicates the second enhanced view The node representation of the nodes, , , Indicates the first The node representation of the nodes, represents the temperature coefficient, represents the cosine similarity function;

[0123] The debiasing loss function for the positive sample pair is:

[0124] ;

[0125] in, , , Indicates the first enhanced view Node representation of nodes;

[0126] Debiasing contrast loss function Expressed as:

[0127] ;

[0128] in, Indicates the total number of nodes.

[0129] Specifically, the representations of the same node in different views are regarded as positive sample pairs, while the representations of other nodes except the node itself in different views are regarded as negative sample pairs. The prior assumption in the enhancement strategy is that each node belongs to the same distribution as its corresponding local samples. Therefore, the nodes within the local samples of each node are directly regarded as false negative samples and are punished by assigning them a weight of 0. The unbiased loss function for each positive sample pair is calculated based on the weighted penalty. The unbiased loss functions for two positive sample pairs are calculated separately and , and the maximized overall objective is defined as the average of all positive sample pairs. Thus, the unbiased contrastive loss function is constructed. The is optimized by the Adam gradient descent algorithm to update the parameters of the graph encoder, and the graph encoder after graph contrastive learning is obtained. The graph encoder after graph contrastive learning is used for the node classification task.

[0130] In the node classification task in the embodiments of the present application, the academic topic classification of papers or the research field classification of paper authors is taken as an example. In the academic topic classification task of papers or the research field classification task of paper authors, the graph encoder after graph contrastive learning is combined with a classifier to construct a paper classification model, and the paper classification model is trained to obtain a trained paper classification model. In the academic topic classification task of papers or the research field classification task of paper authors, the classifier includes a logistic regression classifier and a softmax function layer. Specifically, a graph dataset is constructed according to the definitions of nodes and edges, the labels corresponding to the nodes are marked, and the paper classification models in different classification tasks are trained to meet the classification requirements of the academic topic classification task of papers or the research field classification task of paper authors.

[0131] S2. Obtain the paper set to be classified and construct the corresponding graph dataset, where the nodes in the graph dataset represent papers or authors; the edges between two nodes represent the citation relationship between two papers or the cooperation relationship between two authors. The feature vector of each node in the graph dataset corresponds to the elements of the bag-of-words representation in each paper. Input the feature matrix and adjacency matrix in the graph dataset corresponding to the paper set to be classified into the trained paper classification model. First, pass through the graph encoder after graph contrastive learning to obtain the embedding corresponding to each node. The embedding of each node passes through the classifier to obtain the classification result, and the classification result includes the academic topic of the paper or the research field of the author.

[0132] Specifically, deploy the trained paper classification model. During the inference process, first use the feature matrix X and adjacency matrix in the graph dataset corresponding to the paper set to be classified as inputs, and obtain the embedding through the graph encoder after graph contrastive learning:

[0133] ;

[0134] Embed is fed into a simple logistic regression classifier. First, it passes through a linear combination layer to obtain node representations:

[0135] ;

[0136] where and b are learnable parameters, and T represents the transpose matrix. Then, through the softmax function layer, the node representations are converted into the predicted probabilities for each class, and the class with the highest probability is selected as the predicted field of the paper to be classified.

[0137] [[ID=2|0]]The technical effects of the present invention are further illustrated by specific embodiments below.

[0138] For a comprehensive comparison, on five open graph datasets, including Cora, CiteSeer, Wiki-CS, Coauthor-CS, and Coauthor-Physics, the node classification performance is evaluated. These graph datasets come from real networks in different fields.

[0139] The method of the present invention is compared with classical unsupervised models including DeepWalk and Node2vec. In addition, the method of the present invention is also compared with excellent self-supervised models such as GAE, VGAE, DGI, MVGRL, GRACE, GCA, CCA-SSG, BGRL, COSTA, and CSGCL. The method of the present invention is also compared with the supervised method GCN.

[0140] In the experimental example, each model is first trained in an unsupervised manner, and then the node representations output by the graph encoder are fed into a simple logistic regression classifier. The public split is used for the Wiki-CS dataset, and the remaining four datasets are randomly divided into 10%, 10%, and 80% for training, validation, and testing, respectively. Since the data partitioning is mostly random, it is run 20 times on each dataset, and the average performance is used as the result.

[0141] Table 1 The number of nodes, number of edges, feature dimension, and number of classes of the dataset.

[0142]

[0143] Table 2 Performance comparison of the node classification task (average precision (%) ± standard deviation).

[0144]

[0145] Table 2 shows the performance of the method of the present invention in the node classification task on five datasets. It can be seen from Table 2 that: compared with all baseline methods, the paper classification model proposed by the present invention has achieved the optimal performance. It is worth noting that on the Cora dataset, compared with the second-best baseline, the paper classification model proposed by the present invention has a significant improvement of nearly 3%. These findings provide convincing evidence that the method of the present invention is an effective framework that can effectively utilize local information.

[0146] Further referring to Figure 3 , as an implementation of the methods shown in the above figures, the present application provides an embodiment of a paper classification device based on local enhancement and unbiased contrast. This device embodiment corresponds to Figure 1 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0147] The embodiment of the present application provides a paper classification device based on local enhancement and unbiased contrast, including:

[0148] A model construction module 1, configured to construct and train a paper classification model to obtain a trained paper classification model. The trained paper classification model includes a graph encoder and a classifier that have undergone graph contrast learning. The graph encoder that has undergone graph contrast learning is trained through the following process: constructing a graph dataset based on paper classification training data; constructing local samples for each node in the graph dataset; using the local samples of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively to obtain an enhanced adjacency matrix and an enhanced feature matrix; performing two samplings on the enhanced adjacency matrix and the enhanced feature matrix respectively to obtain a first enhanced view and a second enhanced view; inputting the feature matrix and adjacency matrix of the first enhanced view and the feature matrix and adjacency matrix of the second enhanced view into two weight-sharing graph encoders respectively to obtain the embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view. The embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view are respectively input into two weight-sharing linear projection layers to obtain the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view; constructing an unbiased contrast loss function according to the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view, and training the graph encoder based on the unbiased contrast loss function to obtain a graph encoder that has undergone graph contrast learning;

[0149] The classification module 2 is configured to obtain a collection of papers to be classified and construct a corresponding graph dataset. Among them, the nodes in the graph dataset represent papers or authors, and the edges between two nodes represent the citation relationship between two papers or the cooperation relationship between two authors. The feature vector of each node in the graph dataset corresponds to the elements represented by the bag of words in each paper. The feature matrix and adjacency matrix in the graph dataset corresponding to the collection of papers to be classified are input into the trained paper classification model. First, it passes through the graph encoder with graph contrast learning to obtain the embedding corresponding to each node. The embedding of each node passes through a classifier to obtain the classification result, and the classification result includes the academic theme of the paper or the research field of the author.

[0150] Figure 4 The following is a schematic hardware structure diagram of the electronic device provided by the embodiment of the present invention. As Figure 4 shown, the electronic device of this embodiment includes: a processor 401 and a memory 402; wherein the memory 402 is used to store computer execution instructions; the processor 401 is used to execute the computer execution instructions stored in the memory to implement each step executed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0151] Optionally, the memory 402 can be either independent or integrated with the processor 401.

[0152] When the memory 402 is independently set, the electronic device further includes a bus 403 for connecting the memory 402 and the processor 401.

[0153] The embodiment of the present invention also provides a computer storage medium, in which computer execution instructions are stored. When the processor 401 executes the computer execution instructions, the above method is implemented.

[0154] The embodiment of the present invention also provides a computer program product, including a computer program. When the computer program is executed by the processor 401, the above method is implemented.

[0155] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be in an electrical, mechanical or other form.

[0156] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place or distributed over multiple network units. Some or all of the modules can be selected according to actual needs to implement the solution of this embodiment.

[0157] In addition, in each embodiment of the present invention, each functional module can be integrated in a processing unit, or each module can exist physically alone, or two or more modules can be integrated in one unit. The units formed by the above modules can be implemented in the form of hardware or in the form of a combination of hardware and software functional units.

[0158] The integrated modules implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above software functional modules are stored in a storage medium, including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 401 to execute some steps of the methods in various embodiments of the present application.

[0159] It should be understood that the above processor 401 can be a central processing unit (Central Processing Unit, abbreviated as CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, abbreviated as DSP), application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), etc. The general-purpose processor can be a microprocessor or the processor 401 can also be any conventional processor 401, etc. The steps of the method disclosed in combination with the invention can be directly implemented by the hardware processor 401, or implemented by a combination of hardware and software modules in the processor 401.

[0160] The memory 402 may include high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk, or an optical disc, etc.

[0161] The bus 403 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 403 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the bus 403 in the drawings of the present application is not limited to only one bus 403 or one type of bus 403.

[0162] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk. The storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0163] An exemplary storage medium is coupled to the processor 401, enabling the processor 401 to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 401. The processor 401 and the storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor 401 and the storage medium can also exist as discrete components in an electronic device or a master control device.

[0164] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes various media that can store program codes, such as ROM, RAM, magnetic disks, or optical disks.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A paper classification method based on local enhancement and debiased contrast, characterized in that, Including the following steps: Construct and train a paper classification model to obtain a trained paper classification model. The trained paper classification model includes a graph encoder and a classifier that have undergone graph contrastive learning. The graph encoder that has undergone graph contrastive learning is trained through the following process: construct a graph dataset based on paper classification training data; Construct local samples for each node in the graph dataset; use the local samples of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively, to obtain an enhanced adjacency matrix and an enhanced feature matrix; Perform two samplings on the enhanced adjacency matrix and the enhanced feature matrix respectively to obtain a first enhanced view and a second enhanced view; Input the feature matrix and adjacency matrix of the first enhanced view and the feature matrix and adjacency matrix of the second enhanced view into two graph encoders with shared weights respectively, to obtain the embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view. The embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view are respectively input into two linear projection layers with shared weights to obtain the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view; Construct a debiased contrastive loss function according to the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view, specifically including: Regard the nodes in the local samples of each node as false negative samples and perform penalty by assigning weights, as shown in the following formula: Among them, S i represents the local sample corresponding to the i-th node, and v t represents the node representation Z of the t-th node in the first augmented view Ut or the node representation Z of the t-th node in the second augmented view Vt , and α t represents the weight of the t-th node in the first augmented view or the second augmented view; Regard the node representation of the same node in the first enhanced view and the node representation of the same node in the second enhanced view as a positive sample pair, and regard the node representations of other nodes except itself in the first enhanced view and the node representations of other nodes except itself in the second enhanced view as negative sample pairs; (Z Ui ,Z Vi ) The debiased loss function for positive sample pairs is as follows: Among them, Z Ui represents the node representation of the i-th node in the first enhanced view, and Z Vi represents the node representation of the i-th node in the second enhanced view, Z Vt represents the node representation of the t-th node in the second enhanced view, τ represents the temperature coefficient, and θ(·, ·) represents the cosine similarity function; (Z Vi ,Z Ui ) The debiased loss function for positive sample pairs is as follows: Among them, Z Ut represents the node representation of the t-th node in the first enhanced view; The debiased contrastive loss function is expressed as: where N represents the total number of nodes. Train the graph encoder based on the debiased contrastive loss function to obtain a graph encoder that has undergone graph contrastive learning; Obtain a paper set to be classified and construct a corresponding graph dataset. Among them, the nodes in the graph dataset represent papers or authors, the edges between two nodes represent the citation relationship between two papers or the cooperation relationship between two authors, the feature vector of each node in the graph dataset corresponds to the elements of the bag-of-words representation in each paper. Input the feature matrix and adjacency matrix of the graph dataset corresponding to the paper set to be classified into the trained paper classification model, first pass through the graph encoder that has undergone graph contrastive learning to obtain the embedding corresponding to each node, and the embedding of each node passes through the classifier to obtain a classification result. The classification result includes the academic theme of the paper or the research field of the author.

2. The paper classification method based on local enhancement and debiased contrast according to claim 1, wherein The graph encoder adopts a graph convolution structure, and its expression is as follows: Among them, X represents the feature matrix, A represents the adjacency matrix, is the adjacency matrix with self-loops added, I represents the identity matrix, is the degree matrix, represents the adjacency matrix with self-loops added for the i-th node, σ(·) is the non-linear activation function, W is the weight matrix in the graph convolution structure, f represents the function corresponding to the graph encoder, and H represents the embedding of each node output by the graph encoder.

3. The paper classification method based on local enhancement and debiased contrast according to claim 1, characterized in that, Construct local samples for each node in the graph dataset, specifically including: Obtain the first-order neighbors of each node through the adjacency matrix; Input the feature matrix and adjacency matrix of the graph dataset into the graph encoder to obtain the embedding of each node in the graph dataset; Input the embedding of each node in the graph dataset into the k-NN algorithm to obtain the k nearest neighbors of each node; The first-order neighbors and the k nearest neighbors of each node constitute the local sample of each node.

4. The method for classifying papers based on local enhancement and debiased contrast according to claim 1, wherein Use the local sample of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively, to obtain an enhanced adjacency matrix and an enhanced feature matrix, specifically including: Construct first modeling data according to the row vector corresponding to each node in the adjacency matrix of each node and its local sample, calculate the parameters of the Bernoulli distribution corresponding to each node according to the first modeling data, and the likelihood function of the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data is shown as follows: Among them, represents the likelihood function of the parameter of the Bernoulli distribution corresponding to the i-th node, b jk＇ represents the value of the k'-th dimension of the j-th sample in the i-th node of the first modeling data, m is the number of samples, N is the total number of nodes, p ik′ represents the parameter of the k'-th dimension of the i-th node; The parameters of the Bernoulli distribution corresponding to the i-th node calculated from the first modeling data are as follows: Among them, represents the parameter of the Bernoulli distribution corresponding to the i-th node calculated from the first modeling data; Sample the parameters of the Bernoulli distribution corresponding to each node calculated from the first modeling data to obtain an enhanced adjacency matrix; where Bern represents the multinomial Bernoulli distribution, represents the enhanced adjacency matrix of the i-th node, and "~" means subject to; Construct second modeling data according to the row vector corresponding to each node in the feature matrix of each node and its local sample; In response to the feature vector of the node being binary data, calculate the parameters of the Bernoulli distribution corresponding to each node according to the second modeling data, sample the parameters of the Bernoulli distribution corresponding to each node calculated from the second modeling data, to obtain an enhanced feature matrix, as shown in the following formula: Among them, represents the parameter of the Bernoulli distribution corresponding to the i-th node calculated from the second modeling data, represents the enhanced feature matrix of the i-th node; In response to the feature vector of the node not being binary data, calculate the parameters of the standard Gaussian distribution corresponding to each node according to the second modeling data, and the likelihood function of the parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data is shown as follows: Among them, represents the likelihood function of the parameters of the standard Gaussian distribution corresponding to the i-th node calculated from the second modeling data, C f represents the second modeling data, c j represents the value of the j-th sample in the second modeling data, μ i represents the mean vector of the values of all samples in the i-th node of the second modeling data, ∑ i represents the covariance matrix of the values of all samples in the i-th node of the second modeling data, and f′ represents the probability density function; Sample the parameters of the standard Gaussian distribution corresponding to each node calculated from the second modeling data to obtain an enhanced feature matrix, as shown in the following formula: Among them, N(·,·) represents the multivariate standard Gaussian distribution.

5. The method for classifying papers based on local enhancement and debiased contrast according to claim 1, characterized in that, The embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view are respectively input into two weight-sharing linear projection layers to obtain the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view, specifically including: Use the following formula to calculate the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view: Z U = MLP(H U , W′); Z V = MLP(H V , W′); Among them, MLP represents a linear projection layer, and H U represents the embedding of each node in the first augmented view, and H V represents the embedding of each node in the second augmented view, Z U represents the node representation of each node in the first augmented view, Z V represents the node representation of each node in the second augmented view, and W′ represents the weight matrix in the linear projection layer.

6. A paper classification device based on local enhancement and debiased contrast, characterized in that, Including: The model construction module is configured to construct and train a paper classification model to obtain a trained paper classification model, and the trained paper classification model includes a graph encoder and a classifier through graph contrast learning; the graph encoder through graph contrast learning is trained through the following process: construct a graph dataset based on paper classification training data; Construct a local sample for each node in the graph dataset; use the local sample of each node and the multivariate Bernoulli distribution to perform structure-level enhancement and feature-level enhancement on each node respectively, to obtain an enhanced adjacency matrix and an enhanced feature matrix; Perform two samplings on the enhanced adjacency matrix and the enhanced feature matrix to obtain the first enhanced view and the second enhanced view respectively; Input the feature matrix and adjacency matrix of the first enhanced view and the feature matrix and adjacency matrix of the second enhanced view into two weight - shared graph encoders respectively, to obtain the embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view. The embedding of each node in the first enhanced view and the embedding of each node in the second enhanced view are respectively input into two weight - shared linear projection layers to obtain the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view; Construct a de - biased contrastive loss function according to the node representation of each node in the first enhanced view and the node representation of each node in the second enhanced view, specifically including: Regard the nodes in the local samples of each node as false negative samples and impose penalties by assigning weights, as shown in the following formula: Among them, S i represents the local sample corresponding to the i-th node, and v t represents the node representation Z of the t-th node in the first augmented view Ut or the node representation Z of the t-th node in the second augmented view Vt , and α t represents the weight of the t-th node in the first augmented view or the second augmented view; Regard the node representations of the same node in the first enhanced view and the second enhanced view as a positive sample pair, and regard the node representations of other nodes except the node itself in the first enhanced view and the second enhanced view as negative sample pairs; (Z Ui ,Z Vi ) The debiased loss function for positive sample pairs is as follows: Among them, Z Ui represents the node representation of the i-th node in the first enhanced view, and Z Vi represents the node representation of the i-th node in the second enhanced view, Z Vt represents the node representation of the t-th node in the second enhanced view, τ represents the temperature coefficient, and θ(·, ·) represents the cosine similarity function; (Z Vi ,Z Ui ) The debiasing loss function for positive sample pairs is as follows: Among them, Z Ut represents the node representation of the t-th node in the first enhanced view; The debiased contrastive loss function is expressed as: Where N represents the total number of nodes. Train the graph encoder based on the de - biased contrastive loss function to obtain a graph encoder after graph contrastive learning; A classification module, configured to obtain a collection of papers to be classified and construct a corresponding graph dataset. Wherein, the nodes in the graph dataset represent papers or authors, and the edges between two nodes represent the citation relationship between two papers or the cooperation relationship between two authors. The feature vector of each node in the graph dataset corresponds to the elements of the bag - of - words representation in each paper. Input the feature matrix and adjacency matrix of the graph dataset corresponding to the collection of papers to be classified into the trained paper classification model. First, pass through the graph encoder after graph contrastive learning to obtain the embedding corresponding to each node. The embedding of each node passes through the classifier to obtain a classification result, and the classification result includes the academic theme of the paper or the research field of the author.

7. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims from 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims from 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims from 1 to 5.

Images