Literature classification method based on heterogeneous graph comparison mask auto-encoder

Abstract

The invention discloses a literature classification method based on a heterogeneous graph comparison mask auto-encoder, and relates to the field of graph self-supervised learning. Firstly, a citation network in a certain field is abstracted into a heterogeneous graph, and feature spaces of different types of nodes such as papers, authors and keywords in the citation network are unified through feature projection; constructing a target node view and a non-target node view based on the node type mask to form a global complementary view; the two views are input into an online encoder and a momentum encoder respectively, the output of the online encoder is used for mask feature reconstruction, and the output of the momentum encoder is used for comparative learning, so that the discrimination and consistency of node representation are enhanced. An encoder and a decoder both adopt a relational graph neural network, and global structure and semantic information can be modeled without sub-graph decomposition. According to the method, the distinguishability of node representation in the citation network can be effectively improved, so that the accuracy of citation network academic literature classification is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of graph self-supervised learning, specifically a document classification method based on heterogeneous graph contrast mask autoencoder. Background Technology

[0002] A citation network is a graph composed of citation relationships between academic documents, where each paper can be considered a node, and citation relationships are represented as edges in the graph. Because academic networks contain multiple types of entities such as papers, authors, institutions, and keywords, as well as different types of relationships, citation networks are essentially heterogeneous information networks (HINs). HINs can simultaneously model multiple types of nodes and diverse relationships, providing an important data foundation for academic network analysis, research topic identification, and literature recommendation.

[0003] Graph neural network methods based on Hidden Intentions (HINs) can learn low-dimensional representations of nodes by aggregating information about different types of nodes and their relationships, thus becoming the mainstream method for citation network classification. However, traditional HIN representation learning methods typically rely on large amounts of labeled data, while labeled data is scarce and costly to obtain in real-world academic networks. To reduce reliance on labeled data, self-supervised HIN representation learning methods have gradually gained attention. Currently, these methods are mainly divided into two categories: one is contrastive learning, which constructs different enhanced views of the same HIN to maximize the correlation between information between views and learns discriminative node representations; the other is masked autoencoders, which learn node representations by randomly masking node or edge features and reconstructing the information in the masked portion.

[0004] However, these methods still have significant shortcomings: contrastive learning-based view enhancement strategies often rely on human experience for design, lacking universality and exhibiting significant performance fluctuations across different academic network scenarios; while masked autoencoders avoid complex view enhancement, they typically depend on subgraph decomposition, which can easily lose information about non-target nodes or ignore potential relationships between cross-type nodes, thus affecting the modeling of global structure and semantics. Therefore, how to propose a generalizable, self-supervised HIN representation learning method that can comprehensively model the structure and semantic information of citation networks while avoiding complex view enhancement and subgraph decomposition has become a key technical problem that urgently needs to be solved, and it has significant theoretical value and application prospects. Summary of the Invention

[0005] This invention addresses the aforementioned problems by providing a document classification method based on a heterogeneous graph contrastive mask autoencoder. This method constructs a complementary global view through simple masking operations, avoiding the information loss problem caused by factor graph decomposition in masking modeling methods and eliminating the need for complex view enhancement strategies. By combining contrastive learning with a mask autoencoder, the model can learn node embeddings that reflect both the local structural features of the graph data and possess strong discriminative power.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] This invention proposes a document classification method based on heterogeneous graph contrast mask autoencoder, comprising the following steps:

[0008] Step 1: Construct the original citation network dataset, introduce different types of nodes such as papers, authors, and keywords, as well as their citation relationships and writing relationships into the network, and generate original features for each type of node;

[0009] Step 2: Preprocess the original citation network dataset, map the features of different types of nodes to the same semantic space, and construct positive and negative sample pairs of paper nodes based on the relationship between nodes;

[0010] Step 3: Perform masking operations on paper nodes and other types of nodes according to preset masking rules to obtain two enhanced views; wherein, the target masked view only masks some paper nodes, and the non-target masked view only masks some nodes of other types of nodes, so as to generate two globally complementary views, thereby achieving the complementarity of feature distribution.

[0011] Step 4: Input the target mask view into the online encoder to obtain the intermediate representation of the online encoder, and input the intermediate representation of the online encoder into the decoder for decoding. Then, back-project the decoding result to finally obtain the reconstruction features for the mask reconstruction task.

[0012] Step 5: Input the intermediate representation of the online encoder in step 4 into the online projector to obtain the contrastive features for contrastive learning; input the non-target mask view into the momentum encoder to obtain the intermediate representation of the momentum encoder, and then input the intermediate representation of the momentum encoder into the momentum projector to obtain the contrastive target features for contrastive learning.

[0013] Step 6: Calculate the node feature reconstruction loss of the mask node set using the reconstructed node features generated in Step 4 and the original node features; calculate the contrast loss using the contrast features and contrast target features output in Step 5; and perform a weighted summation of the two types of losses to obtain the contrast reconstruction loss of the target mask view.

[0014] Step 7: Swap the inputs of the two encoders from Step 4 and Step 5, and then repeat Step 4-6 to obtain the contrast reconstruction loss of the non-target mask view. Sum the contrast reconstruction losses of the two views with weights to obtain the final total loss.

[0015] Step 8: Update the model parameters based on the total loss obtained in Step 7. The parameters of the online encoder, decoder and online projector are updated using backpropagation, while the parameters of the momentum encoder and momentum projector are updated using momentum.

[0016] Step 9: Use the output of the trained online encoder as a node embedding and input it into the classifier to classify the papers in the original citation network dataset constructed in Step 1, and obtain the classified papers.

[0017] Furthermore, step 1 includes the following specific steps:

[0018] Step 1.1: Abstract the citation network dataset of a certain domain into a heterogeneous graph, defined as a 4-tuple. ,in Represents a set of nodes. Denotes the set of edges. Represents a set of node types. Represents a set of edge types; nodes of the same type are numbered using adjacent node numbers, using a global adjacency matrix. Store the connection relationships of all nodes, defined in the following form:

[0019] (1)

[0020] in, The type is The nodes and types are The adjacency submatrix corresponding to the nodes, where p and q take values ​​from 1 to m. Representation diagram The number of node types;

[0021] Step 1.2: Generate node features using bag-of-words vectors and one-hot encoding.

[0022] Furthermore, step 2 includes the following specific steps:

[0023] Step 2.1: Feature mapping is achieved through linear transformation, and the calculation process is as follows:

[0024] (2)

[0025] in, For nodes The original characteristics, For node type mapping functions, and They are respectively The learnable projection matrix and bias vector corresponding to the type node. It is a non-linear activation function. Features of the aligned nodes;

[0026] Step 2.2: Construct positive and negative sample pairs for paper nodes as follows:

[0027] First, for the adjacency matrix Perform the conversion for specific node types. This will include all its associated adjacent submatrices. Transform into an all-zero matrix To obtain the remaining nodes and nodes Unconnected adjacency matrix Calculate the corresponding node types for all nodes except the paper node. This is used to calculate the connection strength matrix between any pair of nodes. The calculation process is as follows:

[0028] (3)

[0029] (4)

[0030] (5)

[0031] in, Indicates any node pair passing through node type The length of the node connection is The number of paths, express The degree matrix, That is The normalized matrix, This indicates the set path length threshold. Represents the connection strength matrix;

[0032] Finally, for each paper node From the matrix The first row selected from the corresponding row The paper node corresponding to the maximum value The other paper nodes serve as positive samples, while the positive samples are negative samples.

[0033] Furthermore, step 3 includes the following specific steps:

[0034] Define the paper node set of the citation network dataset as The remaining node set is defined as follows: The target mask node set is The set of non-target mask nodes is ;right The node features in the image are masked, and the remaining node features are retained to obtain the final target mask view. The calculation process is as follows:

[0035] (6)

[0036] in, Represents a node Features after masking Represents nodes in the target mask view The characteristic of this view is denoted as the target mask view. ;

[0037] Use the same operation to Masking the node features in the image yields a non-target masked view. .

[0038] Furthermore, step 4 includes the following specific steps:

[0039] Step 4.1: Define the online encoder as The middle part represents the calculation process as follows:

[0040] (7)

[0041] in, The node features that represent the target mask view. Indicates online encoder

[0042] Step 4.2: Convert the intermediate representation generated by the online encoder The input is fed into the decoder to obtain the decoded intermediate representation, which is defined as follows:

[0043] (8)

[0044] in, Indicate decoder

[0045] Step 4.3: Convert the intermediate representation generated in step 4.2 into a single file. The back projection calculation process is as follows:

[0046] (9)

[0047] in, Represents a node exist The corresponding feature vector in and Node types The corresponding learnable projection matrix and bias vector, It is a non-linear activation function. For reconstructing features.

[0048] Furthermore, step 5 includes the following specific steps:

[0049] Step 5.1: Convert the intermediate representation generated by the online encoder The input is fed into an online projector to generate contrast features, defined as follows:

[0050] (10)

[0051] in, Indicates online projector, Indicates contrasting features;

[0052] Step 5.2: Input the non-target mask view into the momentum encoder to obtain the intermediate representation. The calculation process is as follows:

[0053] (11)

[0054] in, Node features representing non-target mask views Indicates momentum encoder

[0055] Step 5.3: Convert the intermediate representation generated by the momentum encoder The input is fed into the momentum projector to generate contrast target features, defined as follows:

[0056] (12)

[0057] in, Indicates momentum projector. This indicates the target features to be compared.

[0058] The online encoder and momentum encoder in step 5 have the same architecture and initial parameters, and both adopt a relational graph neural network; the online projector and momentum projector have the same architecture and initial parameters, and both adopt a multilayer perceptron.

[0059] Furthermore, step 6 includes the following specific steps:

[0060] Step 6.1: The mask reconstruction loss adopts the scaled cosine error, and the formula is:

[0061] (13)

[0062] in, For the target mask node set, For nodes The original characteristics, To reconstruct features, Representing vectors Norm, Loss due to mask reconstruction;

[0063] Step 6.2: The formula for the contrast loss is:

[0064] (14)

[0065] (15)

[0066] (16)

[0067] in, The cosine similarity function is used. For temperature parameters, Represents a node With nodes The feature similarity is the softmax normalized value among all nodes, used to measure the node's similarity. For nodes Similarity, As an indicator function, based on the positive and negative sample pairs generated in step 2, when the node... It is a node The value is 1 for positive samples and 0 for all other cases. That is, to represent a node With nodes The true similarity distribution between them To compare the losses;

[0068] Step 6.3: Calculate the contrast reconstruction loss of the target view:

[0069] (17)

[0070] in, This represents the contrast reconstruction loss of the target view. , This is the loss balance coefficient.

[0071] Furthermore, step 7 includes the following specific steps:

[0072] Step 7.1: Input the non-target mask view into the in-circuit encoder, input the target mask view into the momentum encoder, and then... During the calculation process Change to Repeat steps 4-6 to calculate the contrast reconstruction loss of the non-target mask view;

[0073] Step 7.2: Calculate the total loss. The calculation process is as follows:

[0074] (18)

[0075] in, For the contrast reconstruction loss of non-target mask views, This is the loss balance coefficient.

[0076] Furthermore, step 8 includes the following specific steps:

[0077] Step 8.1: The online encoder, decoder, and online projector are updated through backpropagation of the total loss;

[0078] Step 8.2: The momentum encoder and momentum projector update the model parameters using the momentum update method. The update formula is:

[0079] (19)

[0080] (20)

[0081] in, , These are the parameters for the online encoder and the online projector, respectively. , These are the parameters for the momentum encoder and the momentum projector, respectively. This is the momentum coefficient.

[0082] Compared with the prior art, the present invention has the following advantages:

[0083] This invention targets document classification tasks, abstracting and modeling the citation network as a heterogeneous graph, and performing masking operations on different types of nodes. This avoids the complex and experience-dependent view augmentation strategies of traditional methods, enabling the method to flexibly adapt to various types of citation network structures. The method uses a relational graph convolutional network as the encoding framework, allowing direct modeling on the entire graph without decomposing subgraphs, fully preserving the global structure and semantic relationships in the citation network, effectively improving the accuracy of document classification. Furthermore, the constructed global complementary views are complementary in feature distribution: non-target masked views focus on the semantic features of the paper nodes themselves, while target masked views emphasize capturing contextual and related node information relevant to the paper. Through multi-perspective semantic modeling and comparative optimization, this invention improves the accuracy and robustness of document classification at both global and local levels. Attached Figure Description

[0084] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0085] Figure 2 This is a schematic diagram of the overall model training process of the present invention;

[0086] Figure 3 This is a flowchart illustrating the positive and negative sample selection strategy. Detailed Implementation

[0087] To further illustrate the technical solution of the present invention, the present invention will be further described below through embodiments.

[0088] This embodiment uses real data from the ACM database as an example for illustration. Specifically, the dataset contains... , Where “write” is an edge from author to paper, “written” is an edge from paper to author, “belong” is an edge from paper to subject, and “contain” is an edge from subject to paper; in the ACM dataset, there are 4019 paper nodes, 7167 author nodes, and 60 subject nodes.

[0089] like Figure 1 and Figure 2 As shown in this embodiment, a document classification method based on a heterogeneous graph contrast mask autoencoder includes the following steps:

[0090] Step 1: Construct the citation network dataset. This dataset is generated based on real data from the ACM database. Specifically, it extracts real paper, author, and subject association information from the ACM database and generates node features for various types of nodes, thereby constructing the citation network dataset. The generated dataset is hereinafter referred to as the ACM dataset. The specific steps are as follows:

[0091] Step 1.1: In the ACM dataset, the paper node numbers are set to 0~4018, the author node numbers to 4019~11185, and the subject node numbers to 11186~11245. Based on the node numbers, the global adjacency matrix of the ACM dataset is as follows:

[0092] (1)

[0093] in, This represents a matrix of all zeros, indicating that there are no edges between nodes of the corresponding type. , , They represent paper, author, and subject, respectively.

[0094] Step 1.2: In this embodiment, keywords for all papers in the ACM dataset are first extracted, and one-hot encoding is performed on all keywords. The features of paper nodes are generated by performing a bitwise OR operation on the keyword encodings they contain; the features of author nodes are obtained by performing a bitwise OR operation on the keyword encodings of the papers they have written, reflecting the author's research direction and interest distribution; the features of topic nodes are directly generated by one-hot encoding to represent different research fields. After the above processing, the feature dimensions of the generated paper, author, and topic nodes are 1902, 1902, and 60, respectively.

[0095] Step 2 involves preprocessing the ACM dataset, including mapping various node features to the same semantic space to obtain type-aligned node embeddings, and constructing positive and negative sample pairs of nodes from the paper based on the relationships between nodes. This specifically includes the following steps:

[0096] Step 2.1: Perform a linear transformation on the three types of nodes in the ACM dataset. The calculation process is as follows:

[0097] (2)

[0098] In this embodiment, the parameter matrix and bias vector corresponding to each type of node are as follows: , , , , , 256 is the pre-defined aligned feature dimension;

[0099] Step 2.2, see the positive and negative sample construction process. Figure 3 By using the adjacency matrix submatrix in and Set as We can obtain the adjacency matrix without passing through the author node. ,Will and Set as We can obtain the adjacency matrix without passing through the subject node. ,use and The number of paths that pass through nodes of type author and subject can be calculated for any node pair. The calculation process is as follows:

[0100] (3)

[0101] (4)

[0102] (5)

[0103] in, Represents path length. Indicates any node pair passing through node type The length of the node connection is The number of paths, in equation (3) Replace with and You can get and , express The degree matrix, That is The normalized matrix will and Substituting into equation (4) yields the normalized matrix. and , This indicates the set path length threshold. , The connection strength matrix is ​​represented by equation (5); the connection strength between node pairs can be obtained through equation (5).

[0104] Finally, for each paper node From the matrix The first row selected from the corresponding row The paper node corresponding to the maximum value As its positive sample, the remaining paper nodes are negative samples, set .

[0105] Step 3: Perform masking operations on the paper nodes and other types of nodes using a 50% masking ratio to obtain two enhanced views. The target masked view masks only a portion of the paper nodes, while the non-target masked view masks only a portion of the other types of nodes, generating two globally complementary views to achieve complementary feature distributions. The specific steps are as follows:

[0106] Define the set of paper nodes in a citation network as The remaining node set is defined as follows: The target mask node set is The set of non-target mask nodes is ;right The target mask view is obtained by masking the node features and retaining the remaining node features. The calculation process is as follows:

[0107] (6)

[0108] in, Represents a node The masked features, this view is denoted as the target mask view. Use the same operation on Masking the node features in the image yields a non-target masked view. .

[0109] Step 4: Input the target mask view into the online encoder. The encoder's output is the final node embedding used for classification. Therefore, the obtained embedding should reflect both the local features of the citation network and possess a certain degree of discriminability. To reflect the local features of the citation network, the encoder's output is input into the decoder for decoding. The decoded output is compared with the original features to measure the effectiveness of the learned node embedding in reflecting local network features. The specific steps are as follows:

[0110] Step 4.1, define the online encoder as The middle part represents the calculation process as follows:

[0111] (7)

[0112] Where A represents the adjacency matrix of the graph. Node features representing the target mask view;

[0113] Step 4.2, the intermediate representation generated by the online encoder The input is fed into the decoder to obtain the decoded features, which are defined as follows:

[0114] (8)

[0115] in, Indicates decoder;

[0116] Step 4.3, use the intermediate representation generated in step 4.2. The back projection calculation process is as follows:

[0117] (9)

[0118] in, Represents a node exist The corresponding feature vector in and Node types The corresponding learnable projection matrix and bias vector, It is a non-linear activation function. These are the reconstructed features.

[0119] Step 5: To make the learned node embeddings discriminative, the output of the online encoder is compared with the output of the momentum encoder; the intermediate representation obtained by the online encoder is input into the online projector to obtain contrastive features for contrastive learning; the non-target mask view is input into the momentum encoder to obtain an intermediate representation, and then the intermediate representation is input into the momentum projector to obtain contrastive target features for contrastive learning; the online encoder and the momentum encoder have the same architecture and initial parameters, both using a two-layer relational graph convolutional network (RGCN); the online projector and the momentum projector have the same architecture and initial parameters, both using a multilayer perceptron composed of two layers of linear transformation; the specific process includes the following steps:

[0120] Step 5.1: The intermediate representation generated by the online encoder... The input is fed into an online projector to generate contrast features, defined as follows:

[0121] (10)

[0122] in, Indicates online projector, Indicates contrasting features;

[0123] Step 5.2: Input the non-target mask view into the momentum encoder to obtain the intermediate representation. The calculation process is as follows:

[0124] (11)

[0125] in, Node features representing non-target mask views This indicates a momentum encoder. This represents the output of the momentum encoder;

[0126] Step 5.3: Convert the intermediate representation generated by the momentum encoder. The input is fed into the momentum projector to generate contrast target features, defined as follows:

[0127] (12)

[0128] in, Indicates momentum projector. This indicates the target features to be compared.

[0129] Step 6: Calculate the feature reconstruction loss using the reconstructed features generated in Step 4 and the original features; calculate the contrast loss using the contrastive features and contrast target features generated in Step 5; perform a weighted sum of the two types of losses to obtain the contrast reconstruction loss of the target mask view; the specific process includes the following steps:

[0130] Step 6.1, the mask reconstruction loss adopts the scaled cosine error, and the formula is:

[0131] (13)

[0132] in, This is the set of mask nodes for the current input online encoder view. For nodes The original characteristics, Representing vectors Norm, Loss due to mask reconstruction;

[0133] Step 6.2, the contrast loss formula is:

[0134] (14)

[0135] (15)

[0136] (16)

[0137] in, The cosine similarity function is used. For temperature parameters, Represents a node With nodes The feature similarity is the softmax normalized value among all nodes, used to measure the node's similarity. For nodes Similarity, As an indicator function, based on the positive and negative sample pairs generated in step 2, when the node... It is a node The value is 1 for positive samples and 0 for all other cases. That is, to represent a node With nodes The true similarity distribution between them To compare the losses;

[0138] Step 6.3, calculate the contrast reconstruction loss of the target view:

[0139] (17)

[0140] in, This represents the contrast reconstruction loss of the target view. , Set the loss balance coefficient. , .

[0141] Step 7: Swap the inputs of the two encoders from steps 4-6, and then repeat steps 4-6 to obtain the contrastive reconstruction loss of the non-target mask view. Weighted sum the contrastive reconstruction losses of the two views to obtain the final total loss. The specific process includes the following steps:

[0142] Step 7.1: Input the node features of the non-target mask view into the online encoder, input the node features of the target mask view into the momentum encoder, and input the feature reconstruction loss. In Change to Calculate the reconstruction loss for the other types of nodes; repeat steps 4-6 to calculate the contrast reconstruction loss for the non-target mask view;

[0143] Step 7.2, calculate the total loss. The calculation process is as follows:

[0144] (18)

[0145] in, The mask reconstruction loss is for non-target mask views. This is the loss balance coefficient.

[0146] Step 8: Update the model parameters based on the loss from Step 6. The parameters of the online branch are updated using backpropagation, while the parameters of the momentum branch are updated using momentum and adjusted according to the parameters of the online branch. The specific process includes the following steps:

[0147] Step 8.1: The online encoder, decoder, and online projector are updated through backpropagation of the total loss;

[0148] Step 8.2: The momentum encoder and momentum projector update the model parameters using the momentum update method. The update formula is:

[0149] (19)

[0150] (20)

[0151] in, , These are the parameters for the online encoder and the online projector, respectively. , These are the parameters for the momentum encoder and the momentum projector, respectively. The momentum coefficient is set to 0.998.

[0152] Step 9: Use the output of the trained online encoder as the final node embedding of the paper nodes, and use it as the input of the classifier to classify the papers in the original citation network dataset constructed in Step 1, thus obtaining the classified papers. In this embodiment, paper classification is achieved using a linear probing method: specifically, all parameters of the online encoder are frozen, and the node embeddings output by the online encoder are directly used as the input features for the node classification task; the classifier adopts a simple linear transformation structure.

[0153] The dataset is divided as follows: from all paper nodes, 60 paper nodes of each class are selected to form the training set, 1000 paper nodes are selected to form the validation set, and the remaining paper nodes are used as the test set. The classification task uses the cross-entropy loss function as the optimization objective and adopts the gradient descent method to iteratively update the classifier parameters. After 100 rounds of training, a stable paper classification result can be output.

[0154] Technical effect evaluation:

[0155] To verify the effectiveness and advancement of the proposed technical solution, this invention will be compared with existing self-supervised heterogeneous graph neural network models RMR, HGMAE, GraphMAE, DGI, and HeCo on a paper classification task. The average Micro-f1 and Macro-f1 of 20 node classification experiments on the ACM dataset will be used as evaluation metrics. The comparison results are shown in Table 1.

[0156] Table 1 Comparison Results

[0157] Method indicators RMR HGMAE GraphMAE DGI HeCo This invention Micro-f1 87.90 91.34 80.77 80.15 88.71 92.64 Macro-f1 87.96 91.59 80.29 80.03 89.04 92.69

[0158] As can be seen from the results in Table 1, the technical solution of the present invention can achieve better classification accuracy when performing node classification tasks.

[0159] The foregoing has shown and described the main features and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0160] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A document classification method based on heterogeneous graph contrast mask autoencoder, characterized in that, Includes the following steps: Step 1: Construct the original citation network dataset, introduce different types of nodes such as papers, authors, and keywords, as well as their citation and writing relationships into the network, and generate original features for each type of node; Step 2: Preprocess the original citation network dataset, map the features of different types of nodes to the same semantic space, and construct positive and negative sample pairs of paper nodes based on the relationship between nodes; Step 3: Perform masking operations on paper nodes and other types of nodes according to preset masking rules to obtain two enhanced views; wherein, the target masked view only masks some paper nodes, and the non-target masked view only masks some nodes of other types of nodes, so as to generate two globally complementary views, thereby achieving the complementarity of feature distribution. Step 4: Input the target mask view into the online encoder to obtain the intermediate representation of the online encoder, and input the intermediate representation of the online encoder into the decoder for decoding. Then, back-project the decoding result to finally obtain the reconstruction node features for the mask reconstruction task. Step 5: Input the intermediate representation output by the online encoder in step 4 into the online projector to obtain the contrastive features for contrastive learning; input the non-target mask view into the momentum encoder to obtain the intermediate representation of the momentum encoder, and then input the intermediate representation of the momentum encoder into the momentum projector to obtain the contrastive target features for contrastive learning. Step 6: Calculate the node feature reconstruction loss of the mask node set using the reconstructed node features generated in Step 4 and the original node features; calculate the contrast loss using the contrast features and contrast target features output in Step 5; and perform a weighted summation of the two types of losses to obtain the contrast reconstruction loss of the target mask view. Step 7: Swap the inputs of the two encoders from Step 4 and Step 5, and then repeat Step 4-6 to obtain the contrast reconstruction loss of the non-target mask view. Sum the contrast reconstruction losses of the two views with weights to obtain the final total loss. Step 8: Update the model parameters based on the total loss obtained in Step 7. The parameters of the online encoder, decoder and online projector are updated using backpropagation, while the parameters of the momentum encoder and momentum projector are updated using momentum. Step 9: Use the output of the trained online encoder as a node embedding and input it into the classifier to classify the papers in the original citation network dataset constructed in Step 1, and obtain the classified papers.

2. The document classification method based on heterogeneous graph contrast mask autoencoder according to claim 1, characterized in that, Step 1 includes the following specific steps: Step 1.1: Abstract the citation network dataset into a heterogeneous graph, defined as a 4-tuple. ,in Represents a set of nodes. Denotes the set of edges. Represents a set of node types. Represents a set of edge types; nodes of the same type are numbered using adjacent node numbers, using a global adjacency matrix. Store the connection relationships of all nodes, defined in the following form: (1) in, Represented as type The nodes and types are The adjacency submatrix corresponding to the nodes, where p and q take values ​​from 1 to m. Representation diagram The number of node types; Step 1.2: Generate node features using methods such as bag-of-words vectors and one-hot encoding.

3. The document classification method based on heterogeneous graph contrast mask autoencoder according to claim 2, characterized in that, Step 2 includes the following specific steps: Step 2.1: The feature mapping method is implemented through linear transformation, and the calculation process is as follows: (2) in, For nodes The original characteristics, For node type mapping functions, and They are respectively The learnable projection matrix and bias vector corresponding to the type node. It is a non-linear activation function. Features of the aligned nodes; Step 2.2: Construct positive and negative sample pairs for nodes: First, for the adjacency matrix Perform the conversion for specific node types. This will include all its associated adjacent submatrices. Transform into an all-zero matrix To obtain the remaining nodes and node types Unconnected adjacency matrix Calculate the corresponding node types for all nodes except the paper node. This is used to calculate the connection strength matrix between any pair of nodes. The calculation process is as follows: (3) (4) (5) in, Indicates any node pair passing through node type The length of the node connection is The number of paths, express The degree matrix, That is The normalized matrix, This indicates the set path length threshold. Represents the connection strength matrix; Finally, for each paper node From the matrix The first row selected from the corresponding row The paper node corresponding to the maximum value The other paper nodes serve as positive samples, while the positive samples are negative samples.

4. The document classification method based on heterogeneous graph contrast mask autoencoder according to claim 3, characterized in that, Step 3 includes the following specific steps: Define the paper node set of the citation network dataset as The remaining node set is defined as follows: The target mask node set is The set of non-target mask nodes is ;right The node features in the image are masked, and the remaining node features are retained to obtain the final target mask view. The calculation process is as follows: (6) in, Represents a node Features after masking Represents nodes in the target mask view The characteristic of this view is denoted as the target mask view. ; Use the same operation to Masking the node features in the image yields a non-target masked view. .

5. The document classification method based on heterogeneous graph contrast mask autoencoder according to claim 4, characterized in that, Step 4 includes the following specific steps: Step 4.1: Define the online encoder as The middle part represents the calculation process as follows: (7) in, The node features that represent the target mask view. Indicates an online encoder; Step 4.2: Convert the intermediate representation generated by the online encoder The input is fed into the decoder to obtain the decoded intermediate representation, which is defined as follows: (8) in, Indicates decoder; Step 4.3: Convert the intermediate representation generated in step 4.2 into a single file. The back projection calculation process is as follows: (9) in, Represents a node exist The corresponding feature vector in and Node types The corresponding learnable projection matrix and bias vector, It is a non-linear activation function. For reconstructing features.

6. A document classification method based on a heterogeneous graph contrast mask autoencoder according to claim 5, characterized in that, Step 5 includes the following specific steps: Step 5.1: Convert the intermediate representation generated by the online encoder The input is fed into an online projector to generate contrast features, defined as follows: (10) in, Indicates online projector, Indicates contrasting features; Step 5.2: Input the non-target mask view into the momentum encoder to obtain the intermediate representation. The calculation process is as follows: (11) in, Node features representing non-target mask views Indicates a momentum encoder; Step 5.3: Convert the intermediate representation generated by the momentum encoder The input is fed into the momentum projector to generate contrast target features, defined as follows: (12) in, Indicates momentum projector. This indicates the target features to be compared.

7. A document classification method based on a heterogeneous graph contrast mask autoencoder according to claim 6, characterized in that, The online encoder and momentum encoder have the same architecture and initial parameters, and both employ a relational graph neural network; the online projector and momentum projector have the same architecture and initial parameters, and both employ a multilayer perceptron.

8. A document classification method based on a heterogeneous graph contrast mask autoencoder according to claim 7, characterized in that, Step 6 includes the following specific steps: Step 6.1: The mask reconstruction loss adopts the scaled cosine error, and the formula is: (13) in, For the target mask node set, For nodes The original characteristics, For nodes The reconstruction features, Representing vectors Norm, Loss due to mask reconstruction; Step 6.2: The formula for the contrast loss is: (14) (15) (16) in, The cosine similarity function is used. For temperature parameters, Represents a node With nodes The feature similarity is the softmax normalized value among all nodes, used to measure the node's similarity. For nodes Similarity, As an indicator function, based on the positive and negative sample pairs generated in step 2, when the node... It is a node The value is 1 for positive samples and 0 for all other cases. That is, to represent a node With nodes The true similarity distribution between them To compare the losses; Step 6.3: Calculate the contrast reconstruction loss of the target view: (17) in, This represents the contrast reconstruction loss of the target view. , This is the loss balance coefficient.

9. A document classification method based on a heterogeneous graph contrast mask autoencoder according to claim 8, characterized in that, Step 7 includes the following steps: Step 7.1: Input the node features of the non-target mask view into the online encoder, and input the node features of the target mask view into the momentum encoder, and then... During the calculation process Change to Repeat steps 4-6 to calculate the contrast reconstruction loss of the non-target mask view; Step 7.2: Calculate the total loss. The calculation process is as follows: (18) in, For the contrast reconstruction loss of non-target mask views, This is the loss balance coefficient.

10. A document classification method based on a heterogeneous graph contrast mask autoencoder according to claim 9, characterized in that, Step 8 includes the following specific steps: Step 8.1: The online encoder, decoder, and online projector are updated through backpropagation of the total loss; Step 8.2: The momentum encoder and momentum projector update the model parameters using the momentum update method. The update formula is: (19) (20) in, , These are the parameters for the online encoder and the online projector, respectively. , These are the parameters for the momentum encoder and the momentum projector, respectively. This is the momentum coefficient.

Images