A text attributed graph node classification method based on adaptive knowledge distillation

Abstract

The application discloses a text attribute graph node classification method based on adaptive knowledge distillation. The method first obtains node deep semantic embedding by fine-tuning a language model, and initializes a graph neural network based on the node deep semantic embedding; then, in iterative optimization, a projection layer is used to realize cross-modal alignment of the language model and the graph neural network, and a mild start weight function is introduced to adjust the proportion of the distillation loss; at the same time, based on a class separability index defined by a class center distance, the model reliability is quantitatively evaluated at a node granularity, and accordingly, a teacher model and a student model are dynamically determined and adaptive knowledge distillation is performed. Finally, the output result of the optimized classifier is obtained. The application can be applied to the field of academic network paper node classification. Compared with the method of directly using a large language model, the application reduces the consumption of computing resources and the inference cost, realizes higher classification accuracy and robustness, and effectively enhances the synergistic effect of semantic and structural information.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and deep learning, specifically involving a text attribute graph node classification method based on adaptive knowledge distillation, which can be applied to scenarios such as academic citation network analysis, social network analysis, financial fraud detection, and recommendation systems. Background Technology

[0002] Node classification on text-attributed graphs (TAGs) is a widely applicable task, applicable to scenarios such as academic citation network analysis, social network analysis, financial fraud detection, and recommender systems. Traditional graph neural networks (GNNs) mainly rely on the graph's topology, aggregating neighbor node information through message passing mechanisms. However, traditional GNNs can only process numerical features. When node features contain rich pure textual information, methods based on shallow text embeddings cannot effectively capture deep semantic nuances in the text, affecting the accuracy of node classification.

[0003] With the success of pre-trained language models (LMs), their powerful natural language understanding capabilities have made it possible to accurately capture sentence semantics. Currently, combining graph neural networks (GNNs) with language models has become the mainstream paradigm in this field. Current methods can be broadly categorized into three types: using the language model as a text encoder to acquire text features and then feeding them to the GNN for prediction; using the GNN as a structural encoder to acquire structural features and then feeding them to the language model for prediction; and deep fusion of language models and GNNs. Among these, the deep fusion alignment strategy based on mutual knowledge distillation has attracted considerable attention because it enables dynamic alignment, allowing the language model to benefit from graph structural information and the GNN to benefit from semantic information. This method first trains the language model and the GNN separately, and then performs mutual distillation alignment.

[0004] However, existing mutual distillation alignment methods have a key drawback: in the initial stage of alignment training, the accuracy of the graph neural network often unexpectedly drops, even falling below the results after initial training. This is because, in the early stages of training, the language model has not yet fully adapted to the graph context and performs poorly. Performing mutual alignment at this time will transfer unreliable knowledge from the language model to the graph neural network, causing misleading and interference. Therefore, a more dynamic and fine-grained alignment strategy is urgently needed to resolve this contradiction. Summary of the Invention

[0005] To address the problem of mutual interference between language models and graph neural networks in the early stages of joint training in existing technologies, this invention provides a text attribute graph node classification method based on adaptive knowledge distillation.

[0006] The technical solution adopted in this invention is as follows: A text attribute graph node classification method based on adaptive knowledge distillation includes the following steps: (1) Obtain the text attribute graph, use the nodes with category labels to fine-tune the language model classifier, use the fine-tuned language model classifier to extract the deep semantic embedding vector of each node from the original text attributes in the text attribute graph, and use the deep semantic embedding vector as the initial node feature, combine the edge information of the text attribute graph to train the graph neural network classifier, output the node embedding vector, and obtain the trained graph neural network classifier. (2) The deep semantic embedding vectors of each node are mapped to the feature space of the node embedding vectors using the projection layer of the neural network to obtain the projection embedding; the class separability index of the trained graph neural network is calculated by using the node embedding vectors. The class separability index of the language model is fine-tuned by projecting embedded computational nodes. The class separability index of a graph neural network trained using node representations. The class separability index of the language model after fine-tuning the representation of nodes. By comparison, the fine-tuned language model classifier and the trained graph neural network classifier are selected as the teacher model and student model based on the magnitude of the index. The teacher model and student model are updated by combining the classification loss and the weighted distillation loss to obtain the updated teacher model and the updated student model. Finally, the optimized language model classifier and the optimized graph neural network classifier are obtained. (3) Input the original text attributes to be processed into the optimized language model classifier, extract the deep semantic embedding vector of each node, and use the deep semantic embedding vector as the initial node feature. Combine the edge information of the text attribute graph with the input into the optimized graph neural network classifier, and output the final node classification result.

[0007] In step (1), the text attribute graph includes nodes and edges. Each node has original text attributes, some nodes have category labels, and edges are used to connect two nodes. More preferably, the text attribute graph is an academic network graph, where nodes represent papers, edges represent citation relationships between papers, original text attributes include paper titles and paper abstracts, and category labels are the subject areas to which the papers belong.

[0008] In step (1), the language model classifier includes a cascaded language model encoder and a multilayer perceptron, with the multilayer perceptron serving as the first classification head. The language model encoder outputs deep semantic embedding vectors. The graph neural network classifier includes a cascaded graph neural network encoder and a multilayer perceptron, with the multilayer perceptron serving as the second classification head. The graph neural network encoder outputs node embedding vectors.

[0009] In step (2), the class separability index of the trained graph neural network is calculated by using the node embedding vectors. Specifically, it includes: Calculate the first class center corresponding to each class label based on the node embedding vector of the node with class label; Calculate the current node The Euclidean distance between the node embedding vector and the first class center corresponding to each category label; Use the trained graph neural network classifier to classify the current node Make a prediction and obtain the first prediction category; The current node The node embedding vector and the current node The Euclidean distance between the first class centers corresponding to the first predicted class is defined as the intra-class distance. ; The current node The Euclidean distance between the node embedding vector and the nearest Euclidean center among all first-class centers other than the first predicted class is defined as the inter-class distance. ; According to the formula Calculate the node The class separability index of a trained graph neural network. .

[0010] In step (2), the class separability index of the language model is fine-tuned by projecting and embedding the computational nodes. Specifically, it includes: Calculate the second class center corresponding to each class label based on the projected embedding of the node with class label; Calculate the current node The Euclidean distance between the projected embedding and the second-class center corresponding to each category label; Use the fine-tuned language model classifier to classify the current node Make a prediction to obtain the second prediction category; The current node Projection embedding with the current node The Euclidean distance between the centers of the second class corresponding to the second predicted class is defined as the intra-class distance. ; The current node The Euclidean distance between the projected embedding and the nearest Euclidean center among all second-class centers other than the second predicted class is defined as the inter-class distance. ; According to the formula Calculate the node The category separability index represents the fine-tuned language model. .

[0011] In step (2), the fine-tuned language model classifier and the trained graph neural network classifier are selected as the teacher model and student model respectively, based on the magnitude of the index. Specifically, this includes: Set a reliability threshold ; like Then it is determined that at node The trained graph neural network classifier is the teacher model, and the fine-tuned language model classifier is the student model. like Then it is determined that at node The finely tuned language model serves as the teacher model, and the trained graph neural network classifier serves as the student model. If none of the above conditions are met, the two classifiers are deemed to have comparable performance and enter into a process of mutual distillation.

[0012] Further optimization reveals a text attribute graph node classification method based on adaptive knowledge distillation, comprising the following steps: (1) Initial model construction stage: Obtain the text attribute map to be classified, extract the original text attributes of each node, and fine-tune the preset language model classifier using the nodes with category labels; use the fine-tuned language model classifier to extract the deep semantic embedding vector of each node, and use the deep semantic embedding vector as the initial node feature, and train the preset graph neural network classifier in combination with the structural information of the text attribute map. (2) Iterative optimization stage: The following steps are executed repeatedly until the preset number of iterations is reached: First, cross-modal alignment is performed, and the first node embedding output by the language model classifier and the second node embedding output by the graph neural network classifier under the current iteration are extracted respectively; the first node embedding is mapped to the same feature space as the second node embedding using a preset projection layer to obtain the projection embedding; Second, adaptive knowledge distillation is performed, and based on the projection embedding and the second node embedding, the class separability index of each node relative to the class center under the language model classifier and the graph neural network classifier is calculated respectively, and the prediction reliability of each node under the two models is determined accordingly; according to the prediction reliability, the teacher model and student model of each node are dynamically determined from the language model classifier and the graph neural network classifier, and the model parameters of both are updated by combining the classification loss and the weighted distillation loss; wherein, during the distillation process, the proportion of distillation loss in the total loss is adjusted by the distillation weight function that changes with the current number of iterations. (3) Node classification stage: Input the original text attributes to be processed into the language model encoder after iterative optimization, extract the deep semantic embedding vector of each node, and use the deep semantic embedding vector as the initial node feature. Combine the structural information of the text attribute graph with the input of the graph neural network classifier after iterative optimization, and output the final node classification result.

[0013] Furthermore, the language model classifier includes a language model encoder and a first classification head. In the initial model building phase, the language model classifier is fine-tuned with all parameters by minimizing the first classification loss function; the first classification head The output before performing Softmax normalization is defined as the language model logical value. .

[0014] Furthermore, the graph neural network classifier includes a graph neural network encoder and a second classification head. In the initial model building phase, the graph neural network classifier is trained by minimizing the second classification loss function; the second classification head The output before performing the Softmax normalization operation is defined as the logistic value of the graph neural network. .

[0015] Furthermore, the specific implementation of the cross-modal alignment is as follows: Preset a projection layer And the projection layer is compared with the second classification head in a frozen state. Cascaded construction of mapping classifier; embedding the first node Input the mapping classifier with the language model logical value Training is performed to supervise the target; the projection layer is trained to... Embed the first node of the high dimension Mapped to the second node embedding The same semantic space yields the projected embedding. .

[0016] Furthermore, in the adaptive knowledge distillation stage, the joint loss function employed is... for: in, For classifying losses, The current distillation weight is determined by the distillation weight function. The knowledge distillation loss is used; the distillation weighting function is based on the current iteration number. Linear adjustment of the distillation weight The calculation formula is as follows: in, The preset target weight value, The preset threshold for the number of gentle initial iterations; the knowledge distillation loss The calculation formula is: in, For temperature parameters, The softened probability distribution for the student model. This represents the softening probability distribution for the teacher model.

[0017] Furthermore, the calculation steps for the category separability index include: Calculate the class centers for each category in both the language model projection feature space and the graph neural network feature space. ; Calculate the current node The Euclidean distance between an embedding and the class center of its predicted class is defined as the intra-class distance. ; Calculate the current node The minimum Euclidean distance between the embedding of a class and the class center of any other non-predicted class is defined as the inter-class distance. ; According to the formula Calculate the index value of the node under the corresponding model.

[0018] Furthermore, the logic for dynamically determining the teacher and student models for each node is as follows: setting a reliability threshold. Compare the index values ​​output by the language model classifier and the graph neural network classifier. and : like Then it is determined that at node The neural network classifier in the Chinese library is the teacher model, and the language model classifier is the student model. like Then it is determined that at node The Chinese language model classifier is used as the teacher model, and the graph neural network classifier is used as the student model. If any of the above conditions are not met, the two classifiers are judged to have comparable performance and enter the mutual distillation mode, that is, the language model classifier and the graph neural network classifier serve as each other's teacher models. The guidance signal provided by the teacher model originates from the logical value output by the corresponding model in the previous iteration. or In the first iteration, the guidance signals provided by the teacher model originate from the initial model construction phase.

[0019] The beneficial effects of this invention are: (1) Effectively overcomes the performance degradation in the heterogeneous information alignment stage and improves the stability of model convergence. This invention introduces a mildly initiated distillation weight function to dynamically adjust the proportion of knowledge distillation loss in the initial stage of joint optimization. This design solves the conflict between text semantics and graph topology in the early stage of alignment in academic networks or social networks, prevents the language model from transmitting misleading information to the graph neural network before it has fully adapted to the graph context, and ensures a smooth transition in the modeling of complex attribute graphs.

[0020] (2) Precise and dynamic knowledge transfer at the node level is achieved. Addressing the limitations of existing static mutual distillation methods, this invention proposes a class separability index based on class center distance. This index can evaluate the prediction confidence of the language model and the graph neural network at a specific node in real time, assessing the performance differences of the models at a specific node without additional supervision signals. The node-level distillation controller constructed based on this index can adaptively select the most reliable model as the teacher model for each node, achieving finer-grained and more intelligent knowledge distillation.

[0021] (3) Significantly reduces the system's consumption of computing resources and inference costs. Compared to schemes using large language models, this invention compresses and transfers deep semantic knowledge to lightweight graph neural networks through knowledge distillation technology. In practical applications, such as large-scale paper citation network classification, this invention can significantly reduce the deployment threshold of the model in the production environment and the hardware overhead of real-time inference while maintaining or even exceeding the classification accuracy of large models.

[0022] (4) Significantly improves the accuracy and robustness of node classification. This invention integrates the powerful deep semantic extraction capabilities of pre-trained language models in natural language processing with the advantages of graph neural networks in processing topological structures in a deeper and safer manner. Compared with existing direct fusion or static distillation methods, this invention can achieve higher classification accuracy on text attribute graph data in multiple fields such as academic networks, social networks, and e-commerce, and has extremely strong generalization ability and application value. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.

[0024] Figure 2 This is a model framework diagram of the method of this invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] Example 1 This embodiment provides a specific execution flow for a text attribute graph node classification method based on adaptive knowledge distillation, such as... Figure 1 As shown, the specific steps include: Step 1: Construct a text attribute graph node classification dataset These datasets contain nodes, edge topology, and the raw text attributes of the nodes. The target dataset is divided into training, validation, and test sets. All nodes in the training and test sets need to have category label information.

[0027] Step 2: Build the model infrastructure A language model classifier and a graph neural network classifier are constructed. In this embodiment, a fully parameterized RoBERTa-large (sentence-transformers / all-roberta-large-v1) is used as the pre-trained language model encoder, and a graph convolutional network (GCN) with a feature dimension set to 128 is used as the graph neural network encoder.

[0028] The first classification head is constructed using single fully connected layers. Second category head and projection layer .in, The input dimension is the embedding dimension of the language model, which is 1024, and the output dimension is the number of node categories. The input dimension is 128, and the output dimension is the number of categories; As a cross-modal projection layer, its input dimension is 1024 (the embedding dimension of the language model), and its output size is 128, so as to facilitate subsequent... Perform cascading.

[0029] Step 3: Configure training hyperparameters The Adam optimizer was used for all training operations. For the graph neural network classifier, the weight decay (i.e., L2 regularization penalty) was set to 5e-4, and the learning rate was set to 0.01. For the language model classifier, the learning rate was set to 2e-5, and the batch size was set to 48.

[0030] Step 4: Fine-tune the language model classifier and extract deep semantic embedding vectors. like Figure 2 As shown in step 1, the original text and labels in the training set are used to train the language model encoder and the first classification head. Supervised fine-tuning training is performed. During training, the loss and accuracy on the validation set are continuously observed. An early stopping mechanism is triggered when the validation set loss stops decreasing or the accuracy decreases to avoid overfitting. After fine-tuning, the deep semantic embedding vector of each node is extracted using this language model.

[0031] Step 5: Train the graph neural network classifier like Figure 2 As shown in the lower half of step 1, the deep semantic embedding vectors extracted in step 4 are used as the initial node features input to the graph neural network. Keeping the dataset partition unchanged, the graph neural network encoder and the second classification head are trained using the training set. Similarly, monitor the loss and accuracy on the validation set, and stop training when the validation set performance deteriorates to avoid model overfitting.

[0032] Step 6: Perform cross-modal alignment mapping Entering the iterative optimization phase. Initializing the projection layer. The parameters, and the projection layer With frozen A cascaded mapping classifier is constructed. The first node features output from the fine-tuned language model encoder are then used... Input projection layer Obtain projection embedding Subsequently, the logical values ​​output by the language model during the fine-tuning phase are... To supervise the target, train and optimize the projection layer. The parameters. Due to the projection layer The second node feature output by the graph neural network encoder All can be accessed through a shared category header. By performing the correct normalization classification, it can be determined that the two have been mapped and aligned to the same semantic feature space.

[0033] Step 7: Calculate the class separability index and determine the distillation direction at each node. like Figure 2 As shown in step 2(b), using the real labels in the training set, for the aforementioned alignment space... and Calculate the class center position of each category under its respective feature representation.

[0034] Subsequently, the Euclidean distance (intra-class distance) from each sample node to the center of its predicted class is calculated. ), and the minimum Euclidean distance to the nearest other non-predicted class center (inter-class distance). According to the formula Divide the intra-class distance by the inter-class distance to obtain the class separability index of the node.

[0035] For each specific node, compare its separability index under the two classifiers: if the index value of one classifier is less than the index value of the other classifier multiplied by a set threshold... If the classifier is more reliable for the node, it is designated as the teacher model, and the other is designated as the student model. If the difference in the index values ​​of the two does not meet the above conditions, it is determined that neither has a significant advantage, and the mutual distillation mode is executed.

[0036] Step 8: Calculate the dynamic distillation weight In the current iteration, based on the set gentle start weight function... Calculate the distillation weight for this round. This embodiment will... Set to 5. The weight is set to 0.5. The distillation weight increases linearly or non-linearly with the number of iterations, thereby weakening unreliable distillation guidance in the early stages of iteration and playing a role in smoothing the transition.

[0037] Step 9: Joint Loss Optimization like Figure 2As shown in step 2, based on the teacher-student roles assigned in step 7, the softened probability distribution output by the teacher model in the previous iteration (or initialization phase) is extracted as a guiding signal. Subsequently, a joint loss function consisting of a classification loss including the true labels and a weighted knowledge distillation loss is used to backpropagate and update the model parameters of the language model classifier and the graph neural network classifier, respectively.

[0038] Step 10: Iterate through the loop Repeat steps 6 to 9 above, dynamically re-evaluating the class separability index of the node and updating the model role in each round, until the preset maximum number of iterations is reached (5 rounds in this embodiment).

[0039] Step 11: Output the final node classification prediction result After multiple rounds of iterative alignment, the final node classification test phase begins. The original text features from the graph network to be predicted are input into the language model encoder, which has undergone five rounds of iterative optimization, outputting the final first node embedding. This node embedding is then used as input features to the iteratively optimized graph neural network encoder for forward propagation, incorporating graph structure information. The iteratively optimized second classification head then predicts the probabilities of each class, ultimately outputting the classification result for that node.

[0040] To illustrate the effectiveness and advancement of the model proposed in this invention, this embodiment details the experimental setup and the corresponding results.

[0041] (1) Dataset and graph construction: This embodiment uses seven publicly available real-world benchmark datasets, covering four different application areas: Academic Networks: Includes the Cora, Citeseer, and PubMed datasets, primarily used for node classification research based on paper titles, abstracts, and citation relationships.

[0042] Web link domain: Includes the WikiCS dataset, which represents a graph network based on the content of Wikipedia articles in the field of computer science and their hyperlink relationships.

[0043] In the social networking domain: Includes Instagram and Reddit datasets, reflecting the topology of user social interaction content and post propagation links.

[0044] In the e-commerce field: It includes a Photo dataset, primarily used for item classification tasks based on product description text and purchase co-occurrence relationships.

[0045] (2) Compare to baseline To fully verify the effectiveness of the method of this invention, this embodiment selects several representative mainstream methods as comparative baselines: Fine-tuning language models: Represented by RoBERTa-large, these models abandon graph structures and rely solely on the plain text attributes of nodes for independent fine-tuning and prediction.

[0046] Classic graph neural network models, including GCN, SAGE, and GAT, primarily rely on the message passing mechanism of the graph for prediction.

[0047] Language models as encoder paradigms: represented by TAPE, language models are used to generate deep embeddings of node text, which are then processed by graph neural networks.

[0048] Language models as predictor paradigms: including GraphGPT and LLaGA, which leverage the powerful understanding capabilities of large language models to directly output prediction results for graph nodes.

[0049] A deep fusion paradigm of language models and graph neural networks: including GLEM and ENGINE, aiming to achieve joint training and alignment of multimodal information, and is currently the state-of-the-art baseline method. (3) Experimental setup For the baseline model, this embodiment uses the original hyperparameters provided in the relevant papers. All experiments were conducted on a server equipped with an NVIDIA A100 80GB PCIe GPU, running PyTorch 2.1.2, CUDA 11.8, and Python 3.10. All experiments were run four times with seeds 0, 1, 2, and 3, and the average of the results was taken.

[0050] (4) Evaluation indicators This embodiment uses accuracy as the evaluation metric to measure the proportion of correctly classified samples out of the total number of predicted samples. The calculation formula is as follows: in, This represents the total number of nodes in the test set. For the model to nodes Predicted category labels, For nodes The true category label, This is an indicator function; it takes a value of 1 when the predicted label matches the true label, and a value of 0 otherwise.

[0051] (5) Experimental results This embodiment compares the method of the present invention with the state-of-the-art baseline method described above on the seven datasets.

[0052] Table 1 Performance comparison between the baseline method and the method of this invention As shown in Table 1, the method of this invention achieved the highest average accuracy across all evaluated datasets. This demonstrates that the mild initiation and node-level dynamic distillation mechanism proposed in this invention can effectively overcome early alignment degradation. Compared to methods such as LLaGA and GraphGPT that directly utilize large language models, this invention achieves higher node classification accuracy and robustness while significantly reducing computational resource consumption and inference costs.

[0053] (6) Ablation test To further verify the impact of each core component in this invention on the final performance of the model, this embodiment designed and executed an ablation experiment. Multiple variant versions of the method of this invention were constructed and compared on typical datasets (Cora, Citeseer, WikiCS, Photo).

[0054] This embodiment sets up the following two variant models for comparative study: Variant 1: Remove the mild start strategy and perform distillation directly on the model during the early stages of training to verify the effectiveness of the mild start strategy.

[0055] Variant 2: Remove the node-level dynamic distillation mechanism to verify the effectiveness of node-level dynamic distillation.

[0056] The specific results of the ablation experiment are shown in Table 2.

[0057] Table 2 Ablation Experiment Results Table 2 shows that both variants outperform the full model on the dataset, demonstrating the effectiveness of the mild start-up and node-level dynamic distillation mechanisms. Compared to the variants, the full method shows a significant improvement, proving the synergistic effect between the modules of this invention, achieving superior text attribute graph node classification performance, and effectively enhancing the synergistic effect between semantic and structural information.

[0058] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A text attribute graph node classification method based on adaptive knowledge distillation, characterized in that, Includes the following steps: (1) Obtain the text attribute graph, use the nodes with category labels to fine-tune the language model classifier, use the fine-tuned language model classifier to extract the deep semantic embedding vector of each node from the original text attributes in the text attribute graph, and use the deep semantic embedding vector as the initial node feature, combine the edge information of the text attribute graph to train the graph neural network classifier, output the node embedding vector, and obtain the trained graph neural network classifier. (2) The deep semantic embedding vectors of each node are mapped to the feature space of the node embedding vectors using the projection layer of the neural network to obtain the projection embedding; the class separability index of the trained graph neural network is calculated by using the node embedding vectors. The class separability index of the language model is fine-tuned by projecting embedded computational nodes. The class separability index of a graph neural network trained using node representations. The class separability index of the language model after fine-tuning the representation of nodes. By comparison, the fine-tuned language model classifier and the trained graph neural network classifier are selected as the teacher model and student model based on the magnitude of the index. The teacher model and student model are updated by combining the classification loss and the weighted distillation loss to obtain the updated teacher model and the updated student model. Finally, the optimized language model classifier and the optimized graph neural network classifier are obtained. (3) Input the original text attributes to be processed into the optimized language model classifier, extract the deep semantic embedding vector of each node, and use the deep semantic embedding vector as the initial node feature. Combine the edge information of the text attribute graph with the input into the optimized graph neural network classifier, and output the final node classification result.

2. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 1, characterized in that, In step (1), the text attribute graph includes nodes and edges. Each node has original text attributes, some nodes have category labels, and edges are used to connect two nodes.

3. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 2, characterized in that, In step (1), the text attribute graph is an academic network graph, where nodes represent papers, edges represent citation relationships between papers, the original text attributes include paper titles and paper abstracts, and the category labels are the subject areas to which the papers belong.

4. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 1, characterized in that, In step (1), the language model classifier includes a cascaded language model encoder and a multilayer perceptron, with the multilayer perceptron serving as the first classification head. The language model encoder outputs a deep semantic embedding vector.

5. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 1, characterized in that, In step (1), the graph neural network classifier includes a cascaded graph neural network encoder and a multilayer perceptron, with the multilayer perceptron serving as the second classification head. The graph neural network encoder outputs node embedding vectors.

6. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 1, characterized in that, In step (2), the class separability index of the trained graph neural network is calculated by using the node embedding vectors. Specifically, it includes: Calculate the first class center corresponding to each class label based on the node embedding vector of the node with class label; Calculate the current node The Euclidean distance between the node embedding vector and the first class center corresponding to each category label; Use the trained graph neural network classifier to classify the current node Make a prediction and obtain the first prediction category; The current node The node embedding vector and the current node The Euclidean distance between the first class centers corresponding to the first predicted class is defined as the intra-class distance. ; The current node The Euclidean distance between the node embedding vector and the nearest Euclidean center among all first-class centers other than the first predicted class is defined as the inter-class distance. ; According to the formula Calculate the node The class separability index of a trained graph neural network. .

7. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 1, characterized in that, In step (2), the class separability index of the language model is fine-tuned by projecting and embedding the computational nodes. Specifically, it includes: Calculate the second class center corresponding to each class label based on the projected embedding of the node with class label; Calculate the current node The Euclidean distance between the projected embedding and the second-class center corresponding to each category label; Use the fine-tuned language model classifier to classify the current node Make a prediction to obtain the second prediction category; The current node Projection embedding with the current node The Euclidean distance between the second-class centers corresponding to the second-predicted class is defined as the intra-class distance. ; The current node The Euclidean distance between the projected embedding and the nearest Euclidean center among all second-class centers other than the second predicted class is defined as the inter-class distance. ; According to the formula Calculate the node The category separability index represents the fine-tuned language model. .

8. The text attribute graph node classification method based on adaptive knowledge distillation according to claim 1, characterized in that, In step (2), the fine-tuned language model classifier and the trained graph neural network classifier are selected as the teacher model and student model respectively, based on the magnitude of the index. Specifically, this includes: Set a reliability threshold ; like Then it is determined that at node The trained graph neural network classifier is the teacher model, and the fine-tuned language model classifier is the student model. like Then it is determined that at node The finely tuned language model serves as the teacher model, and the trained graph neural network classifier serves as the student model. If none of the above conditions are met, the two classifiers are deemed to have comparable performance and enter into a process of mutual distillation.

Images