Text graph node classification method based on large language model dual-view semantic fusion

Abstract

This invention discloses a text graph node classification method based on dual-view semantic fusion using a large language model, belonging to the interdisciplinary field of graph neural networks and natural language processing. It includes: a dual-view contrastive learning module, which constructs independent views of the original text of the nodes and the interpretations generated by the large language model, respectively, and obtains the node embeddings of the two views through supervised contrastive learning using independent pre-trained language encoders and graph neural network encoders; a pseudo-label edge perturbation module, which uses pseudo-labels generated by the large language model to evaluate the semantic consistency between nodes, dynamically adjusts the edge deletion probability based on the consistency score, and generates a semantically aware enhanced graph; and a hierarchical bidirectional cross-attention fusion module, which gradually fuses the semantic information and graph structure information of the two views through alternating iterations of multi-layer bidirectional cross-attention mechanisms and graph convolution operations to obtain the final node representation for classification. This invention significantly improves the accuracy and robustness of text attribute graph node classification.

Description

Technical Field

[0001] This invention relates to a text graph node classification method based on dual-perspective semantic fusion of a large language model. It is a text attribute graph node classification method based on dual-perspective semantic hierarchical fusion guided by a large language model. It relates to artificial intelligence technology, especially to the fields of graph neural networks, node classification, and natural language processing. Background Technology

[0002] With the rapid development of internet technology, graph data is ubiquitous in our lives, appearing in social networks, reference networks, e-commerce platforms, and more. Graph Neural Networks (GNNs), as deep learning models capable of processing graph-structured data, have achieved remarkable results in tasks such as node classification and graph classification. However, traditional GNN methods primarily focus on the structural information of the graph, neglecting the textual attribute information of the nodes themselves. Text-Attributed Graphs (TAGs), which combine textual information with graph structure, can capture both semantic and structural information simultaneously, and therefore have received widespread attention in node classification tasks.

[0003] In recent years, the rapid development of Large Language Models (LLMs) has brought new opportunities for node classification in text attribute graphs. LLMs possess powerful semantic understanding capabilities, enabling the generation of high-quality text representations and interpretations. Combining LLMs with Generative Neural Networks (GNNs) has become a research hotspot in this field.

[0004] The following are technical solutions that are similar to the present invention:

[0005] Option 1: GLEM method;

[0006] Technical Solution: GLEM (Learning on large-scale text-attributed graphs via variational inference) proposes a joint training framework for GNN and language models based on variational inference. This method alternately updates the language model and GNN using the variational GLEM algorithm, enabling them to jointly learn text semantics and graph structure information.

[0007] Specific steps:

[0008] A variational autoencoder framework is adopted to jointly model PLM and GNN;

[0009] The EM algorithm is used for alternating optimization: the pseudo-labels are updated in the E-step, and the model parameters are updated in the M-step.

[0010] Training is performed on a large-scale text attribute graph to achieve effective interaction between the two.

[0011] Problems: Although GLEM achieves joint training of PLM and GNN, it fails to fully utilize the explanatory information generated by LLM and lacks a mechanism for processing LLM noise.

[0012] Option 2: Tape method;

[0013] Technical solution: TAPE (Harnessing explanations: LLM-to-LM interpreter for enhanced text-attributed graph representation learning) is the first to propose using text interpretations generated by LLM to enhance node feature representations.

[0014] Specific steps:

[0015] Use LLM to generate a text explanation for each node;

[0016] The original node text is concatenated with the interpretation generated by LLM;

[0017] The concatenated features are input into the GNN for classification.

[0018] Problems: TAPE directly incorporates LLM interpretation as an additional feature into the model, which presents the following issues:

[0019] The lack of assessment of the quality of LLM interpretation may introduce noise;

[0020] Failed to distinguish the different roles of the original text and the LLM interpretation;

[0021] Over-reliance on the accuracy of LLM output;

[0022] Option 3: A Social Network Node Classification Method Based on Graph Neural Networks (Application Publication No.: CN120105163A)

[0023] Technical solution: This patent proposes a social network node classification method based on GNN, which mainly focuses on the fusion of structural information and node features in the social network.

[0024] Specific steps:

[0025] Extracting topological features of social networks;

[0026] It integrates multimodal information such as user profiles and behavioral characteristics of nodes;

[0027] Node classification using graph convolutional networks;

[0028] Problems: This method is mainly designed for traditional social network data, fails to make full use of the rich text information attached to nodes, and does not involve the application of LLM technology.

[0029] Option 4: A method and apparatus for interpreting graph neural networks based on FPGA acceleration. Application publication number: CN117688972A

[0030] Technical solution: This patent proposes a method for accelerating GNN inference using FPGA hardware and interprets the output results of the GNN.

[0031] Specific steps:

[0032] Design a dedicated FPGA hardware architecture to accelerate GNN computation;

[0033] Visualize and interpret the prediction results of GNN;

[0034] Optimize model inference efficiency;

[0035] Problem: This patent mainly focuses on optimizing computational efficiency and does not address how to use LLM to improve the classification performance of GNN on text attribute graphs.

[0036] 3. The shortcomings of the above technical solution are:

[0037] Based on the above existing technical solutions, the current text attribute graph node classification technology has the following problems:

[0038] 1. Failure to effectively utilize LLM semantic information: Existing methods either treat LLM only as a feature enhancement tool or lack correction for LLM output;

[0039] 2. Lack of semantic-structural co-learning mechanism: Existing methods have failed to establish an effective connection between semantic information and graph structural information.

[0040] 3. Insufficient handling of LLM noise: Existing methods directly use LLM output, ignoring the potential for erroneous predictions caused by LLM. Summary of the Invention

[0041] This invention aims to solve the following technical problems:

[0042] (1) In terms of effectively utilizing the semantic information generated by LLM while avoiding its noise interference, existing methods directly merge the LLM-generated interpretation with the original text, causing LLM errors to directly affect downstream tasks. A mechanism is needed to separate the learning spaces of the original text and the LLM interpretation, so that the model can selectively utilize the LLM signal.

[0043] (2) When introducing semantic awareness technology into the graph augmentation process, traditional graph contrastive learning uses random edge perturbation, which lacks semantic awareness and may delete important edges or retain noisy edges. It is necessary to design a semantic-based edge perturbation strategy to prioritize the deletion of semantically inconsistent edges.

[0044] (3) In terms of integrating complementary semantic information from different perspectives, the original text and the LLM interpretation contain complementary semantic information, and simple splicing or averaging cannot capture the complex interaction between them. A sophisticated fusion mechanism needs to be designed to achieve deep integration of multi-perspective information.

[0045] (4) In terms of maintaining performance while ensuring the scalability of the method, LLM inference is costly and requires the design of an efficient framework to avoid performing LLM queries on every edge while maintaining excellent classification performance.

[0046] To achieve the above objectives, the technical solution adopted by this invention is: a text graph node classification method based on dual-view semantic fusion of a large language model, comprising the following steps:

[0047] Step 1: Dual-perspective comparative learning;

[0048] Each node in the text attribute graph is constructed as a first view based on the original text and a second view based on the explanation and pseudo-label generated by the large language model. The first view and the second view are encoded by independent first pre-trained language encoders and second pre-trained language encoders to obtain the first view node embedding and the second view node embedding. The first view node embedding and the second view node embedding are subjected to supervised comparative learning by independent first graph neural network encoders and second graph neural network encoders to optimize the embedding space of the two views.

[0049] Step 2: Perturb the edge of the pseudo-label;

[0050] The pseudo-labels generated by the large language model are used to evaluate the semantic consistency between connected nodes. The deletion probability of each edge in the graph is dynamically adjusted according to the consistency score to generate a semantically aware first augmented graph and a second augmented graph, which are used for comparative learning of the first view and the second view, respectively. Among them, the edges connecting semantically consistent nodes are preferentially retained, and the edges connecting semantically inconsistent nodes are deleted with a higher probability.

[0051] Step 3: Hierarchical bidirectional cross-attention fusion;

[0052] The embeddings of the first and second view nodes are input into a multi-layered hierarchical bidirectional cross-attention network. Each layer performs the following operations in sequence: bidirectional cross-attention operation, which enables the first and second views to pay attention to each other and generate a fused representation; graph convolution operation, which updates the embeddings of the two views respectively to integrate graph structure information; the fused representations generated by each layer are aggregated to obtain the final node representation; and the final node representation is input into a classifier to predict the node category.

[0053] Further, in step one, constructing structured prompts for each node to query the large language model specifically includes: calculating the cosine similarity of the text embeddings between nodes using a pre-trained language model, where the cosine similarity is obtained by calculating the dot product of the embedding vectors of two nodes and dividing it by the product of the magnitudes of the two vectors; selecting the K nodes with the highest similarity as the semantic neighbors of the node, where K ranges from 3 to 10; concatenating the original text of the target node, the texts of the K semantic neighbors, and the guidance instructions into a structured prompt; inputting the structured prompt into the large language model, parsing and outputting the pseudo-label of the node and the corresponding text explanation.

[0054] Further, in step one, the first view node embedding matrix is ​​obtained by applying the first pre-trained language encoder to the original text set of all nodes, and the dimension of the first view node embedding matrix is ​​the number of nodes multiplied by the embedding dimension; the second view node embedding matrix is ​​obtained by applying the second pre-trained language encoder to the large language model generating the interpretation set of all nodes, and the dimension of the second view node embedding matrix is ​​the number of nodes multiplied by the embedding dimension; the network structure of the second pre-trained language encoder is the same as that of the first pre-trained language encoder but the parameters are independent; the supervised contrastive loss function of the first view is calculated in the following way: for each node in the current training batch, calculate the... The similarity between a node and all positive sample node embeddings, divided by the temperature parameter, is multiplied by the sum of the similarity between that node and all other node embeddings, divided by the temperature parameter. The logarithm is then negative, and the sum is averaged over all positive samples and all nodes. The positive sample nodes are those with the same pseudo-label as the current node, including embeddings of the same node in different augmented views. The temperature parameter is a hyperparameter with a value ranging from 0.05 to 0.5. The supervised contrast loss for the second view is calculated using the same method, but with a second-view neural network encoder. The total loss for dual-view contrast learning is the sum of the supervised contrast loss for the first view and the supervised contrast loss for the second view.

[0055] Furthermore, in step two, for each edge, a label consistency score is defined as an indicator function, which takes the value of 1 when the pseudo-labels of two nodes are the same, and 0 otherwise. The edge deletion probability is calculated as: the base deletion rate multiplied by one minus the label consistency score; where the base deletion rate ranges from 0.1 to 0.5; when the label consistency score is 1, the deletion probability is 0, and the edge is forcibly retained; when the label consistency score is 0, the deletion probability is equal to the base deletion rate; in step two, when generating the first enhanced graph and the second enhanced graph, the edge deletion probability is applied independently, and the random seeds for the two perturbations are different, so that the two enhanced graphs have differentiated structures.

[0056] Furthermore, in step three, each layer of the hierarchical bidirectional cross-attention network specifically performs the following operations:

[0057] First, a bidirectional cross-attention operation is performed, specifically: the fused representation of the first view is calculated using the output of the previous layer of the first view as the query and the output of the previous layer of the second view as the key and value, through a multi-head cross-attention mechanism; the fused representation of the second view is calculated using the output of the previous layer of the second view as the query and the output of the previous layer of the first view as the key and value, through a multi-head cross-attention mechanism; in the initial layer, the outputs of the previous layers of the two views are the embeddings of the first view node and the embeddings of the second view node, respectively.

[0058] Then, graph convolution operations are performed to update the embeddings of the two views. Specifically, the first view obtained by the bidirectional cross-attention operation is fused and input into the graph convolutional network. The normalized Laplace transform after adding self-loops to the adjacency matrix is ​​multiplied by the trainable weight matrix, and then processed by a nonlinear activation function to obtain the updated first view embedding. The second view is processed in the same way, using an independent trainable weight matrix to obtain the updated second view embedding. The updated embeddings of the two views are used as the input to the next layer.

[0059] Furthermore, in step three, the aggregation of the fused representations of each layer is performed as follows: the bidirectional cross-attention outputs of each layer are weighted and summed before concatenation, where the weight parameters of each layer are learnable and have equal initial values; or the outputs of each layer are averaged and concatenated; the number of network layers ranges from 2 to 6; the classifier is a multilayer perceptron, whose input is the final node representation and output is the probability distribution of the node belonging to each category; during training, the cross-entropy loss function is used to jointly optimize all trainable parameters, and the cross-entropy loss function is calculated as follows: for each labeled node, the cross-entropy between its one-hot encoding vector of the true label and the probability distribution predicted by the classifier is calculated, and the average is summed over all labeled nodes.

[0060] Furthermore, both the first and second pre-trained language encoders employ pre-trained language models based on the BERT series, RoBERTa series, or Sentence-BERT architecture; both the first and second graph neural network encoders employ graph convolutional networks, graph attention networks, or graph isomorphic network architectures; in the structured prompts generated by the large language model, the guiding instructions include the following: requiring the large language model to predict the category of the target node based on the text information of the target node and its semantically related neighbors, and generating a brief explanation of the prediction results; the output format is specified as a label field and an explanation field; in the supervised contrastive learning of the first and second views, the graph enhancement process of each view simultaneously employs two enhancement strategies: feature mask enhancement and pseudo-label edge perturbation enhancement; the feature mask enhancement randomly sets some dimensions in the node embedding to zero with a preset probability; the pseudo-label edge perturbation enhancement employs an edge deletion strategy based on semantic consistency.

[0061] A text attribute graph node classification system based on dual-view semantic hierarchical fusion guided by a large language model includes:

[0062] The dual-view comparison learning module is used to execute the dual-view comparison learning steps described above.

[0063] The pseudo-label edge perturbation module is used to perform the pseudo-label edge perturbation steps.

[0064] The hierarchical bidirectional cross-attention fusion module is used to perform the hierarchical bidirectional cross-attention fusion steps;

[0065] The classification module is used to perform the classification steps.

[0066] The joint optimization module is used to simultaneously optimize the contrastive loss of the dual-view contrastive learning module and the cross-entropy loss of the classification module. The total loss function is the cross-entropy loss plus a balance coefficient multiplied by the total loss of the dual-view contrastive learning, where the balance coefficient ranges from 0.1 to 1.0. The joint optimization module adopts an end-to-end joint training method, and in each iteration, it simultaneously updates the trainable weight matrices of the first pre-trained language encoder, the second pre-trained language encoder, the first graph neural network encoder, the second graph neural network encoder, the hierarchical bidirectional cross-attention network, and the parameters of the classifier.

[0067] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method.

[0068] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method.

[0069] Compared with the prior art, the present invention has the following technical advantages:

[0070] 1. Dual-view comparative learning architecture;

[0071] Encoding the original text and the LLM-generated interpretation as independent views avoids semantic conflicts caused by direct merging. The two views can be optimized independently, capturing complementary semantic features and enhancing the quality and robustness of node representations.

[0072] 2. Pseudo-label edge perturbation strategy;

[0073] The pseudo-labels generated by LLM are used to evaluate the semantic consistency between connected nodes, and the edge deletion probability is dynamically adjusted according to the consistency score. Compared with the traditional random edge perturbation method, it can more effectively retain semantically consistent edges and delete noisy edges, thus improving the quality of the augmented graph.

[0074] 3. Hierarchical bidirectional cross-attention fusion mechanism;

[0075] By employing bidirectional cross-attention, a fine-grained interaction between the original text and the LLM interpretation is achieved, enabling the model to dynamically emphasize relevant semantic signals. The hierarchical fusion design promotes the gradual alignment of semantic information and graph structure information, improving the discriminative ability of node representations.

[0076] 4. Significantly improves classification accuracy

[0077] It achieves an accuracy of 90.41% on the Cora dataset, 86.15% on the Citeseer dataset, and 94.50% on the PubMed dataset; outperforming all existing baseline methods based on GNN, PLM, and LLM.

[0078] It can be applied to various text attribute graph scenarios such as citation networks, social networks, and e-commerce platforms; it supports datasets of different sizes, effectively processing data ranging from thousands to tens of thousands of nodes. Attached Figure Description

[0079] Figure 1 This is a flowchart of the method of the present invention.

[0080] Figure 2 This is a schematic diagram of the prompt structure.

[0081] Figure 3 It is a comparative learning flowchart, which shows the comparative learning process from each perspective.

[0082] Figure 4 This is a schematic diagram of the pseudo-label edge perturbation strategy, demonstrating how to perform adaptive edge deletion based on label consistency scores.

[0083] Figure 5It is bidirectional cross-attention.

[0084] Figure 6 This is a schematic diagram of the hierarchical integration process. Detailed Implementation

[0085] The technical innovation of this invention lies in:

[0086] The dual-view contrastive learning module divides the node representations in the text attribute graph into two independent views—the original text view and the LLM interpretation view—each independently encoded using a pre-trained language model. Supervised contrastive learning optimizes the embedding space of each view. The module details the architecture design, independent encoding mechanism, and contrastive loss function of the dual-view contrastive learning module.

[0087] Pseudo-label edge perturbation strategy: Use pseudo-labels generated by LLM to evaluate the semantic consistency between connected nodes, and dynamically adjust the edge deletion probability based on the consistency score, prioritizing the retention of semantically consistent edges and deleting semantically inconsistent edges; based on pseudo-label edge perturbation method, semantic consistency scoring mechanism and edge deletion probability adjustment strategy.

[0088] Hierarchical Bidirectional Cross-Attention Fusion Module: This module achieves information interaction between the original text and the LLM interpretation through a bidirectional cross-attention mechanism. It employs a hierarchical design with multiple rounds of iterative fusion and integrates graph structure information by combining it with a graph convolutional network. The module discusses the hierarchical bidirectional cross-attention fusion mechanism, its multi-layer iterative architecture, and its integration with graph convolution.

[0089] This invention constructs a semantic-structural collaborative learning framework that organically combines dual-view comparative learning, pseudo-label edge perturbation, and hierarchical fusion to form a complete semantic-structural collaborative learning framework. This framework utilizes LLM semantic information while maintaining the ability to model graph structures. The overall technical framework and the collaborative working mechanism between its modules are also described.

[0090] The technical solution of this invention is a text attribute graph node classification method based on dual-view semantic hierarchical fusion guided by a large language model, comprising the following three core modules:

[0091] 1. Dual-perspective comparative learning module;

[0092] Directly merging the original text with the LLM-generated interpretation can lead to conflicts or semantic interference, negatively impacting the model's learning process. To address this issue, a dual-view contrastive learning module is designed, where the two information sources learn in independent and separate spaces. This approach preserves complementary information from both perspectives, ensuring that each view focuses on its unique semantic features, ultimately improving the quality and robustness of node representations.

[0093] 1.1 Prompt for building;

[0094] To extract high-quality semantic interpretations and pseudo-labels from LLM, a structured cue was constructed that includes not only information about the target node but also its semantically most relevant neighbors, thus providing rich local context.

[0095] To select these information-rich neighbors, a similarity-based retrieval strategy is employed. Specifically, text embeddings obtained using a pre-trained language model (e.g., BERT) are used to calculate the cosine similarity between the target node and all other nodes. Formally, given a node... and Embedded vector and The cosine similarity is calculated as follows:

[0096]

[0097] in and These represent the text embedding vectors extracted from two nodes by a pre-trained language model (such as BERT), numerator and molecule. The dot product of two vectors, denominator It is the product of the L2 norms of two vectors.

[0098] Then, the final prompt is constructed by connecting three components: the original text attributes of the target node, the text content of the top-k most similar neighbors sorted by cosine similarity, and carefully designed instructions to guide the LLM in performing pseudo-label prediction and interpretation generation.

[0099] 1.2 Dual-view representation;

[0100] Each node Associate a text property Obtain the pseudo-labels for each node from the LLM. and corresponding explanations Then, the semantics of the nodes are modeled from two complementary perspectives.

[0101] The first view, called the text view, represents each node based on its raw text attributes. The input text is encoded into node embeddings using a pre-trained language model fine-tuned for the task. Let... Represents the set of original text. Calculation:

[0102] in This represents a node representation matrix based on the original text. It is a language model for fine-tuning text views. It is the collection of all the original text attributes of the nodes. , The total number of nodes. is the dimension of the embedding vector.

[0103] The second view, called the interpretation view, captures independent semantic perspectives based on LLM-generated interpretations. These interpretations utilize language models. Encoding, the model and They have the same architecture and initialization, but are trained separately. Let... This represents the generated set of explanations. The embedding computation based on the explanations is as follows:

[0104] in It is a node representation matrix generated and interpreted based on LLM. It is a language model trained independently for interpreting views (and) (Same architecture but no shared parameters) It is the set of all nodes corresponding to the LLM-generated interpretations. , and The meaning is the same as that in formula (2).

[0105] 1.3 Visual comparison of learning objectives;

[0106] For the two views—a text view and an interpretation view—a graph contrastive learning framework is applied to learn robust and semantically meaningful node embeddings. Specifically, for the text view, two augmented graphs (A1, X1) and (A2, X2) are generated through random feature masks and pseudo-label edge perturbations. These graphs are then passed through a shared GNN encoder to generate node embeddings. and .

[0107] Using supervised contrastive loss, nodes with the same label are embedded closer together in different augmented views, while nodes with different labels are embedded further apart. The loss for the text view is defined as:

[0108]

[0109] in This represents the contrast loss value of the text view. It is the set of all nodes. The total number of nodes. Includes nodes All node indices sharing the same label (excluding all enhanced views) itself), The size of the set, and They are nodes and its positive sample nodes Embedding vectors generated in the contrastive learning framework It is a temperature hyperparameter that controls the sharpness of the distribution, and in the denominator Indicates except The embedding vectors of all other nodes.

[0110] The interpretive view is processed using the same contrastive learning pipeline to produce its own set of embeddings. and embeddings from the original text .

[0111] These embeddings serve as input to a hierarchical bidirectional cross-attention fusion module, which integrates information from both modalities for downstream classification.

[0112] 2. Pseudo-label edge disturbance module;

[0113] In traditional graph contrastive learning, random edge perturbation is widely used to generate diverse views. However, this strategy lacks semantic awareness and may unintentionally delete meaningful edges or retain semantically inconsistent edges, thereby reducing the quality of the augmented view and decreasing the discriminative power of the learned node representations. To address this issue, a pseudo-labeled edge perturbation strategy incorporating LLM semantic signals is introduced.

[0114] We use pseudo-labels generated by LLM to evaluate the semantic consistency between connected nodes. For each edge... Define label consistency score for:

[0115]

[0116] in Indicates the connection node and The edge, and These are the pseudo-labels generated by LLM for the two nodes. and It is a hyperparameter that controls the degree of edge preservation and satisfies Both take values ​​in open intervals. When two nodes share the same pseudo-label, the score is... Otherwise, the score is A higher score indicates stronger semantic consistency.

[0117] Further adjust the edge deletion probability based on the consistency score. Let... Indicates the base deletion rate. Adjusted deletion probability. The calculation is as follows:

[0118]

[0119] in Indicates the base deletion rate. The consistency score defined by formula (5) To adjust the back The actual deletion probability. This formula makes the edges between semantically similar nodes ( Larger nodes are deleted with a lower probability, while edges between semantically inconsistent nodes ( Smaller (those with a higher probability) are deleted.

[0120] This formula ensures that edges connecting semantically similar nodes (i.e., nodes sharing the same pseudo-labels) are preferentially preserved, while edges between semantically inconsistent nodes are more likely to be deleted. By using LLM-derived pseudo-labels as soft guides during the construction of the augmented graph, this method replaces traditional random perturbations with a semantically aware strategy. This encourages the augmented graph to faithfully capture the underlying latent class structure, resulting in more robust and discriminative node representations.

[0121] 3. Hierarchical bidirectional cross-attention fusion module;

[0122] To effectively integrate complementary semantic information from the original text and the LLM-generated interpretation, a hierarchical bidirectional cross-attention fusion module is proposed. Initial embedding and From the dual-view comparison learning framework.

[0123] In each layer A bidirectional cross-attention mechanism is applied to enable information exchange between the two modalities. Specifically, each modality pays attention to the other modality, and their attention outputs are concatenated to produce a fused representation.

[0124]

[0125] in For the first The fusion features of the layer output, Indicates text view embedding Embedded as a query and explanation view As the result of cross-attention calculation for keys and values Conversely, This represents a vector concatenation operation. Initial embedding. and From the dual-view comparison learning framework.

[0126] After fusion, the graph convolutional network updates the embeddings for each modality to integrate the structural context:

[0127]

[0128] in and The first Embedding of layered text views and explanation views, This represents a graph convolutional network operation that integrates graph structural context information into the embedding of each modality.

[0129] Updated embedding and This serves as the input for the next fusion layer. This alternating process of cross-attention fusion and graph convolution is repeated. Layers facilitate the gradual and hierarchical integration of semantic and structural information.

[0130] After completing all layers, the fused features of each layer are aggregated to obtain the final joint representation:

[0131]

[0132] in This is the final joint representation obtained after aggregating the features from all layers. This represents the total number of floors. For the first Layer fusion features. This formula sums the fusion results of all layers to achieve comprehensive utilization of multi-level semantic information.

[0133] Finally, the aggregated representation is input into the multilayer perceptron to predict node labels:

[0134]

[0135] in The node labels predicted by the model. This represents a multilayer perceptron classifier. This is the final joint representation obtained from formula (9). Inputting the aggregated representation into a multilayer perceptron will output the predicted label for each node.

[0136] This hierarchical fusion design clearly distinguishes between semantic integration and structural learning: embedding The topology of the captured graph is propagated through GCN, while Multi-source semantics are fused through cross-attention. and The iterative interactions between components facilitate the gradual alignment of structural and semantic information, improving the robustness of the representation and reducing modal bias. This tightly coupled, modular framework enhances the accuracy and generalization ability of node classification.

[0137] References:

[0138] [1] Learning on large-scale text-attributed graphs via variationalinference.

[0139] [2] Harnessing explanations: LLM-to-LM interpreter for enhanced text-attributed graph representation learning.

[0140] [3] A method for classifying social network nodes based on graph neural networks. Application publication number: CN120105163A.

[0141] [4] A graph neural network interpretation method and apparatus based on FPGA acceleration, application publication number: CN117688972A.

Claims

1. A text graph node classification method based on dual-view semantic fusion of a large language model, characterized in that, Includes the following steps: Step 1: Dual-perspective comparative learning; Each node in the text attribute graph is constructed as a first view based on the original text and a second view based on the explanation and pseudo-labels generated by the large language model. The first view and the second view are encoded by independent first pre-trained language encoders and second pre-trained language encoders respectively to obtain the first view node embedding and the second view node embedding. The embedding spaces of the first view node embedding and the second view node embedding are optimized by performing supervised comparative learning on the first view node embedding and the second view node embedding through independent first graph neural network encoder and second graph neural network encoder respectively. Step 2: Perturb the edge of the pseudo-label; The pseudo-labels generated by the large language model are used to evaluate the semantic consistency between connected nodes. The deletion probability of each edge in the graph is dynamically adjusted according to the consistency score to generate a semantically aware first augmented graph and a second augmented graph, which are used for comparative learning of the first view and the second view, respectively. Among them, the edges connecting semantically consistent nodes are preferentially retained, and the edges connecting semantically inconsistent nodes are deleted with a higher probability. Step 3: Hierarchical bidirectional cross-attention fusion; The embeddings of the first and second view nodes are input into a multi-layered hierarchical bidirectional cross-attention network. Each layer performs the following operations in sequence: bidirectional cross-attention operation, which enables the first and second views to pay attention to each other and generate a fused representation; graph convolution operation, which updates the embeddings of the two views respectively to integrate graph structure information; the fused representations generated by each layer are aggregated to obtain the final node representation; and the final node representation is input into a classifier to predict the node category.

2. The text graph node classification method based on dual-view semantic fusion of a large language model according to claim 1, characterized in that, In step one, a structured prompt is constructed for each node to query the large language model. Specifically, this includes: calculating the cosine similarity of the text embeddings between nodes using a pre-trained language model. The cosine similarity is obtained by calculating the dot product of the embedding vectors of two nodes and dividing it by the product of the magnitudes of the two vectors; selecting the K nodes with the highest similarity as the semantic neighbors of the node, where K ranges from 3 to 10; concatenating the original text of the target node, the texts of the K semantic neighbors, and the guidance instructions into a structured prompt; inputting the structured prompt into the large language model, parsing and outputting the pseudo-label of the node and the corresponding text explanation.

3. The text graph node classification method based on dual-view semantic fusion of a large language model according to claim 1, characterized in that, In step one, the first view node embedding matrix is ​​obtained by applying the first pre-trained language encoder to the original text set of all nodes, and the dimension of the first view node embedding matrix is ​​the number of nodes multiplied by the embedding dimension. The second-view node embedding matrix is ​​obtained by applying the second pre-trained language encoder to the large language model generated from all nodes to generate an interpretation set. The dimension of the second-view node embedding matrix is ​​the number of nodes multiplied by the embedding dimension. The network structure of the second pre-trained language encoder is the same as that of the first pre-trained language encoder, but the parameters are independent. The supervised contrastive loss function of the first view is calculated as follows: for each node in the current training batch, the similarity between the node and the embeddings of all positive sample nodes is calculated by dividing the exponent by the temperature parameter, then divided by the sum of the exponents of the similarity between the node and the embeddings of all other nodes divided by the temperature parameter, the logarithm is taken and the negative value is taken, and then the sum is averaged over all positive samples and all nodes. Wherein, the positive sample node is a node with the same pseudo-label as the current node, including the embeddings of the same node under different augmented views. The temperature parameter is a hyperparameter with a value range of 0.05 to 0.

5. The supervised contrastive loss of the second view is calculated in the same way, but using the second graph neural network encoder. The total loss of dual-view contrastive learning is the sum of the supervised contrastive loss of the first view and the supervised contrastive loss of the second view.

4. The text graph node classification method based on dual-view semantic fusion of a large language model according to claim 1, characterized in that, In step two, for each edge, the label consistency score is defined as an indicator function, which takes the value of 1 when the pseudo-labels of two nodes are the same, and takes the value of 0 otherwise; the edge deletion probability is calculated as: the base deletion rate multiplied by one minus the label consistency score. The base deletion rate ranges from 0.1 to 0.

5. When the label consistency score is 1, the deletion probability is 0 and the edge is forcibly retained. When the label consistency score is 0, the deletion probability is equal to the base deletion rate. In step two, when generating the first and second enhanced graphs, the edge deletion probabilities are applied independently. The random seeds for the two perturbations are different, resulting in differentiated structures for the two enhanced graphs.

5. The text graph node classification method based on dual-view semantic fusion of a large language model according to claim 1, characterized in that, In step three, each layer of the hierarchical bidirectional cross-attention network performs the following specific operations: First, a bidirectional cross-attention operation is performed, specifically: the fused representation of the first view is calculated by using the output of the previous layer of the first view as the query and the output of the previous layer of the second view as the key and value, through a multi-head cross-attention mechanism. The fusion representation of the second view is calculated by using the output of the previous layer of the second view as the query and the output of the previous layer of the first view as the key and value, through a multi-head cross-attention mechanism. In the initial layer, the outputs of the layer above the two views are the first view node embedding and the second view node embedding, respectively; Then, graph convolution operations are performed to update the embeddings of the two views. Specifically, the first view fusion representation obtained by the bidirectional cross-attention operation is input into the graph convolutional network, multiplied by the normalized Laplace transform after adding self-loops to the adjacency matrix and the trainable weight matrix, and then processed by a nonlinear activation function to obtain the updated first view embedding. The second view is processed in the same way, using an independent trainable weight matrix to obtain the updated second view embedding. The updated two views are embedded as inputs to the next layer.

6. The text graph node classification method based on dual-view semantic fusion of a large language model according to claim 5, characterized in that, In step three, the aggregation of the fused representations of each layer is performed as follows: the bidirectional cross-attention outputs of each layer are weighted, summed, and then concatenated, where the weight parameters of each layer are learnable and have equal initial values; or the outputs of each layer are averaged and then concatenated; the number of network layers ranges from 2 to 6; the classifier is a multilayer perceptron, whose input is the final node representation and output is the probability distribution of the node belonging to each category; during training, the cross-entropy loss function is used to jointly optimize all trainable parameters, and the cross-entropy loss function is calculated as follows: for each labeled node, the cross-entropy between its one-hot encoding vector of the true label and the probability distribution predicted by the classifier is calculated, and the average is summed over all labeled nodes.

7. The text graph node classification method based on dual-view semantic fusion of a large language model according to claim 1, characterized in that, Both the first and second pre-trained language encoders employ pre-trained language models based on the BERT series, RoBERTa series, or Sentence-BERT architecture; both the first and second graph neural network encoders employ graph convolutional networks, graph attention networks, or graph isomorphic network architectures; in the structured prompts generated by the large language model, the guiding instructions include the following: requiring the large language model to predict the category of the target node based on the text information of the target node and its semantically related neighbors, and generate a brief explanation of the prediction results; The output format is specified as label fields and explanation fields; in the supervised contrastive learning of the first and second views, the graph augmentation process of each view simultaneously employs two augmentation strategies. Feature mask enhancement and pseudo-label edge perturbation enhancement; The feature mask enhancement randomly sets some dimensions of the embedded node to zero with a preset probability; the pseudo-label edge perturbation enhancement adopts the edge deletion strategy based on semantic consistency as described in claim 5.

8. A text graph node classification system based on dual-view semantic fusion of a large language model, characterized in that, include: A dual-view comparison learning module is used to execute the dual-view comparison learning steps as described in claim 1; A pseudo-label edge perturbation module is used to perform the pseudo-label edge perturbation step as described in any one of claims 1; A hierarchical bidirectional cross-attention fusion module is used to perform the hierarchical bidirectional cross-attention fusion step as described in claim 1; The classification module is used to perform the classification steps. The joint optimization module is used to simultaneously optimize the contrastive loss of the dual-view contrastive learning module and the cross-entropy loss of the classification module. The total loss function is the cross-entropy loss plus a balance coefficient multiplied by the total loss of the dual-view contrastive learning, where the balance coefficient ranges from 0.1 to 1.

0. The joint optimization module adopts an end-to-end joint training method, and in each iteration, it simultaneously updates the trainable weight matrices of the first pre-trained language encoder, the second pre-trained language encoder, the first graph neural network encoder, the second graph neural network encoder, the hierarchical bidirectional cross-attention network, and the parameters of the classifier.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 1 to 7.

Images