Semi-supervised node classification method based on semantic guidance of multi-layer structure

Abstract

The application discloses a kind of based on the multi-layer structure of semantic guidance's semi-supervised node classification method, belong to semi-supervised node classification technical field.The method includes: the graph structure with the local level of semantic guidance is established and the structure model with the global level of node long-range dependency relationship.In the graph structure learning of local level, adaptive fusion topological and node semantic similarity graph structure, utilize graph convolution network to carry out multi-layer aggregation learning node's local embedding;In the structure learning of global level, based on the long-range dependency relationship between node pairs of light weight transformer encoder network learning, obtain the comprehensive representation of node;Classification is carried out based on final node representation, and the classification label of each node is obtained as the result of semi-supervised node classification.The semi-supervised node classification method based on the layered structure of semantic guidance in the application can consider the performance and the simplicity of complexity of classification on benchmark dataset.

Description

Technical Field

[0001] This invention belongs to the field of semi-supervised node classification technology, and mainly relates to a semi-supervised node classification method based on a semantically guided multi-layer structure. Background Technology

[0002] Semi-supervised learning can effectively utilize a small amount of labeled data and a large amount of unlabeled data for training. The semi-supervised node classification problem based on graph nodes uses the graph structure constructed from the relationships between nodes and the labels of some nodes to predict the labels of unlabeled nodes.

[0003] For example, in the node classification problem of citation networks, a relationship graph of academic papers can be constructed. Each paper represents a node, represented by a bag-of-words attribute vector, and the citation links between papers form the edges of the graph. To accurately classify nodes, it is necessary to comprehensively consider node attributes, node labels, and the edges between nodes.

[0004] Graph Neural Networks (GNNs) are deep learning models based on graph data. They employ a recursive neighborhood aggregation scheme, where each node updates its current feature vector by aggregating features from its neighborhood and predicts the node's label. GNNs have gained widespread attention for solving various machine learning problems involving graph data, including node-centric tasks (such as node classification and link prediction) and graph-centric tasks (such as graph similarity detection and graph classification).

[0005] The previous work had the following problems:

[0006] (1) The degree of nodes in the graph usually follows a long-tail distribution, that is, many nodes are low-degree nodes with few neighboring nodes. This means that in traditional GNN models, these nodes can only obtain the least information from their neighbors, resulting in poor performance, reduced effectiveness of the learned representations, and impact on the performance of downstream tasks.

[0007] (2) GNN-based methods ignore the semantic information that exists between nodes. For example, in a citation network, academic papers of the same category will not cite each other; they are semantically similar, but they are not connected in the graph. Semantic similarity is obtained by measuring the similarity between document character sequences.

[0008] (3) The GNN method effectively broadens the receptive domain by aggregating information from the external local neighborhood through stacked layers. However, when too many graph convolutional layers are stacked, the node embeddings tend to converge to a specific value, making classification more difficult. In these tasks, some nodes may also depend on the representations of distant nodes, and GNNs with limited receptive domains may not be able to capture these dependencies.

[0009] (4) The self-attention mechanism optimizes the representation of each position by aggregating features from all other positions in the sample, which leads to quadratic computational complexity in terms of the number of positions in the sample. At the same time, self-attention only focuses on the self-affinity between a single sample and other samples at different positions, while ignoring the potential correlation between other samples.

[0010] Therefore, the motivation for this invention is to propose a technique capable of integrating semantic associations between nodes, local topological structures between nodes, and long-range dependencies. The relevant background technology is described in detail below.

[0011] (1) Semi-supervised learning

[0012] Semi-supervised learning is a model constructed using both labeled and unlabeled data. Semi-supervised learning methods can improve learning performance by exploring latent patterns in additional unlabeled data. Semi-supervised learning reduces the need for large amounts of labeled data.

[0013] (2) Semi-supervised node classification based on GNN

[0014] Semi-supervised learning methods based on GNNs typically learn data representations by constructing better message passing patterns and graph structures for the graph neural network. Kipf et al. (2017) proposed Graph Convolutional Networks (GCNs) to generate new node representations through the Chebyshev unfolding of the Laplacian graph. (2018) et al. proposed Graph Attention Network (GAT), which uses a self-attention mechanism to assign different weights to each pair of adjacent nodes and generate new representations of the nodes.

[0015] To acquire more graph information, other works extend graph convolution beyond the neighborhood, improving the ability to aggregate information from non-local regions by expanding topological nearest neighbors. Abu et al. (2020) proposed N-GCN, which trains a GCN on node pairs at different distances discovered in random walks and learns the fusion of multiple GCN outputs. Pei et al. (2020) proposed Geom-GCN, which maps the graph to a continuous latent space through node embeddings, uses geometric relations defined in the latent regions to construct structural neighborhoods, and performs aggregation. Zhu et al. (2020) proposed H2GCN, which learns corresponding aggregation rules for neighbors of different classes. Wang et al. (2021) proposed MAGNA, a graph neural network that considers the contextual dependencies between nodes through a multi-hop self-attention mechanism. Yang et al. (2021) proposed SelfSAGCN, which achieves semantic alignment of class center similarity by learning features of similar nodes from semantics and graph structure. Li et al. (2022) proposed GloGNN, which utilizes the global homogeneity of nodes in the graph by adding all nodes to their neighborhoods.

[0016] (3) Transformer framework

[0017] The Transformer is a prominent deep learning model that has been widely applied in various fields, such as Natural Language Processing (NLP), Computer Vision (CV), and Speech Processing. The Transformer was initially proposed in 2017 as a sequence-to-sequence model for machine translation. A core component of the Transformer is Self-Attention, which updates the features at each location by calculating a weighted sum of features using pairwise similarities across all locations, thereby capturing long-range dependencies within individual instances. Summary of the Invention

[0018] To address the shortcomings of existing technologies, this invention provides a semi-supervised node classification method based on semantically guided multi-layer structures, proposing a novel model for learning ideal node representations—Semantically Guided Multi-Layer Semi-Supervised Graph Node Classification (SGHSA). This model learns the network structure from both local and global variables. In local structure learning, a novel graph structure is designed to integrate graph structure and node semantic relationships in a data-driven manner. This invention incorporates the ideas of the Transformer framework and proposes a lightweight attention mechanism that considers potential associations between other nodes, increasing the amount of information learned.

[0019] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0020] A semantically guided, multi-layered semi-supervised node classification method, comprising the following steps:

[0021] Construct a graph structure with semantically guided local hierarchy to obtain local embeddings of nodes containing semantic associations;

[0022] Establish a global hierarchical structural model with long-range dependencies between nodes, and obtain a comprehensive representation of the nodes by taking the local embeddings of nodes with semantic associations as input.

[0023] A classification model is established, which takes the comprehensive representation of nodes as input and obtains the node classification results; the classification model includes cascaded linear layers and classifiers.

[0024] Furthermore, in establishing the local hierarchical graph structure: first, a semantic similarity graph is constructed based on the node topology graph; then, the node topology graph and the semantic similarity graph are fused to obtain the local hierarchical structure graph; finally, a graph convolutional network (GCN) is used to aggregate nodes in the local hierarchical structure graph to obtain local embeddings of nodes containing semantic associations.

[0025] Furthermore, a fractional vector is introduced to adaptively fuse the topological graph and semantic similarity graph of the nodes, resulting in a local hierarchical structure graph, where the fused graph matrix P of the l-th layer is... (l) Represented as:

[0026]

[0027] In the formula, A is the adjacency matrix of the node topology graph, D is the degree matrix of A, and A (k) D is the adjacency matrix of the K-hop nearest neighbors of a node. (k) It is A (k) The degree matrix, u (l) ∈R n It is used to balance A and A (k) The fractional vector a⊙B represents the product of each element of vector a with the corresponding row of matrix B.

[0028] Furthermore, u (l) Calculated using the following formula:

[0029]

[0030] In the formula, H (l-1) Let l be the representation matrix of the (l-1)th hidden layer. and All of them are H (l-1) The transformation matrix transforms the vector into a fractional vector, where σ(·) is the activation function and n is the number of nodes.

[0031] Furthermore, a graph convolutional network (GCN) is used to aggregate nodes in the local hierarchical structure graph, resulting in local embeddings of nodes containing semantic relationships.

[0032]

[0033] In the formula, P (l+1) P (l) These are the fusion graph matrices of the (l+1)th and lth layers in the local hierarchical structure diagram, respectively. For P (l+1) The degree matrix, W( l H is the weight matrix of the l-th layer. (l) H is the hidden layer representation matrix of the l-th layer. (l) The expression is:

[0034]

[0035] In the formula, For P (l) The degree matrix is ​​denoted by , and Relu(·) is the activation function.

[0036] Furthermore, the global hierarchical structure model is an improved Transformer encoder consisting of N cascaded identical layers, with each layer having two cascaded sub-modules. The first sub-module is a multi-head attention mechanism (MHA), and the second sub-module is a position-fully connected feedforward network.

[0037] Furthermore, the first submodule employs a multi-head attention mechanism (MHA) to calculate and aggregate the attention values ​​between nodes, where:

[0038] The single-head attention LWA is represented as:

[0039]

[0040] F out =QM v

[0041] In the formula, M k M v ∈R s×d These are two different shared embedding units, serving as the key and value in the attention mechanism, respectively. For local embedding of nodes with semantic relationships, it serves as the query in the attention mechanism; Norm(.) is the normalization function, Q is the attention map inferred from the input, and F... out It aggregates neighborhood information according to the size of Q;

[0042] Multi-head attention (MHA) is represented as:

[0043] h i =LWA(F i M k M v )

[0044]

[0045] In the formula, F i ∈R n×(d / t) yes The matrix after dimensional transformation, i.e. h i This is the attention mapping for the i-th head, where t is the number of heads for attention, and Concat(.) is the join operation; W o It is a linear transformation matrix.

[0046] Furthermore, each improved Transformer encoder layer uses a residual connection after its two sub-modules, followed by layer normalization, specifically:

[0047]

[0048] In the above formula, LayerNorm(.) is the layer normalization function, and Z2 is the node output representation of the improved Transformer encoder for each layer.

[0049] Furthermore, the classification model employs the cross-entropy loss function, with the classifier being a softmax classifier.

[0050] The present invention also provides a semantically guided, multi-layered semi-supervised node device, including one or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include steps for performing the methods described above.

[0051] The significant advantages of this invention compared to existing technologies are:

[0052] 1) An end-to-end hierarchical network structure is proposed, which simultaneously learns the local and global spatial structure information of the graph, thereby improving the representation capability;

[0053] 2) When learning local information, a new adaptive balancing mechanism was designed by combining the semantic information and topological structure of the data;

[0054] 3) When learning global information, the global structural relationships of nodes are learned based on the improved Transformer;

[0055] 4) A novel lightweight multilayer perceptron (MLP-like) attention mechanism is proposed, which differs from existing methods and reduces the complexity of the Transformer. Attached Figure Description

[0056] Figure 1This is a logic block diagram of the semi-supervised node classification method of this patent;

[0057] Figure 2 It is graph learning of local semantic relationships;

[0058] Figure 3 It involves learning long-range node dependencies within the global structure;

[0059] Figure 4 These are visualizations of the ablation experiments of the SGHSA method of this invention and existing technologies (Cora dataset). Among them, (a) is a visualization of the classification effect based on the GCN model, (b) is a visualization of the classification effect based on the semantic fusion graph (GCN+Sem), (c) is a visualization of the classification effect based on the combination of GCN and the traditional Transformer encoder (GCN+Trans), (d) is a visualization of the classification effect based on the combination of the semantic fusion graph and the traditional Transformer encoder (GCN+Sem+Trans), (e) is a visualization of the classification effect based on the combination of GCN and the improved Transformer encoder in this patent (GCN+Trans+LWA), and (f) is a visualization of the model (SGHSA) proposed in this invention. Detailed Implementation

[0060] This invention relates to a semi-supervised node classification method based on a semantically guided multi-layer structure, comprising: establishing a graph structure with semantically guided local layers and a global layer structure model with long-range dependencies between nodes. In the local layer graph structure learning, the topology and node semantic similarity graph structure are adaptively fused, and a graph convolutional network is used to perform multi-layer aggregation to learn the local embedding of nodes; in the global layer structure learning, a lightweight transformer encoder network is used to learn the long-range dependencies between node pairs to obtain a comprehensive representation of the nodes; and classification is performed based on the final node representation to obtain a classification label for each node, which serves as the result of the semi-supervised node classification.

[0061] like Figure 1 As shown, the specific steps of the present invention are as follows:

[0062] Construct a graph structure with semantically guided local hierarchy to obtain local embeddings of nodes containing semantic associations;

[0063] Establish a global hierarchical structural model with long-range dependencies between nodes, and obtain a comprehensive representation of the nodes by taking the local embeddings of nodes with semantic associations as input.

[0064] A classification model is established, which takes the comprehensive representation of nodes as input and obtains the node classification results; the classification model includes cascaded linear layers and classifiers.

[0065] Furthermore, such as Figure 2 As shown, in the establishment of the local hierarchical graph structure: first, a semantic similarity graph is constructed based on the node topology graph; then, the node topology graph and the semantic similarity graph are fused to obtain the local hierarchical structure graph; finally, the graph convolutional network (GCN) is used to aggregate the nodes of the local hierarchical structure graph to obtain the local embedding of nodes with semantic associations.

[0066] Specifically, given the original graph structure G = (V, E, X) of the nodes, where V = {v1, v2, ..., v...} n Let} be a set of n nodes, E represent the original topological associations between the nodes, and the node vector X = {x1, x2, ..., xn}. n}∈R n×d Let be the features of n nodes, and d be the dimension. The graph structure G can be derived from the adjacency matrix A∈{0,1}. n×n This means that here a i,j =1 indicates that there is an edge between nodes i and j; otherwise, a i,j =0. The degree matrix of A is represented by D.

[0067] Let G represent the first K-hop (K-hop) adjacency matrix of node X. The K-hop graph is represented as G. (k) For G (k) The semantic similarity is calculated for each pair of nodes with a K-hop relationship. And construct the adjacency matrix of the semantic similarity graph, i.e., A ij =S ij A (k) The degree matrix is ​​represented as D (k) .

[0068] After calculating the semantic similarity between nodes, a fusion graph is constructed using the following two adaptive features:

[0069] (1) Each node adaptively balances information from the topological graph and the semantically similar graph;

[0070] (2) Each node can adjust the contribution of its node characteristics.

[0071] After calculating the semantic similarity matrix, the fusion graph matrix P of the l-th layer is... (l) It can be represented as:

[0072]

[0073] Among them, u (l) ∈R n It is a fractional vector used to balance the topological and semantic graphs; a⊙B represents the product of each element of vector a with the corresponding row of matrix B. (l)It allows nodes to adaptively combine information from the two graphs, assigning nodes different scores. (l) The model is as follows:

[0074]

[0075] in, The hidden layer representation matrix (H) represents the (l-1)th hidden layer. (0) =X), and To convert H (l-1) to fractional vector u (l) The transformation matrix; σ(·) is the activation function, which is defined here as the Sigmoid function.

[0076] After calculating the fused graph, GCN is used for node aggregation. Input node features H (0) The l-th hidden layer representation matrix H of node X (l) :

[0077]

[0078] In the above formula, Relu(.) is the activation function.

[0079] The final output is a node embedding containing semantically related local structures:

[0080]

[0081] Furthermore, such as Figure 3 As shown, the global hierarchical structure model is an improved Transformer encoder, consisting of N cascaded identical layers. Each layer has two cascaded sub-modules. The first sub-module is a multi-head attention mechanism (MHA) similar to an MLP, and the second sub-module is a position-fully connected feedforward network.

[0082] Specifically, the first submodule employs a multi-head attention mechanism (MHA) similar to MLP to calculate and aggregate the attention values ​​between nodes; after each layer of the improved Transformer encoder, the two submodules are connected by residuals and then the layers are normalized to obtain the node output representation of each layer of the improved Transformer encoder.

[0083] Specifically, based on the obtained GCN encoding of each node The resulting node vectors are then provided to the improved Transformer encoder.

[0084] Specifically, firstly, a low-dimensional embedding unit matrix M of learnable nodes, independent of the input, is randomly initialized. k ∈Rs×d Its function is similar to MLP, local embedding. This refers to the query in the attention mechanism.

[0085] The expression for single-head attention (LWA) is:

[0086]

[0087] F out =QM v

[0088] in This indicates that the attention value is calculated for each query and embedding unit, where s is the dimension of the attention value, Norm(.) is the normalization function, and Q = (q i,j )∈R n×d The attention map inferred from the input; M k M v ∈R s×d Two different shared embedding units, serving as keys and values ​​in the attention mechanism; F out To aggregate neighborhood information according to the size of Q.

[0089] The Norm(.) function here performs standardization on both the row and column levels separately:

[0090]

[0091] Specifically, first calculate the product of the query and the key value. Then First normalize by row, then normalize by column to get q. ij .

[0092] Multi-head attention (MHA) calculates the attention values ​​between nodes multiple times to obtain different relationships between nodes.

[0093] h i =LWA(F i M k M v )

[0094]

[0095] In the formula, F i ∈R n×(d / t) yes The matrix after dimensional transformation, i.e. h i This is the i-th head attention mapping, where t is the number of heads for attention, Concat(.) is the join operation, and W... oIt is a linear transformation matrix that maintains the same dimension for the input and output matrices.

[0096] Specifically, each improved Transformer encoder layer uses a residual connection after its two sub-modules, followed by layer normalization.

[0097]

[0098] In the above formula, LayerNorm(.) is the layer normalization function, and Z2 is the node output representation of the improved Transformer encoder for each layer.

[0099] Furthermore, such as Figure 3 As shown, the classification model uses a softmax classifier and cross-entropy loss.

[0100] Based on the final node representation, classification is performed to obtain the classification label for each node. The classification labels of each node are then used as the semi-supervised node classification results. The specific operation steps are as follows:

[0101] Based on Z-classification, predicted labels for n nodes are obtained.

[0102]

[0103] Based on predicted labels Calculate the cross-entropy loss L with the corresponding true label Y, which is the loss of the training set data:

[0104]

[0105] In the formula, c represents the number of node categories.

[0106] The present invention will be further described in detail below through specific embodiments.

[0107] This embodiment is based on a semantically guided, multi-level, semi-supervised structural node classification method, including model structure learning, classification of test set data, and model ablation studies.

[0108] I. Learning the Model Structure

[0109] 1. This patent uses a real citation dataset for evaluation, and the data is described below:

[0110] The Cora dataset (published in Prithviraj Sen, Galileo Namata, et al. (2008) "Collective Classification in Network Data") is a research citation network dataset where nodes represent publications and edges represent citation links. Node attributes are represented as bag-of-words representations of papers, and all nodes are categorized into 7 research domains. Specifically, it contains 2708 nodes, denoted by X, with n nodes and 5429 edges, denoted by E. Each node has a 1433-dimensional feature vector, denoted by d, and all nodes are categorized into 7 classes, denoted by c.

[0111] The Cora dataset is split into three parts at 60%, 20%, and 20% ratios, serving as the training, validation, and test sets, respectively. Specifically, the number of samples in the training, validation, and test sets are 1624, 542, and 542, respectively, denoted as X. train X val X test The corresponding labels are Y. train Y val ,Y test .

[0112] 2. Model structure parameter learning

[0113] The node semantic similarity graph structure is trained using the following steps:

[0114] Given the original graph structure G = (V, E, X) with all nodes, and the node set V = {v1, v2, ..., v...} 2708 The edge set E = {e1, e2, ..., e} 5429} represents the original topological association between nodes. Graph G is defined by the adjacency matrix A∈{0,1} 2708×2708 It means that if a i,j =1 indicates that each edge in E connects two nodes; otherwise, a i,j =0. The degree matrix of A is represented as D = diag(d1,...,d n ),

[0115] Let represent the K-hop adjacency matrix of node X, where K∈{1,2,3,4,5,6} in the semantic graph construction. Calculate the semantic similarity S for each pair of nodes with K-hop relationships. ij And construct a semantic similarity graph, A (k) The degree matrix is ​​represented as D (k) .

[0116] After calculating the semantic similarity matrix, the fusion graph matrix P of the l-th layer is... (l) It can be represented as:

[0117]

[0118] u (l) The information from the two graphs is adaptively combined to assign different scores to the nodes.

[0119]

[0120] in, This represents the implicit representation of the previous layer. and To convert H (l-1) to fractional vector u (l) The parameter matrix; σ(.) is the activation function, which is defined here as the Sigmoid function.

[0121] After calculating the fused graph, a two-layer GCN is used for node aggregation. The GCN parameter dropout rate is 0.5, and the embedding dimensions are {16, 32, 64, 128, 256, 512}. Input node features H (0) The l-th hidden layer representation of node X is H. (l) :

[0122]

[0123] Two-layer GCN aggregation yields node embeddings with semantically related local structures:

[0124]

[0125] Get the code This is then provided to the improved Transformer encoder, with the encoder's layer count adjustment parameter set to N. T For ∈{1,2}, the parameter deletion ratio adjustment range is {0,0.1,0.2,0.3,0.4,0.5,0.6,0.7}:

[0126] Each layer of the encoder contains two sub-layers: a lightweight MLP-like multi-head attention mechanism layer and a positionally fully connected feedforward network layer. The hidden layer embedding dimension adjustment range is {32, 64, 128, 256, 512}.

[0127] The potential relationships between nodes obtained by a single-layer MLP-like multi-head attention mechanism are...

[0128]

[0129] F out =QMv

[0130] Here M k M v For embedded units, the Norm(.) function normalizes the rows and columns separately.

[0131] Bullish attention is represented as:

[0132] h i =LWA(F i M k M v )

[0133] F out =MHA(F out M k M v = Concat(h1,…,h) t W o

[0134] In the formula, F i ∈R n×t×(d / t) yes The matrix after dimensional transformation, h i This is the i-th head attention mapping, with the head number adjustment parameter being t∈{1,2,4,8}, W o To keep the dimensions of the input and output matrices the same.

[0135] The final representation of the nodes is calculated using residual connectivity and layer normalization methods:

[0136]

[0137] Based on Z2 classification, predicted labels for n nodes are obtained.

[0138]

[0139] Based on predicted labels Calculate the cross-entropy loss L with the corresponding true label Y, which is the loss of the training set data:

[0140]

[0141] In the above formula, c represents the number of node categories.

[0142] Determine the optimal parameters of the model based on the validation dataset.

[0143] Using the parameters in the model, test the data X in the validation dataset. val Classify to obtain category labels And according to label Yval By comparing and validating, the accuracy (ACC) of the validation dataset is obtained. val .

[0144] The parameters are optimized using backpropagation to further reduce the value of the loss function L, for X val Verify again, until Acc val The iteration count will not decrease further, and the early stopping iteration count will be satisfied. In this invention, the maximum number of iterations (epoch) is set to 3000, and the number of early stopping iterations (early stopping) is set to 300.

[0145] The Adam SGD optimizer was used with a learning rate of 0.01 and a weight decay of 0.0005.

[0146] II. Classification of test set data.

[0147] After stopping the iteration, test set data X test Classify the data to obtain category labels. With Y test By comparison, X is obtained. test Classification accuracy ACC test This serves as the classification result for the Cora dataset.

[0148] III. Ablation Experiment Study of the Model

[0149] The final model underwent ablation experiments in four aspects:

[0150] 1. A visualization of the classification performance based on the GCN model (e.g.) Figure 4 (a) in the middle;

[0151] 2. Visualization of classification results based on semantic fusion graph (GCN+Sem) (e.g.) Figure 4 (b) in the middle;

[0152] 3. Visualization of classification performance based on the combination of GCN and traditional Transformer encoder (GCN+Trans) (e.g.) Figure 4 (c) in the middle;

[0153] 4. Visualization of classification performance based on the combination of semantic fusion graph and traditional Transformer encoder (GCN+Sem+Trans) (e.g.) Figure 4 (d) in the middle;

[0154] 5. Visualization of classification performance based on the combination of GCN and the improved Transformer encoder in this patent (GCN+Trans+LWA) (e.g.) Figure 4 (e) in the middle;

[0155] And a visualization of the model (SGHSA) proposed in this invention (e.g.) Figure 4 (f) in the middle.

[0156] from Figure 4 Compared to (a), (c), and (e), (b), (d), and (f) show that the model that takes semantic information into account has clearer boundaries between different types of nodes. This indicates that the fusion graph that takes semantic information into account has narrowed the distance between nodes within a class and increased the distance between nodes between classes.

[0157] from Figure 4 As can be seen from (c), (d), (e), and (f), the addition of the Transformer module to the model can further discover similar nodes at distant locations.

[0158] from Figure 4 In (f), compared to (d), when calculating long-range dependencies between nodes, potential associations between other nodes are also considered, making the weight allocation of associations between nodes more conducive to the classification of query nodes, and the visualization classification effect is more obvious.

[0159] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A semi-supervised node classification method based on semantically guided multi-layered structures, characterized in that, This method is used for classifying nodes in a network and includes the following steps: Construct a graph structure with semantically guided local hierarchy to obtain local embeddings of nodes containing semantic associations; in the graph structure, nodes are publications and edges are citation links. Establish a global hierarchical structural model with long-range dependencies between nodes, and obtain a comprehensive representation of the nodes by taking the local embeddings of nodes with semantic associations as input. A classification model is established, which takes the comprehensive representation of nodes as input and obtains the node classification results; the classification model includes cascaded linear layers and classifiers. In establishing the local hierarchical graph structure: First, a semantic similarity graph is constructed based on the node topology graph; then, the node topology graph and the semantic similarity graph are fused to obtain the local hierarchical structure graph; finally, a graph convolutional network (GCN) is used to aggregate nodes in the local hierarchical structure graph to obtain local embeddings of nodes with semantic associations. By introducing fractional vectors to adaptively fuse the topological graph and semantic similarity graph of nodes, a local hierarchical structure graph is obtained, where the first... Layer fusion graph matrix Represented as: ， In the formula, It is the adjacency matrix of the node topology graph. yes The degree matrix, It is the adjacency matrix of the K-hop nearest neighbors of the nodes. yes The degree matrix, It is used for balance and fractional vectors, Representing vectors Each element and matrix Multiply the corresponding rows; Calculated using the following formula: ， In the formula, For the first Hidden layer representation matrix, and All of them will The transformation matrix that transforms a vector into a fractional vector. For activation function, The number of nodes; The global hierarchical structural model is composed of cascaded... An improved Transformer encoder consisting of identical layers, each layer having two cascaded sub-modules: the first sub-module is a multi-head attention mechanism (MHA), and the second sub-module is a position-fully connected feedforward network.

2. The semi-supervised node classification method based on semantic guidance and multi-layer structure according to claim 1, characterized in that, A graph convolutional network (GCN) is used to aggregate nodes in the local hierarchical structure graph, resulting in local embeddings of nodes with semantic relationships. : ， In the formula, , The first in the local hierarchical structure diagram Layer, First The fusion graph matrix of the layers, for The degree matrix, For the first The weight matrix of the layer, For the first Hidden layer representation matrix, The expression is: ， In the formula, for The degree matrix, This is the activation function.

3. The semi-supervised node classification method based on semantic guidance and a multi-layered structure according to claim 1, characterized in that, The first submodule employs a multi-head attention mechanism (MHA) to calculate and aggregate the attention values ​​between nodes, where: The attention LWA for each head is represented as: ， In the formula, , These are two different shared embedding units, serving as the key and value in the attention mechanism, respectively. This involves local embedding of nodes with semantic relationships, serving as the query in the attention mechanism; Norm(.) is the normalization function. It is an attention map inferred from the input. According to Information about the size of the aggregated neighborhood; Multi-head attention (MHA) is represented as: ， In the formula, yes The matrix after dimensional transformation, i.e. ; It is the first Size attention mapping The number of attention heads; Concat(.) is the join operation. It is a linear transformation matrix.

4. The semi-supervised node classification method based on semantic guidance and multi-layer structure according to claim 3, characterized in that, Each layer of the improved Transformer encoder is followed by two sub-modules with residual connections, and then layer normalization is performed. Specifically: ， In the above formula, LayerNorm(.) is the layer normalization function. Provides the node output representation for each layer of the improved Transformer encoder.

5. The semi-supervised node classification method based on semantic guidance and a multi-layered structure according to claim 1, characterized in that, The classification model uses the cross-entropy loss function, and the classifier is a softmax classifier.

6. A semi-supervised node device with a semantically guided multi-layer structure, characterized in that, It includes one or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include steps for performing the method as described in any one of claims 1 to 5.

Images