A method for learning node representation in heterogeneous graphs based on multi-level autoencoders

Abstract

The present invention belongs to the field of information technology, and discloses a heterogeneous graph node representation learning method based on a multi-level autoencoder, including data collection and preprocessing; constructing a heterogeneous graph data representation, constructing a heterogeneous graph node representation learning model based on a multi-level autoencoder method, designing a dual encoder at the node level and the relationship level, generating a more comprehensive node representation by capturing node and relationship information at different levels, then reconstructing node features through a global decoder, capturing the interaction information between nodes and relationships, and training and optimizing the constructed heterogeneous graph node representation learning model; inputting a data set to be tested into the trained heterogeneous graph node representation learning model, and classifying the nodes in the heterogeneous graph, so as to solve the problem that the existing computational complexity is high and requires a large amount of domain knowledge, so as to solve the problem that the existing method relies too much on node type embedding and ignores relationship features.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a heterogeneous graph node representation learning method based on a multi-level autoencoder. Background Art

[0002] Heterogeneous graph learning has attracted extensive attention in recent years in dealing with complex heterogeneous data. Heterogeneous graphs consist of multiple types of nodes and relationships and are widely used in academic networks, social media analysis, and e-commerce.

[0003] By capturing both node features and relationship features, heterogeneous graph learning can improve the performance in downstream tasks such as node classification and clustering. However, although existing methods utilize both node and relationship information, they usually overemphasize relationship features and fail to fully utilize node features, which limits the performance of the model in complex tasks.

[0004] Traditional heterogeneous graph neural networks learn through meta-paths or meta-relationships. Meta-path-based methods construct semantic relationship paths that can reflect domain knowledge, but they rely on effective meta-path design, which requires a lot of domain expertise and is computationally expensive in large-scale heterogeneous graphs. Although meta-relation-based methods can handle multiple relationship types more generally, they often focus on relationship information and do not make better use of node information.

[0005] For example, the Chinese invention patent with publication number CN117669648A discloses a dual-channel heterogeneous graph node representation learning method based on meta-paths. After introducing distance weights in the first channel, weighted aggregation is performed to obtain the node features of the meta-graph update, and then all meta-graphs are weighted averaged to generate the node feature representation of channel one; then a new weight matrix is ​​generated in channel two by weighted summation of the adjacency matrix, and the node feature representation of channel two is obtained based on this weight matrix. Finally, the node feature representations of the two channels are spliced ​​into the final node feature representation. However, this patent mainly relies on the design of meta-paths, but meta-path design requires a lot of domain knowledge, and in large-scale heterogeneous graphs, the computational complexity increases significantly with the increase in the number of meta-paths.

[0006] For example, the Chinese invention patent with publication number CN113326884A discloses an efficient learning method and device for large-scale heterogeneous graph node representation. It uses the attention mechanism and PPR (Personalized PageRank, a random walk algorithm) algorithm to sample the neighbors of the node; then, based on the type information of the node in the heterogeneous graph, the node type embedding vector is learned; then, the neighbor's type embedding vector and attribute features are fused to obtain the vector representation of the neighbor. However, the patent relies on the PPR algorithm to sample neighbor nodes. Although it can filter important neighbors, the computational complexity is high, and the PPR algorithm cannot fully capture complex relationships. At the same time, this method relies too much on node type embedding and ignores the diversity of relationship features, resulting in insufficient capture of node relationship interaction information.

[0007] In summary, current methods find it difficult to balance multi-level features and achieve ideal results when learning heterogeneous graph representations. Summary of the invention

[0008] In view of the shortcomings of the above-mentioned background technology, the present invention provides a heterogeneous graph node representation learning method based on a multi-level autoencoder. Specifically, a dual encoder at the node level and the relationship level is designed to generate a more comprehensive node representation by capturing node and relationship information at different levels. Then, the node features are reconstructed by a global decoder to capture the interaction information between nodes and relationships. In order to further optimize the proposed method, the present invention introduces a dual loss mechanism, including reconstruction loss and prediction loss. The reconstruction loss ensures that the decoder output is consistent with the initial node features, while the prediction loss optimizes the prediction accuracy of the model through downstream tasks based on the reconstructed features.

[0009] The object of the present invention is achieved by the following technical solution: a heterogeneous graph node representation learning method based on a multi-level autoencoder, the method comprising the following steps:

[0010] Step S1, data collection and preprocessing;

[0011] Step S2: construct heterogeneous graph data representation, treating the different preprocessed data as nodes in the heterogeneous graph ,node Source Node and the target node , different relationships between data are constructed as edges , the relationship between nodes is defined as relationship type , each node corresponds to a node feature;

[0012] Step S3, constructing a heterogeneous graph node representation learning model based on a multi-level autoencoder method,

[0013] Step S3.1, taking the heterogeneous graph data representation as the input of the model, and using the pre-conversion layer to map the node features to the same dimension,

[0014] Step S3.2, before encoding begins, the output of the pre-transformation layer is projected into the latent space through the encoder transformation layer to generate a preliminary representation of the node.

[0015] Step S3.2: Input the preliminary representation of the node into the node-level encoder and the relationship-level encoder to encode the node-level features and the relationship-level features.

[0016] Step S3.3, feature fusion of node-level features and relationship-level features.

[0017] Step S3.4, input the fused data into the global decoder with a cross attention mechanism to reconstruct the features, and finally output the results through the prediction layer;

[0018] Step S4, training and optimizing the constructed heterogeneous graph node representation learning model;

[0019] Step S5, inputting the data to be classified into the trained heterogeneous graph node representation learning model to obtain the classification result.

[0020] Preferably, step S2 specifically includes: defining the heterogeneous graph as a directed graph , where each node and each edge Both of them have their type mapping functions and Related, Represents a collection of nodes, represents the set of edges, Represents a collection of node types, Represents a collection of edge types; connects source nodes and the target node Edge The relationship is expressed as , ; The relationship between nodes is defined as the relationship type , based on the relationship type The target node The neighbor node set of .

[0021] Preferably, the formula for mapping node features to the same dimension using the pre-conversion layer in step S3.1 is:

[0022]

[0023] in, Representation Node The original properties of Representation Node The projected properties, i.e. the output of the pre-transformation layer, and Represent the dimensions of the original space and the projection space respectively, represents the weight matrix for a particular node type, is the activation function.

[0024] Preferably, in step S3.2, before encoding begins, the output of the pre-conversion layer is projected into the latent space through the encoder conversion layer to generate a preliminary representation of the node:

[0025]

[0026] in, Representation Node The latent state before conversion. The input of the first layer of the encoder is the output of the pre-conversion layer. , Representation Node The potential state after conversion, Indicates The node-type-specific weight matrices of the layer encoder, Represents the dimension of the projection space.

[0027] Preferably, in step S3.3, the preliminary representation of the node is input into the node-level encoder to be encoded to obtain the node-level features. In order to focus on capturing the information of the node without considering the heterogeneity, the node-level encoder is designed as follows:

[0028]

[0029]

[0030] in, Represents the source node The transformed latent state, i.e. the output of the encoder transformation layer, represents node-level features, For the The node-level weight matrix of the layer node-level encoder is used to transform neighbor node features, Indicates the target node The neighbor set of is the activation function, is a symmetric normalization coefficient based on node degree, Represents the source node The degree, Indicates the target node The degree.

[0031] Preferably, in step S3.3, in order to capture the relational information in the heterogeneous graph, a relation-level encoder is designed to abstract a specific relational subgraph, i.e., a specific type of edge and its connected nodes, and then aggregate them in the subgraph of the specific relation to obtain the relational information based on the relational features. Obtain node representations, and finally aggregate the node representations of specific relationships to obtain relationship-level features. The preliminary representation of the node is input into the relationship-level encoder to encode the relationship-level features, as follows:

[0032] In order to accurately reflect the structure of the graph and distinguish different types of relations, the relation-level encoder encodes each relation type. The source node To perform conversions on a specific relation type:

[0033]

[0034] in, for The converted representation is Represents the source node The transformed latent state, i.e. the output of the encoder transformation layer, is the weight matrix for a specific relationship type;

[0035] Introduce an attention mechanism containing specific relationship information to distinguish different relationship types for target nodes Contributions:

[0036]

[0037]

[0038] in, Represents the source node In the relationship type Target Node The importance weight of Indicates the target node The transformed latent state, i.e. the output of the encoder transformation layer, is the weight matrix of a specific relationship, represents the dimension of the projection space, Indicated in Layer-wise relation embedding, represents the concatenation operation, and the relation embedding is initialized to the one-hot encoding generated using one-hot encoding in the first layer of relation level encoder , Representation based on relationship type The target node The set of neighbor nodes of is the activation function, The target node for a specific relationship '

[0039] In order to obtain a more comprehensive node embedding by combining multiple relational information, The average aggregation is:

[0040]

[0041] in, Represents the total number of relationship types, represents the relationship level features, Represents a collection of edge types.

[0042] Preferably, in order to explicitly learn the semantic representation of relations, a relation embedding propagation mechanism is introduced:

[0043]

[0044] in, Indicated in Layer-wise relation embedding, is the relational feature conversion weight matrix for a specific relation, and the relation embedding is initialized as one-hot encoding in the first layer of relation level encoder. .

[0045] Preferably, in step S3.3, in order to obtain a comprehensive node representation from the node level and relationship level encoders, the node level features and the relationship level features are fused, and the specific steps are as follows:

[0046] The two-level features are concatenated and optimized through a fully connected layer. The formula is:

[0047]

[0048] in, It is The weight matrix of the layer is used for feature fusion, Represents a splicing operation, represents the relationship level features, represents node-level features, The optimized target node Representation of;

[0049] In order to improve the stability and convergence of the method, the nodes Potential state before conversion Add to optimized node Representation superior:

[0050]

[0051] in, is the output of the encoder.

[0052] Preferably, in step S3.4, the fused data is input into the global decoder to reconstruct the features, and finally the result is output through the prediction layer; the specific steps are as follows:

[0053] First, we get the query, key, and value vectors from all nodes by transforming:

[0054]

[0055] in, They are query vector, key vector and value vector respectively. is the weight matrix of the query for a specific node type, the weight matrix of the key, and the weight matrix of the value. Represents the dimension of the projection space, the first layer input of the global decoder Initialized to the output of the encoder , Represents the global encoder The input of the layer,

[0056] The calculation formula of attention value is as follows:

[0057]

[0058]

[0059] in, Represents the source node In the relationship type Target Node The attention weight, The attention weight matrix representing the global decoder specific relation type, is with dimension The associated scaling factor, Representation based on relationship type The target node The set of neighbor nodes of Represents the source node The key vector of Indicates the target node The transpose of the query vector,

[0060] The node features of a specific relationship are aggregated according to the attention weight and value vector. The formula is as follows:

[0061]

[0062] in, Represents the target node of a specific relationship Features, Represents the source node The value vector of Representation based on relationship type The target node The set of neighbor nodes of

[0063] Set all target nodes of a specific relationship The feature aggregation of is as follows:

[0064]

[0065] in, Represents the total number of relationship types, is the target node The characteristics of aggregation, Represents a set of edge types;

[0066] In order to stabilize the learning process and ensure the smoothness of the gradient flow, a normalization step and a residual connection are combined to obtain the output of the global decoder , the formula is as follows:

[0067]

[0068] in, , is the normalization function, Represents the global decoder The output of the layer, the input of the global encoder in the first layer The output of the encoder , Representation Node Aggregate characteristics.

[0069] Preferably, in order to optimize the learning process of the proposed method, the heterogeneous graph node representation learning model is optimized by combining the dual loss mechanism of reconstruction loss and prediction loss.

[0070] Reconstruction loss function:

[0071]

[0072] in, represents the output of the global decoder, is the output of the pre-conversion layer, Represents a collection of nodes;

[0073] Prediction loss function:

[0074] in, Is a node The real label, is the predicted probability, is the set of labeled nodes;

[0075] Overall loss function:

[0076]

[0077] in, is a coefficient used to balance the reconstruction loss and prediction loss.

[0078] The present invention has the following technical effects: 1. The present invention proposes a multi-level autoencoder method that does not rely on meta-path design to solve the problem that the existing patents rely on meta-path design, resulting in high computational complexity and requiring a lot of domain knowledge. Through the multi-level structure, it is possible to effectively capture the multi-level information of nodes and relationships in heterogeneous graphs while reducing the computational complexity, thereby improving the scalability and applicability of the method.

[0079] 2. This invention solves the problem that existing patents rely too much on node type embedding and ignore relationship features by combining the aggregation mechanism of node features and relationship features. The proposed method can fully capture the interactive information of nodes and relationships in heterogeneous graphs, enhance the integrity of node representation, improve the performance of the method in complex heterogeneous graph tasks, and has higher accuracy and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A flowchart of a heterogeneous graph node representation learning method based on a multi-level autoencoder of the present invention;

[0081] Figure 2 This is an architectural diagram of the heterogeneous graph representation learning model of the present invention. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described in conjunction with an implementation case. Obviously, the described implementation case is a part of the implementation case of the present invention, not all implementation cases. The components of the embodiments of the present invention described and shown here can be arranged and designed in various different configurations.

[0083] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features therein can be combined with each other; and all model training is completed using Python programming on Intel Core 13th generation 13900 processor and NVIDIA RTX 4090 graphics card.

[0085] The present invention aims at the node classification task of heterogeneous graphs, and uses the node information and relationship information in the heterogeneous graphs to learn node representation and perform training, aiming to efficiently learn the vector representation of nodes and train to predict node labels, thereby improving the scalability of heterogeneous graph training algorithms. The process of heterogeneous graph node representation learning method based on multi-level autoencoders is as follows: Figure 1 shown.

[0086] A heterogeneous graph node representation learning method based on a multi-level autoencoder comprises the following steps:

[0087] Step S1: Data collection and preprocessing, collect heterogeneous data to ensure data quality and consistency. Considering the characteristics of heterogeneous data, the data needs to be standardized, normalized or otherwise preprocessed to make it suitable for the neural network model. In this embodiment, the paper data is processed using a bag-of-word representation of its keywords. Author and subject data are processed using one-hot encoding.

[0088] Step S2: Construct heterogeneous graph data representation and treat different preprocessed data as nodes in the heterogeneous graph , in the encoder, to indicate the relationship node Source Node and the target node ,Each node corresponds to a node feature, and different relationships between data are constructed as edges;

[0089] Specifically: A heterogeneous graph is defined as a directed graph , where each node and each edge Both of them have their type mapping functions and Related, Represents a collection of nodes, represents the set of edges, is a collection of node types, is a collection of edge types. and the target node Edge , and its relationship is expressed as , There are usually multiple relationships between nodes. Given a relationship type , based on the relationship type The target node The neighbor node set of .

[0090] Step S3: In order to capture both node-level features and relationship-level features in a heterogeneous graph, a heterogeneous graph representation learning model is constructed based on a multi-level autoencoder method. A complete heterogeneous graph representation learning model requires a pre-conversion layer to pre-convert node features. At the beginning of encoding, the node features processed by the pre-conversion layer are projected into the latent space through the encoder conversion layer to generate a preliminary representation of the node, which is then input into the node-level encoder and the relationship-level encoder to encode the node-level features and the relationship-level features respectively. After feature fusion, the features are input into the global decoder to reconstruct the features. Finally, the prediction layer is used to output the final output result. The system model architecture is as follows: Figure 2 As shown;

[0091] S3.1: To ensure that nodes of different relationship types The feature dimension of each node is consistent with that of The original attributes of are projected into a unified vector space for subsequent network learning. This process is for each node type To do type-specific conversions:

[0092]

[0093] in, Representation Node The original properties of Representation Node The projected attributes are the outputs of the pre-transformation layer. Here, and Represent the dimensions of the original space and the projected space respectively. represents the weight matrix for a particular node type, is the activation function. This step ensures that all node attributes are represented in a consistent spatial form during subsequent processing.

[0094] S3.2: At the beginning of encoding, the node features are projected into the latent space through a transformation of a specific node type (encoder transformation layer). This transformation prepares the node representation for the subsequent encoding step. The specific transformation formula is as follows:

[0095]

[0096] in, Representation Node The potential state before the conversion, For Node The transformed latent state, i.e. the output of the encoder transformation layer, It is The node-type-specific weight matrix of the layer encoder. The input to the first layer of the encoder is , which is the output of the pre-conversion layer.

[0097] S3.3: In order to focus on capturing node information without considering heterogeneity, a node-level encoder is designed. The node-level encoder focuses on capturing node features, ignores the relationship information in the heterogeneous graph, and uses a symmetric normalization strategy to aggregate features from neighbor nodes to ensure that the contribution of each neighbor is fair. The specific aggregation formula is as follows:

[0098]

[0099]

[0100] in, Represents the source node The transformed latent state, i.e. the output of the encoder transformation layer, represents node-level features, For the The node-level weight matrix of the layer node-level encoder is used to transform neighbor node features. Here, Indicates the target node The neighbor set of is the activation function. It is a symmetric normalization coefficient based on the node degree, which plays a key role in maintaining the balance of influence of neighbor nodes, thereby enhancing the robustness of the feature aggregation process. Represents the source node The degree (number of connected edges), Indicates the target node The degree (number of connected edges) of .

[0101] S3.4: Relation-level encoder captures nodes in heterogeneous graphs. Characteristics of different types of relationships, based on each relationship type Perform feature transformation of specific relationships to generate representations of nodes under specific relationships. Then, by introducing the attention mechanism, calculate the source node For the target node The influence weight of the relationship is calculated, and the node features under different relationships are combined to generate the node representation at the relationship level (relationship level features);

[0102] S3.4.1: In order to accurately reflect the structure of heterogeneous graphs and distinguish different types of relations, the relation-level encoder encodes each relation type. The source node Apply transformations for specific relationship types, rather than for specific node types. Related source nodes The conversion formula is as follows:

[0103]

[0104] in, Represents the source node The transformed latent state, i.e. the output of the encoder transformation layer, is the weight matrix for a specific relationship type.

[0105] S3.4.2: To distinguish different relationships to target nodes The contribution of the paper introduces an attention mechanism that contains specific relationship information. This mechanism enables the method to better capture the importance of different relationships in affecting the representation of target nodes, thereby improving its ability to adapt to and distinguish different types of relationships. The calculation formula of the attention mechanism is as follows:

[0106]

[0107]

[0108] in, Represents the source node In the relationship type Target Node The importance weight of Indicates the target node The transformed latent state, i.e. the output of the encoder transformation layer, is the weight matrix of a specific relationship, Indicated in The embedding of the layer relationship, represents the concatenation operation, and the relation embedding is initialized as one-hot encoding in the first layer of relation level encoder , is generated using one-hot encoding, Representation based on relationship type The target node The set of neighbor nodes is the activation function, The target node for a specific relationship Representation.

[0109] S3.4.3: In order to obtain a more comprehensive node embedding by combining multiple relation information, for all target nodes of a specific relation The representation of average aggregation is performed, and the node representation after relationship aggregation is:

[0110]

[0111] in, Represents the total number of relationship types, Represents relationship level characteristics.

[0112] Furthermore, in order to explicitly learn the semantic representation of relations, a relation embedding propagation mechanism is introduced. This mechanism optimizes the relation embedding at each layer through the fully connected layer, and the formula is:

[0113]

[0114] in, Indicated in Layer-wise relation embedding, Indicated in Layer-wise relation embedding, is the relational feature conversion weight matrix for a specific relation, and the initial relation embedding is initialized as one-hot encoding in the first layer .

[0115] S3.5: In order to obtain a comprehensive node representation from the node-level encoder and the relation-level encoder, the features of these two levels are concatenated and optimized through a fully connected layer, as follows:

[0116]

[0117] in, It is The weight matrix of the layer is used for feature fusion, Represents a splicing operation, The optimized target node Representation of;

[0118] In order to improve the stability and convergence of the method, a residual connection is applied to connect the nodes Potential state before conversion Add to optimized node Representation Above, the formula is:

[0119]

[0120] in, is the final output of the multi-level encoder.

[0121] S3.6: In order to efficiently reconstruct node features and capture the interaction information between nodes and relationships, the global decoder captures the interaction information between nodes and relationships through the cross-attention mechanism to reconstruct node features. First, the features of the target node are converted into a query vector, and the source node is converted into a key and value vector. The weight matrix of the global decoder is used to calculate the attention value of the query and key vectors. Finally, the global decoder aggregates the information of neighbor nodes according to the attention value and value vector to generate the final node reconstruction representation;

[0122] S3.6.1: To compute the attention value, first obtain the query, key, and value vectors from all nodes via the following transformation:

[0123]

[0124] in, are query, key, and value vectors respectively, is the weight matrix of query, key, and value for a specific node type, and the first layer input of the global decoder Initialized to the final output of the encoder , Represents the global encoder The input of the layer.

[0125] S3.6.2: The global decoder calculates the attention value to capture the importance of neighbor nodes. The calculation formula of the attention value is as follows:

[0126]

[0127]

[0128] in, Represents the source node In the relationship type Target Node The attention weight, The attention weight matrix representing the specific relation type of the global decoder is used to distinguish different relations. According to the dimension The resulting scaling factor is used to normalize the attention calculation, Representation based on relationship type The target node The set of neighbor nodes of Represents the source node The key vector of Indicates the target node The transpose of the query vector .

[0129] S3.6.3 Aggregate node features of a specific relationship based on attention weights and value vectors, as follows:

[0130]

[0131] in, Represents the target node of a specific relationship Features, Represents the source node A vector of values ​​of .

[0132] S3.6.4 In order to obtain features containing node and relationship interaction information, all target nodes of a specific relationship are The feature aggregation of is as follows:

[0133]

[0134] in, is the target node Final features of the aggregation.

[0135] S3.6.5 In order to stabilize the learning process and ensure the smoothness of the gradient flow, the normalization step and residual connection are combined. The residual connection connects the global decoder Output of the layer Add to the aggregated node Features Finally, the output of the global decoder is , the formula is as follows:

[0136]

[0137] in, is the final output of the global decoder, is the normalization function.

[0138] Step S4: Training and optimization.

[0139] S4.1 Data partitioning: Divide the dataset into training set, validation set, and test set to ensure the model's predictive ability in future time periods.

[0140] S4.2 Model training: The model is trained using the training set and optimized by combining a dual loss mechanism of reconstruction loss and prediction loss. The reconstruction loss is used to ensure that the decoded node representation is consistent with the original node features, while the prediction loss improves the performance of the model by optimizing the accuracy of the classification task.

[0141] The reconstruction loss maintains the integrity of the original node features by minimizing the difference between the reconstructed node features and the initial node features. The specific formula is as follows:

[0142]

[0143] in, is the node feature reconstructed by the global decoder, is the output of the pre-conversion layer, Represents a collection of nodes.

[0144] In order to improve the accuracy of downstream tasks (such as node classification), a multi-layer perceptron is used to extract Inferring node labels , and optimized by cross entropy loss:

[0145]

[0146] in, Is a node The real label, is the predicted probability, is the set of labeled nodes.

[0147] The overall loss function is the weighted sum of reconstruction loss and prediction loss, and the formula is as follows:

[0148]

[0149] in, is a coefficient used to balance the reconstruction loss and prediction loss.

[0150] S4.3 Hyperparameter adjustment: Evaluate model performance on the validation set and adjust the model’s hyperparameters, such as the number of hidden units, to optimize the model’s generalization ability.

[0151] S4.4, Prediction and Evaluation: After the model training is completed, the test set is used to predict node features, and the model performance is evaluated by the F1 score indicator to ensure its performance in the node classification task.

[0152] Step S5, through the above implementation steps, a heterogeneous graph node representation learning model based on a multi-level autoencoder can be constructed, and the test data set is input into the trained heterogeneous graph node representation learning model to classify the nodes in the heterogeneous graph.

[0153] The present invention uses the real dataset ACM to verify the effectiveness of the proposed heterogeneous graph node representation learning method. The ACM dataset is a citation network from the Association for Computing Machinery (ACM). From the original dataset, a subset is selected, which contains 60 disciplines, 4025 papers and 7167 authors. The papers are divided into three categories according to the conferences in which they are published, namely data mining, databases and communications. The attributes of the paper are represented by the bag of words of its keywords, while the attributes of the author and discipline are encoded as one-hot vectors, as shown in Table 1.

[0154] Table 1 ACM dataset

[0155]

[0156] The results of the comparative test are shown in Table 2. HAN and HGT are basic heterogeneous graph node representation learning methods. The Micro-F1 and Macro-F1 of the method of the present invention are significantly better than other methods, proving that the patented method is superior to other current models.

[0157] Table 2 Comparison of method results (%)

[0158]

Claims

1. A heterogeneous graph node representation learning method based on multi-level autoencoders, characterized in that: The method comprises the following steps: Step S1, data collection and preprocessing; Step S2: construct heterogeneous graph data representation, treating the different preprocessed data as nodes in the heterogeneous graph ,node Source Node and the target node , different relationships between data are constructed as edges , the relationship between nodes is defined as relationship type , each node corresponds to a node feature; Step S3, constructing a heterogeneous graph node representation learning model based on a multi-level autoencoder method, Step S3.1, taking the heterogeneous graph data representation as the input of the model, and using the pre-conversion layer to map the node features to the same dimension, Step S3.2, before encoding begins, the output of the pre-transformation layer is projected into the latent space through the encoder transformation layer to generate a preliminary representation of the node. Step S3.2: Input the preliminary representation of the node into the node-level encoder and the relationship-level encoder to encode the node-level features and the relationship-level features. Step S3.3, feature fusion of node-level features and relationship-level features. Step S3.4, input the fused data into the global decoder with a cross attention mechanism to reconstruct the features, and finally output the results through the prediction layer; Step S4, training and optimizing the constructed heterogeneous graph node representation learning model; Step S5, inputting the data to be classified into the trained heterogeneous graph node representation learning model to obtain the classification result.

2. The heterogeneous graph node representation learning method based on multi-level autoencoders according to claim 1 is characterized in that: Step S2 is as follows: define the heterogeneous graph as a directed graph , where each node and each edge Both its type mapping function and Related, Represents a collection of nodes, represents the set of edges, Represents a collection of node types, Represents a collection of edge types; connects source nodes and the target node Edge The relationship is expressed as , ; The relationship between nodes is defined as the relationship type , based on the relationship type The target node The set of neighbor nodes is defined as .

3. The heterogeneous graph node representation learning method based on multi-level autoencoder according to claim 1 is characterized in that: The formula for mapping node features to the same dimension using the pre-conversion layer in step S3.1 is: ； in, Representation Node The original properties of Representation Node The projected properties, i.e. the output of the pre-transformation layer, and Represent the dimensions of the original space and the projection space respectively, represents the weight matrix for a particular node type, is the activation function.

4. The heterogeneous graph node representation learning method based on multi-level autoencoder according to claim 1 is characterized in that: In step S3.2, before encoding begins, the output of the pre-transformation layer is projected into the latent space through the encoder transformation layer to generate a preliminary representation of the node: ； in, Representation Node The latent state before conversion. The input of the first layer of the encoder is the output of the pre-conversion layer. , Representation Node The potential state after conversion, Indicates The node-type-specific weight matrices of the layer encoder, Represents the dimension of the projection space.

5. The heterogeneous graph node representation learning method based on multi-level autoencoder according to claim 1 is characterized in that: In step S3.3, the preliminary representation of the node is input into the node-level encoder to encode the node-level features, as follows: ； ； in, Represents the source node The potential state after conversion, represents node-level features, For the The node-level weight matrix of the layer node-level encoder, Indicates the target node The neighbor set of is the activation function, is a symmetric normalization coefficient based on node degree, Represents the source node The degree, Indicates the target node The degree.

6. The heterogeneous graph node representation learning method based on multi-level autoencoder according to claim 1 is characterized in that: In step S3.3, the preliminary representation of the node is input into the relation-level encoder to encode the relation-level features, as follows: The relation-level encoder encodes each relation type The source node To perform conversions on a specific relation type: ； in, for The converted representation is Represents the source node The potential state after conversion, is the weight matrix for a specific relationship type; Introduce an attention mechanism containing specific relationship information to distinguish different relationship types for target nodes Contributions: ； ； in, Represents the source node In the relationship type Target Node The importance weight of Indicates the target node The potential state after conversion, is the weight matrix of a specific relationship, represents the dimension of the projection space, Indicated in Layer-wise relation embedding, represents the concatenation operation, and the relation embedding is initialized to the one-hot encoding generated using one-hot encoding in the first layer of relation level encoder , Representation based on relationship type The target node The set of neighbor nodes of is the activation function, The target node for a specific relationship ' For all target nodes of a specific relationship The average aggregation is: ； in, Represents the total number of relationship types, represents the relationship level features, Represents a collection of edge types.

7. The heterogeneous graph node representation learning method based on multi-level autoencoders according to claim 6 is characterized in that: In order to explicitly learn the semantic representation of relations, a relation embedding propagation mechanism is introduced: ； in, Indicated in Layer-wise relation embedding, is the relational feature conversion weight matrix for a specific relation, and the relation embedding is initialized as one-hot encoding in the first layer of relation level encoder. .

8. The heterogeneous graph node representation learning method based on multi-level autoencoders according to claim 1 is characterized in that: Step S3.3, feature fusion of node-level features and relationship-level features. The specific steps are as follows: The two-level features are concatenated and optimized through a fully connected layer. The formula is: ； in, It is The weight matrix of the layer is used for feature fusion, Represents a splicing operation, represents the relationship level features, represents node-level features, The optimized target node ' The node Potential state before conversion Add to optimized node Representation superior: ； in, is the output of the encoder.

9. The heterogeneous graph node representation learning method based on multi-level autoencoders according to claim 1 is characterized in that: Step S3.4, input the fused data into the global decoder to reconstruct the features, and finally output the results through the prediction layer; the specific steps are as follows: First, we get the query, key, and value vectors from all nodes by transforming: ； in, They are query vector, key vector and value vector respectively. is the weight matrix of the query for a specific node type, the weight matrix of the key, and the weight matrix of the value. Represents the dimension of the projection space, the first layer input of the global decoder Initialized to the output of the encoder , Represents the global encoder The input of the layer, The calculation formula of attention value is as follows: ； ； in, Represents the source node In the relationship type Target Node The attention weight, The attention weight matrix representing the global decoder specific relation type, Is with dimension The associated scaling factor, Representation based on relationship type The target node The set of neighbor nodes of Represents the source node The key vector of Indicates the target node The transpose of the query vector, The node features of a specific relationship are aggregated according to the attention weight and value vector. The formula is as follows: ； in, Represents the target node of a specific relationship Features, Represents the source node The value vector of Representation based on relationship type The target node The set of neighbor nodes of Set all target nodes of a specific relationship The feature aggregation of is as follows: ； in, Represents the total number of relationship types, is the target node The characteristics of aggregation, Represents a set of edge types; Combined with the normalization step and residual connection, we get the output of the global decoder , the formula is as follows: ； in, , is the normalization function, Represents the global decoder The output of the layer, the input of the global encoder in the first layer The output of the encoder , Representation Node Aggregate characteristics.

10. The heterogeneous graph node representation learning method based on multi-level autoencoder according to claim 1, characterized in that: The heterogeneous graph node representation learning model is optimized by combining the dual loss mechanism of reconstruction loss and prediction loss. Reconstruction loss function: ； in, represents the output of the global decoder, is the output of the pre-conversion layer, Represents a collection of nodes; Prediction loss function: ; in, Is a node The real label, is the predicted probability, is the set of labeled nodes; Overall loss function: ； in, is a coefficient used to balance the reconstruction loss and prediction loss.

Images