A multi-path convolution residual width network heterogeneous graph representation learning method and system

Abstract

This invention provides a method and system for learning heterogeneous graph representations using residual width networks with multi-path convolution, belonging to the field of heterogeneous graph representation learning technology. The method first divides the original heterogeneous graph into multiple semantic subgraphs based on predefined meta-paths. Multi-layer residual multi-head graph attention convolutions are then performed on each subgraph to extract multi-view semantic features and alleviate deep propagation feature degradation. The multi-path features are then concatenated and fused, directly used as input to a width network to construct enhanced nodes. Ridge regression is used to solve for the output weights, completing tasks such as node classification, clustering, and link prediction. This invention fully utilizes the multi-semantic information of heterogeneous graphs, improves the stability of deep feature propagation, reduces training costs, and balances representation quality and solution efficiency. It is widely applicable to complex heterogeneous graph scenarios such as academic networks and knowledge graphs, and has strong versatility and practical value.

Description

Technical Field

[0001] This invention relates to the field of data technology, and specifically to a method and system for learning heterogeneous graph representations in residual width networks using multi-path convolution. Background Technology

[0002] With the rapid development of artificial intelligence and big data technologies, graph data, as a typical data structure for depicting complex relationships between entities, has been widely applied in key areas such as academic networks, knowledge graphs, social networks, and recommendation systems. Compared to traditional data such as independent vectors and tables, graph data combines the attributes of nodes themselves with the topological structure information between nodes, making it naturally adaptable to the interactive modeling needs of multi-type entities and multi-dimensional relationships in real-world scenarios, and thus becoming the mainstream representation form for complex relational data.

[0003] In real-world applications, graph data is often not isomorphic graphs containing only a single node type and edge type, but rather heterogeneous graphs composed of multiple node types and edge types. For example, academic networks contain nodes such as authors, papers, conferences, and terms, exhibiting differentiated relationships such as "author-writing-paper," "paper-publication-conference," and "paper-related-terms." Knowledge graphs contain nodes such as entities, attributes, and concepts, as well as multi-level semantic relationships. Heterogeneous graphs, through the differences in node and edge types, carry far richer semantic information than isomorphic graphs, but this also brings more complex semantic modeling challenges to graph representation learning—if the semantic sources and value differences of different relationships cannot be effectively distinguished, the model is prone to incorporating irrelevant information, reducing the quality and discriminative ability of node representations.

[0004] The core objective of heterogeneous graph representation learning is to map nodes into low-dimensional dense vectors while preserving topological structure and semantic information, supporting downstream tasks such as node classification, node clustering, link prediction, and personalized recommendation. This is currently a core research direction in the field of graph learning. For the semantic modeling needs of heterogeneous graphs, the mainstream existing approach is to aggregate neighborhood information based on graph neural networks (GNNs), graph convolutional networks (GCNs), and graph attention networks (GATs), and then complete downstream applications through task modules. To improve semantic discriminative capabilities, the academic community has introduced meta-path technology—meta-paths are predefined composite semantic relationship patterns that can clearly characterize the association logic of different types of nodes in a heterogeneous graph. For example, "author-paper-author" represents a collaboration relationship, and "author-paper-conference" represents a publication relationship. Constructing semantic subgraphs based on meta-paths can reduce the direct mixing of different semantic information, becoming a mainstream technical solution for heterogeneous graph representation learning.

[0005] The current technical solution involves the following overall process: extracting several semantic subgraphs from the original heterogeneous graph based on predefined meta-paths → performing graph convolution / graph attention feature extraction on each subgraph → concatenating / weighted fusion of multi-path representations → outputting the result through a fully connected layer or a conventional task module. While this solution can utilize multiple semantic information, it suffers from four core drawbacks in practical applications: 1. Insufficient utilization of meta-path semantics; different meta-paths correspond to independent semantic perspectives, and there are significant differences in node neighborhood distribution and key neighbor contributions. Existing methods only use meta-paths as the basis for relation partitioning, and do not fully characterize semantic differences in modeling. Different relation information is still mixed and processed, and the semantic expressive power of node representation is limited.

[0006] 2. The fusion methods for multi-path features are rudimentary; existing fusion methods mostly adopt simple strategies such as direct splicing, averaging, and fixed weight summation, without considering the differences in importance of different meta-paths to different nodes. The effective information of key paths is easily weakened, and the loss of discriminative ability is particularly prominent in heterogeneous graphs with complex relationships and diverse semantic sources.

[0007] 3. Deep propagation is prone to feature degradation; shallow models can only capture neighbor information and it is difficult to mine higher-order semantic relationships; increasing the number of convolutional / attention layers to expand the receptive field will cause problems such as node representation convergence, original feature decay, and training instability, and cannot take into account both the utilization of higher-order structures and the preservation of effective shallow features.

[0008] 4. Inefficient connection between representation learning and task modules: After completing graph representation learning, existing methods directly connect to traditional task modules such as conventional fully connected classifiers. The complex structure and high-dimensional features at the front end will significantly increase the training cost at the back end, resulting in low overall training efficiency. At the same time, although wide networks have the advantage of training efficiency, traditional wide networks rely on random mapping to generate input features, which cannot adapt to the input requirements of semantic information of heterogeneous graph structures and is difficult to play an efficient feature enhancement role.

[0009] In summary, existing heterogeneous graph representation learning methods have significant shortcomings in key aspects such as metapath semantic mining, multi-feature fusion, deep propagation stability, efficient task module connection, and wide network adaptation. They cannot simultaneously meet the comprehensive requirements of semantic modeling of complex heterogeneous graphs, stable propagation of deep features, and efficient task solving, thus restricting the representation quality and application efficiency of the models in real-world scenarios. Summary of the Invention

[0010] The technical problem to be solved by this invention is to provide a method and system for learning heterogeneous graph representations using residual width networks with multi-path convolution. This aims to address the problems existing in current heterogeneous graph representation learning methods, such as insufficient utilization of multi-path semantics, easy feature degradation during deep propagation, relatively simple feature fusion methods under different semantic paths, and low training efficiency of subsequent task modules. To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A residual width network heterogeneous graph representation learning method using multi-path convolution includes the following steps: Step 1: Input heterogeneous graph data and construct corresponding semantic subgraphs based on multiple predefined meta-paths; Step 2: Perform multi-layer residual graph attention convolution on each meta-path semantic subgraph to extract the feature representation of the node in the corresponding semantic space; Step 3: Fuse the node features obtained from different meta-paths to obtain multi-path fusion features; Step 4: Use the multi-path fusion features as input to the wide network to construct enhanced nodes and form an overall feature matrix; Step 5: Based on the overall feature matrix, use ridge regression closed-form solution to solve the output weights and generate the output results of the corresponding downstream tasks.

[0011] The heterogeneous graph input in Step 1 above is represented as follows: ; in, Represents a set of nodes. Denotes the set of edges. Represents a set of node types. Represents a set of edge types; The metapath set is: ; in, This represents the number of metapaths, and each metapath... Used to describe a complex semantic relationship between nodes in a heterogeneous graph; For each predefined metapath Construct its corresponding isoproton graph: ; in, This represents the set of nodes that satisfy the semantic constraints of the metapath. This represents the set of edges that satisfy the meta-path connection relationship.

[0012] The residual map attention convolution in Step 2 above includes: The node features are linearly mapped to obtain subspace features; the attention scores of the node and its neighbors are calculated and normalized to obtain attention coefficients; the neighbor features are weighted and aggregated based on the attention coefficients to obtain multi-head attention output; a residual connection is added after each convolution layer to weightedly fuse the current layer output with the previous layer input.

[0013] The above residual connections satisfy: ; in, Indicates the number of convolutional layers. Represents a node In metapath The second chapter Intermediate features obtained from multi-head graph attention convolution. It is a learnable scaling factor used to balance the new features learned in the current layer with the features retained in the previous layer; The attention coefficient is asymmetric to accommodate the directionality and diversity of node relationships in heterogeneous graphs.

[0014] In Step 3 above, the multi-path feature fusion adopts a feature dimension concatenation method, assuming a total of A single path, then the node The fusion features are represented as follows: ; in, Represents a node In the Passing through the element path Features obtained after attention convolution of layer residual maps This represents the final node embedding representation.

[0015] In Step 4 above, the width network eliminates the traditional random feature mapping layer and directly uses multi-path fusion features as network input; The enhanced nodes of the width network are constructed as follows: ; in, Indicates the enhanced node group number, The multi-path fusion characteristics of each node The input feature matrix is ​​formed by stacking rows. The weight matrix is ​​generated randomly. For bias terms, It is a non-linear activation function.

[0016] In Step 4 above, all augmented nodes are concatenated column-wise to obtain the augmented node matrix. ; Input feature moments Concatenate matrix with enhanced nodes By concatenating the dimensions, we obtain the overall feature matrix. .

[0017] Step 5 above, which uses ridge regression to solve for the output weights in a closed-form solution, specifically includes: After obtaining the overall feature matrix Then, the weight matrix is ​​output. Generate task-related outputs; output weight matrix Solving using ridge regression: ; in, This is the regularization coefficient, used to suppress overfitting; The supervision matrix corresponding to the input feature matrix X. It is an identity matrix.

[0018] The residual graph attention convolution mentioned above employs a multi-head attention mechanism to model neighborhood dependencies in parallel across multiple subspaces, thereby capturing multi-semantic information of heterogeneous graphs with fine granularity.

[0019] Systems using the residual width network heterogeneous graph representation learning method described above with multi-path convolution include: The subgraph construction module is used to decompose a heterogeneous graph into multiple semantic subgraphs based on predefined meta-paths; The residual convolution module is used to perform multi-layer residual graph attention convolutions on each semantic subgraph; The feature fusion module is used to fuse node features from different meta-paths; The width enhancement module is used to construct enhanced nodes and generate the overall feature matrix; The output solver module is used to solve for the output weights using ridge regression closed-form and generate the task output.

[0020] The present invention discloses a heterogeneous graph representation learning method and system based on residual width networks using multi-path convolution. This method extracts node features from multiple semantic perspectives by performing residual graph attention convolution on semantic subgraphs induced by different meta-paths. The multi-path features are then fused, and a width network is introduced to complete feature enhancement and efficient solution, thereby improving the quality of heterogeneous graph representation learning results and providing more effective representation support for downstream tasks related to heterogeneous graph representation learning.

[0021] This invention achieves comprehensive improvements in semantic representation, feature propagation, model efficiency, training stability, and generality of heterogeneous graph representation learning through an overall architecture that combines multi-path semantic modeling, residual graph attention convolution, efficient wide-network enhancement, and closed-form solving. Specific technical effects are as follows: 1. By fully exploring and utilizing the semantics of heterogeneous graph multi-path relationships, this invention significantly improves the semantic completeness of node representations. Semantic subgraphs are independently constructed according to different meta-paths, and feature extraction and representation learning are performed separately within each subgraph. This allows the model to characterize node relationships from multiple semantic perspectives, avoiding direct mixing and interference of different relationship information. The resulting node representations simultaneously contain node attributes, topological structure, and multi-level composite semantics, fully reflecting the complex relationship logic of heterogeneous graphs and greatly enhancing the semantic expressive power of the representation.

[0022] 2. Effectively alleviates feature degradation and oversmoothing in deep networks, balancing high-order semantic mining with shallow feature preservation; residual connections with learnable scaling factors are introduced after the attention convolutions in each layer, enabling deep networks to aggregate high-order neighborhood information while retaining effective shallow semantic features, significantly improving problems such as gradient vanishing, node representation convergence, and feature decay. The model can safely increase the number of network layers to expand the receptive field, stably mining high-order structures and semantic relationships in heterogeneous graphs.

[0023] 3. Multi-head attention enables fine-grained neighborhood weighting and adaptively distinguishes the importance of neighbors. The multi-head graph attention mechanism models node dependencies in multiple subspaces in parallel. By adaptively allocating neighbor weights through asymmetric attention coefficients, it accurately captures the directionality and differences of node relationships in heterogeneous graphs. Compared with single-head attention and equal aggregation, neighborhood feature extraction is more accurate and discriminative.

[0024] 4. The fusion of multi-path features is more reasonable and effectively preserves key semantic information. After learning in an independent semantic space, the features of each multi-path are uniformly spliced ​​and fused. The complementary information of different paths is fully preserved, avoiding the problem of weakening key information caused by simple splicing, averaging or fixed weighting. The fused features are complete in dimension and rich in semantics, which can provide high-quality input for downstream tasks.

[0025] 5. Combining wide-width networks with graph representation depth significantly improves training and inference efficiency; abandoning the random feature mapping layer of traditional wide-width networks, it directly uses high-quality graph convolutional embeddings as input, and expands the feature space with enhanced nodes; the output weights are solved using ridge regression closed-form, without the need for a large number of backpropagation iterations, which significantly reduces training overhead, improves convergence speed and overall operating efficiency while ensuring representation effect.

[0026] 6. The model training is more stable and adaptable to complex heterogeneous graph scenarios; the residual structure ensures stable deep propagation, and the wide network uses analytical solutions to reduce optimization difficulty. The overall model can still be trained stably in large-scale heterogeneous graphs with many node types, complex relationships, and rich semantics. It is less prone to oscillations, overfitting, and representation collapse, making it more practical.

[0027] 7. It has strong versatility and scalability, and can be quickly adapted to various downstream tasks of heterogeneous graphs; with the meta-path as the unified semantic modeling unit, it can be adapted to different scenarios such as academic networks, knowledge graphs, social networks, and recommendation systems simply by changing the meta-path definition. It directly supports mainstream tasks such as node classification, node clustering, link prediction, and personalized recommendation. The framework is flexible, easy to implement and expand.

[0028] 8. Overall, it achieves a balance between representation quality and computational efficiency; the front end focuses on high-quality heterogeneous graph representation extraction, while the back end focuses on efficient feature enhancement and task solving. It avoids blindly increasing network depth and complexity, controlling computational costs while improving representation accuracy, making it suitable for use in resource-constrained environments and real-time tasks. Attached Figure Description

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart illustrating the overall steps of the residual width network modeling method based on multi-path convolution of the present invention. Figure 2 This is a diagram showing the input-output relationship of each step in the residual width network modeling method based on multi-path convolution of the present invention; Figure 3 This is a schematic diagram illustrating the impact of changes in the number of feature enhancement groups on model performance in this invention; Figure 4 This is a schematic diagram illustrating the impact of feature enhancement dimension changes on model performance in this invention. Detailed Implementation

[0030] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0031] The technical solution of this invention mainly includes the following parts: heterogeneous graph input and meta-path subgraph construction, residual graph attention convolution on the meta-path subgraph, multi-path feature fusion, wide network input construction, enhanced node generation, and output weight solution. These parts together constitute a complete heterogeneous graph modeling and analysis process. Specifically, the meta-path subgraph construction is used to explicitly distinguish different semantic relationships; the residual graph attention convolution is used to extract high-order neighborhood features and alleviate feature degradation problems in deep propagation; and the wide network is used to achieve feature enhancement and task-related output solution with lower training costs.

[0032] Step 1: Heterogeneous graph input and metapath subgraph construction Let the input heterogeneous graph be represented as: ; in, Represents a set of nodes. Denotes the set of edges. Represents a set of node types. Represents a set of edge types.

[0033] In heterogeneous graphs, different types of nodes and edges correspond to different semantic relationships. To distinguish and model these relationships, this invention predefines a set of meta-paths: ; in, This represents the number of metapaths, and each metapath... This is used to describe a complex semantic relationship between nodes in a heterogeneous graph. For example, in academic networks, "author-paper-author" represents a collaborative relationship between authors, while "author-paper-conference-paper-author" represents an association between authors at the conference level.

[0034] For each predefined metapath Construct its corresponding isoproton graph: ; in, This represents the set of nodes that satisfy the semantic constraints of the metapath. This represents the set of edges that satisfy the meta-path connectivity relationship. Through the above processing, the original heterogeneous graph can be decomposed into multiple meta-path subgraphs with different semantic information, thus providing a basis for extracting node features in different semantic spaces.

[0035] In this step, the original heterogeneous graph is first divided into multiple metapath subgraphs with different semantics according to the predefined metapaths, so that the subsequent feature extraction process can be carried out in different semantic spaces, reducing the interference caused by the direct mixing of different relational information.

[0036] Step 2: Residual graph attention convolution on the metapath subgraph On the subgraph corresponding to each metapath, multiple layers of residual graph attention modules are stacked to learn the local features and high-order neighborhood information of the node under the semantics of the metapath.

[0037] Let's assume the metapath On the corresponding subgraph, the first Layer, First The node feature mapping process for each attention head is as follows: ; in, For the first Layer The shared weight matrix corresponding to each attention head. Represents a node In metapath The feature representation of the next higher layer, This represents the node features after mapping to the corresponding subspace.

[0038] Then, for the nodes and its neighboring nodes Calculate attention score: ; in, This is a learnable attention parameter vector. This represents a vector concatenation operation. This represents the activation function.

[0039] The attention score is normalized to obtain the node. For nodes Attention coefficient: ; in, Represents a node The set of neighboring nodes in the corresponding metapath subgraph, containing nodes After the above normalization process, the importance of different neighboring nodes to the target node can be compared. Since different nodes have different neighborhood structures, the normalized attention coefficients are usually asymmetric, thus reflecting the directionality and differences in node relationships in heterogeneous graphs.

[0040] After obtaining the attention coefficients, the features of neighboring nodes are weighted and aggregated to obtain the first... Intermediate output under individual attention: ; in, The ReLU function is used as the activation function in the implementation of this invention.

[0041] Will The outputs of each attention head are concatenated according to the feature dimension to obtain the multi-head attention output of this layer: ; In this step, neighborhood dependencies are modeled in multiple subspaces using a multi-head graph attention mechanism, and residual connections are combined to alleviate gradient vanishing and feature degradation problems in deep propagation, thereby improving the representation ability of nodes under the meta-path semantics.

[0042] Step 3: Residual Link Construction To mitigate the vanishing gradient and oversmoothing features that arise with increasing convolutional layers, a residual connection is introduced after each graph attention convolution to fuse the current layer's output with the previous layer's input, resulting in: ; in, is a learnable scaling factor used to balance the new features learned in the current layer with the features retained in the previous layer.

[0043] By introducing this residual connection, the current layer can retain effective semantic information from shallow features while learning higher-order neighborhood information, and improve the gradient propagation path in deep networks, thereby enhancing training stability. The improvement lies in the fact that instead of simply relying on layer-by-layer neighborhood propagation to update node representations, it preserves the original shallow features during deep propagation to avoid representation degradation.

[0044] Repeat the graph attention convolution and residual connection operations described above on each metapath subgraph. Next, the node is in the metapath Next The characteristics of a layer can be represented as: ; After passing through multiple residual graph attention convolutions, the feature representation of a node under the semantics of each meta-path can be obtained.

[0045] Step 4: Multi-path feature splicing and fusion After performing residual graph attention convolutions on all meta-path subgraphs, the node features learned from different meta-paths are concatenated and fused to form the final embedded representation of the node. Let there be a total of... A single path, then the node The fusion features are represented as follows: ; in, Represents a node In the Passing through the element path Features obtained after attention convolution of layer residual maps This represents the final node embedding representation.

[0046] By fusing feature representations from different meta-paths, information learned from multiple semantic perspectives can be uniformly incorporated into the final node representation, thereby improving the semantic integrity and expressive power of the representation results.

[0047] Unlike traditional wide networks that rely directly on random mapping weights to generate feature mapping layers, this invention uses the node features obtained through multivariate path residual convolution as the input features of the wide network, thereby improving the quality and expressive power of the input data and reducing the inefficient feature problem caused by random mapping.

[0048] Step 5: Width network input construction Let the training sample set be: ; in, For nodes The fusion feature representation, For nodes The corresponding task tags or supervision information.

[0049] The features of all nodes in the training sample set are stacked row-wise to form the input matrix of the wide network: ; in, Indicates the number of training samples. This represents the feature dimension of each node.

[0050] The corresponding supervision matrix is ​​represented as: ; in, This represents the output dimension, and its value can be determined based on the specific downstream task.

[0051] This step clarifies the input and output formats of the wide network, enabling those skilled in the art to directly construct training and supervision matrices based on the fused node representations.

[0052] Step 6: Building Width Network Enhancement Nodes In the wide network, instead of setting up a separate random feature mapping layer in the traditional sense, the fused features obtained by the aforementioned residual map attention convolution are directly used as input features, and enhancement nodes are constructed on this basis.

[0053] No. Group-enhanced nodes can be represented as: ; in, Indicates the enhanced node group number, The multi-path fusion characteristics of each node The input feature matrix is ​​formed by stacking rows. The weight matrix is ​​generated randomly. For bias terms, As a non-linear activation function, the ReLU function can be used in the implementation of this invention. This indicates the number of enhanced node groups.

[0054] Concatenate all augmented nodes column-wise to obtain the augmented node matrix: ; Then, the input feature matrix With enhanced node matrix The features are concatenated to form the overall input feature matrix of the wide network. ; The feature space can be further expanded by enhancing nodes in the wide network, enabling the fused multi-path representation to gain stronger expressive power while maintaining the original semantic structure information.

[0055] Step 7: Output weight calculation and task output generation After obtaining the overall feature matrix Then, the weight matrix is ​​output. Generate task-related outputs; network output is as follows: ; The loss function for a wide network is defined as follows: ; in, This is the regularization coefficient, used to suppress overfitting.

[0056] Output weight matrix The solution is obtained using ridge regression, and its expression is as follows: ; The output weight matrix can be directly obtained using the above method, yielding the output result corresponding to the specific task. Unlike traditional deep task modules that solve parameters iteratively through multiple rounds of backpropagation, this invention utilizes the structural characteristics of wide networks to solve the output weights through feature enhancement and ridge regression in a closed-form manner, thereby improving training efficiency while ensuring representation utilization.

[0057] By linking the front-end representation learning results with the subsequent output solving process, the efficient utilization of heterogeneous graph representation learning results can be achieved.

[0058] Example 1: This embodiment describes the specific implementation process of the method of the present invention on a typical heterogeneous graph dataset.

[0059] In this embodiment, the input heterogeneous graph data is processed according to the aforementioned technical solution. First, the heterogeneous graph is semantically partitioned into metapaths to construct corresponding metapath subgraphs. Then, residual multi-head graph attention convolution is performed on each metapath subgraph to obtain the feature representations of nodes in different semantic spaces. Subsequently, the node features under each metapath are fused, and the fusion result is input into a wide network to complete the construction of enhanced nodes and the solution of output weights. Finally, the node representation or the corresponding downstream task output result is obtained.

[0060] In this embodiment, the input heterogeneous graph can be represented as:

[0061] in, Represents a set of nodes. Denotes the set of edges. Represents a set of node types. Represents a set of edge types.

[0062] To address the diverse semantic associations formed by different types of nodes and relationships in heterogeneous graphs, this embodiment pre-defines a set of meta-paths:

[0063] in, Indicates the number of metapaths. Indicates the first Path of element.

[0064] In this embodiment, the DBLP dataset, Freebase dataset, and ACM dataset can be selected as the heterogeneous graph data sources. Different meta-path sets are pre-defined to account for the differences in node type composition and relationship types among the different datasets, and the corresponding target node types are determined.

[0065] The DBLP dataset is a heterogeneous citation network dataset, including four types of nodes: papers, conferences, authors, and terms. In this dataset, author nodes can be used as target nodes for representation learning and related tasks. For this dataset, the following meta-paths can be pre-defined: Author-Paper-Author (APA); Author—Paper—Conference—Paper—Author (APCPA); Author—Paper—Terminology—Paper—Author (APTPA).

[0066] The Freebase dataset is a heterogeneous movie relation dataset, including four types of nodes: movie, actor, director, and author. In this dataset, movie nodes can be used as target nodes for representation learning and related tasks. For this dataset, the following meta-paths can be pre-defined: Film—Director—Film (MDM); Movie—Actor—Movie (MAM); Film—Auteur—Film (MWM).

[0067] The ACM dataset is an academic network heterogeneous graph dataset, including three types of nodes: papers, authors, and research topics. In this dataset, paper nodes can be used as target nodes for representation learning and related tasks. For this dataset, the following meta-paths can be pre-defined: Paper—Author—Paper (PAP); Paper—Topic—Paper (PSP).

[0068] After setting up the metapath set, construct the corresponding metapath subgraph from the original heterogeneous graph based on each metapath. Each metapath subgraph can be represented as follows:

[0069] in, Indicates that the first The set of nodes constrained by the semantics of a single path. Indicates that the first A set of edges representing the connections between metapath subgraphs. This method allows each metapath subgraph to retain node connection information under a composite semantic relationship.

[0070] Subsequently, node features are extracted layer by layer from each metapath subgraph using the aforementioned residual multi-head graph attention convolution method. For the metapath... Next node In the The feature update process of a layer can be represented as:

[0071] in, Represents a node In metapath Next The input features of the layer Indicates the first Layer output features, This indicates the attention convolution operation for multi-head graphs. This represents the residual scaling factor. By introducing residual connections during convolution, shallow semantic features can be preserved while extracting high-order neighborhood information, and the feature degradation and oversmoothing problems that may occur during deep propagation can be improved.

[0072] After obtaining the node feature representations under each meta-path, the node features corresponding to each meta-path are concatenated and fused according to the feature dimension to obtain the comprehensive embedding representation of the target node:

[0073] in, This indicates the number of layers in the residual multi-head graph attention convolution. Represents a node The comprehensive embedding representation contains both structural and semantic information from multiple meta-path semantic spaces, thus possessing stronger expressive power compared to node representations obtained under a single meta-path.

[0074] In this embodiment, the comprehensive embedding representation is directly used as the input feature of the wide network, instead of relying on random mapping to generate the feature mapping layer output as in traditional wide networks. This approach allows the high-quality node representations learned in the graph convolution stage to be directly passed to subsequent task processing stages, thereby improving the expressive power and effectiveness of the wide network's input features.

[0075] Furthermore, let the training sample set be:

[0076] in, Represents a node The fused feature vector, Represents a node The corresponding monitoring information or task objectives.

[0077] Arrange the fused features of all training samples row-wise to form the input feature matrix of the wide network:

[0078] in, Indicates the number of training samples. This represents the feature dimension of the comprehensive embedding representation.

[0079] In wide networks, based on the input feature matrix Constructing an enhanced node matrix The input feature matrix is ​​then concatenated with the augmented node matrix to form the overall feature matrix.

[0080] Based on the overall feature matrix, the output of the width network can be expressed as:

[0081] in, This represents the output weight matrix. This indicates the task output result.

[0082] To solve for the output weight matrix, this embodiment uses ridge regression, and its expression is as follows:

[0083] in, The regularization coefficient is . It is the identity matrix. This represents the task objective matrix. Compared to traditional deep task modules that require multiple rounds of backpropagation to continuously update the output layer parameters, this embodiment uses wide network enhancement and closed-form solving of output weights to complete the task solution with lower training complexity.

[0084] Depending on the specific downstream task requirements, the output results obtained in this embodiment can be used for tasks such as node classification, node clustering, link prediction, and recommendation. For example, in the node classification task, it can be based on... The response values ​​corresponding to each category determine the category to which a node belongs; in other application scenarios, the obtained nodes can also be embedded for further analysis and prediction tasks.

[0085] In summary, this embodiment uses a typical heterogeneous graph dataset as an example to illustrate the implementation process of the method of the present invention in practical applications. By combining meta-path semantic modeling, residual graph attention convolution feature extraction, multi-path feature fusion, and wide network output solving, effective modeling and efficient utilization of heterogeneous graph representation learning results can be achieved, and it is applicable to various downstream task scenarios of heterogeneous graphs.

[0086] Example 2: This embodiment illustrates the application of the method of the present invention on the DBLP dataset.

[0087] Step 1: Input DBLP heterogeneous map data The DBLP dataset consists of four types of nodes: papers, conferences, authors, and terms. In this embodiment, author node classification is used as the validation method, with authors divided into four categories. The data subset used in the experiment contains 14,328 papers, 4,057 authors, 20 conferences, and 8,789 term keywords. This dataset is a typical heterogeneous citation network dataset, suitable for validating the heterogeneous graph representation learning ability of the method of this invention in academic network scenarios.

[0088] Step 2: Define metapath Based on the semantic relationships between nodes in the DBLP dataset, the following meta-paths are selected: Author-Paper-Author (APA); Author—Paper—Conference—Paper—Author (APCPA); Author—Paper—Terminology—Paper—Author (APTPA).

[0089] Among them, APA reflects the collaboration relationship between authors, APCPA reflects the association relationship between authors at the conference publication level, and APTPA reflects the association relationship between authors at the research terminology or research topic level. By setting the above three types of meta-paths, semantic modeling of author nodes can be performed from three perspectives: collaboration, conference, and research content.

[0090] Step 3: Construct the meta-path subgraph and extract features Based on predefined metapaths APA, APCPA, and APTPA, three metapath subgraphs are constructed from the original heterogeneous graph. Then, residual multi-head graph attention convolutional modules are stacked on each metapath subgraph to aggregate the neighborhood information of the author node. In this way, the structural and semantic representations of the author node in different semantic spaces can be learned.

[0091] Step 4: Perform feature fusion and wide network enhancement After obtaining the author node features from each metapath subgraph, these features are concatenated to obtain a comprehensive embedding representation of the author nodes. This comprehensive embedding representation is then input into a wide network to construct enhanced nodes and form an overall feature matrix. Subsequently, ridge regression is used to solve for the output weight matrix, thereby generating the corresponding task output results. In this embodiment, the author node classification results are used as the verification method.

[0092] Step 5: Generate task-related output The experimental results of the method of this invention on the DBLP dataset are as follows, with training set proportions of 20%, 40%, and 60% respectively: The Macro-F1 scores were 93.57, 93.87, and 93.91, respectively. The Micro-F1 scores were 93.87, 94.01, and 94.33, respectively.

[0093] Compared with the baseline model, the method of this invention outperforms the compared model in the validation results on the DBLP dataset, indicating that the joint design of meta-path subgraph modeling, residual multi-head graph attention convolution and width network output solution mechanism can effectively improve the performance of author node classification in the validation scenario.

[0094] Example 3 This embodiment illustrates the application of the method of the present invention on the Freebase dataset.

[0095] Step 1: Input Freebase heterogeneous graph data The Freebase dataset consists of four types of nodes: movies, actors, directors, and authors. In this embodiment, the movie node classification task is used as the validation method, with movies divided into three categories. In the experiment, the dataset contained 3492 movies, 33401 actors, 2502 directors, and 4459 authors. Due to the large number of relationship types and the more complex semantic relationships in this dataset, it is suitable for validating the representation learning ability of the method of this invention in complex heterogeneous relationship scenarios.

[0096] Step 2: Define metapath Based on the semantic relationships in the Freebase dataset, the following meta-paths are selected: Film—Director—Film (MDM); Movie—Actor—Movie (MAM); Film—Auteur—Film (MWM).

[0097] Here, MDM represents the director relationship between films, MAM represents the actor relationship between films, and MWM represents the author or screenwriter relationship between films. Through the above meta-path settings, the relationships between film nodes can be modeled from different levels of creative and participation relationships.

[0098] Step 3: Construct the metapath subgraph and perform convolution. Based on predefined metapaths MDM, MAM, and MWM, three metapath subgraphs are constructed from the original heterogeneous graph. Residual multi-head graph attention convolutions are then performed on each metapath subgraph to aggregate neighborhood information related to movie nodes. This allows for the extraction of feature representations of movie nodes in three semantic spaces: director relationships, actor relationships, and author relationships.

[0099] Step 4: Perform feature fusion and wide network enhancement The movie node features obtained from the three meta-paths are concatenated to form a comprehensive embedding representation. This comprehensive embedding representation is then input into a wide network to construct enhanced nodes. Subsequently, the output weight matrix is ​​solved using ridge regression to obtain the corresponding task output results. In this embodiment, the movie node classification results are used as the verification method. This step can unify and integrate representations from different semantic perspectives and further enhance the expressive power of features through a wide network.

[0100] Step 5: Generate task-related output The experimental results of the method of this invention on the Freebase dataset are as follows, with training set proportions of 20%, 40%, and 60% respectively: The Macro-F1 scores were 60.78, 61.87, and 60.98, respectively. The Micro-F1 scores were 62.65, 64.57, and 63.87, respectively.

[0101] Experimental results show that the method of this invention outperforms the baseline model on the Freebase dataset, indicating that the method is well-suited for learning heterogeneous graph representations with complex structural relationships and diverse semantic types. In this embodiment, node classification is used as a specific verification method.

[0102] Example 4: This embodiment illustrates the application of the method of the present invention on the ACM dataset.

[0103] Step 1: Input ACM heterogeneous graph data The ACM dataset is a commonly used heterogeneous graph dataset for academic networks, consisting of three types of nodes: papers, authors, and research topics. Paper nodes represent academic papers, author nodes represent the authors of those papers, and topic nodes represent the research areas to which the papers belong. In this embodiment, a paper node classification task is used as the validation method, and the papers are divided into three categories.

[0104] The dataset used in this embodiment contains 3025 papers, 5835 authors, and 56 research topic nodes. Semantic relationships can be established between papers through author and research topic relationships, making it suitable for verifying the representation learning performance of the method of this invention in multi-semantic heterogeneous graph scenarios.

[0105] Step 2: Define metapath Based on the semantic relationships in the ACM dataset, the following meta-paths are selected: Paper—Author—Paper (PAP); Paper—Topic—Paper (PSP).

[0106] Here, PAP indicates that two papers are written by the same author, and PSP indicates that two papers have the same research topic. Through these two types of meta-paths, paper nodes can be modeled from two main semantic perspectives: author relationship and topic relationship.

[0107] Step 3: Construct the meta-path subgraph and perform residual multi-head graph attention convolution. Based on predefined metapaths PAP and PSP, two metapath subgraphs are constructed from the original heterogeneous graph. Then, residual multi-head graph attention convolutions are performed on each metapath subgraph to aggregate features of the neighborhood information of paper nodes. This step is used to learn the representations of paper nodes under author collaboration semantics and topic similarity semantics, respectively.

[0108] Step 4: Fuse multi-path features and input them into a wide network The paper node features obtained from each metapath subgraph are concatenated and fused to obtain a comprehensive embedding representation of the paper nodes. This comprehensive embedding representation is then input into a wide network to construct enhanced nodes. Subsequently, ridge regression is used to solve for the output weight matrix, yielding the corresponding task output results. In this embodiment, the paper node classification results are used as the verification method.

[0109] Step 5: Generate task-related output The experimental results of the method of this invention on the ACM dataset are as follows, with training set proportions of 20%, 40%, and 60% respectively: The Macro-F1 scores were 91.68, 91.06, and 91.83, respectively. The Micro-F1 scores were 91.57, 91.01, and 91.73, respectively.

[0110] Experimental results show that the method of this invention outperforms the baseline model in validation on the ACM dataset. Through the joint design of meta-path subgraph modeling, residual multi-head graph attention convolution, and width network output solving mechanism, the method of this invention can effectively extract node features in different semantic spaces and achieve better results in scenarios where node classification is used as the validation method.

[0111] Example 5 This embodiment illustrates the parameter settings and their effects in the method of the present invention.

[0112] Step 1: Set the number of enhanced node groups In the method of this invention, since the input features of the wide network no longer use the traditional random mapping layer, but directly use the node features after residual convolution, the parameter analysis mainly focuses on the enhancement layer parameters.

[0113] In this embodiment, the number of feature enhancement groups is examined separately. and feature enhancement dimensions Impact on model performance. Specifically, the number of feature enhancement groups. The values ​​are 10, 15, 20, 25, and 30 respectively; feature enhancement dimension. The values ​​were set to 10, 15, 20, 25, and 30 respectively. The model was run under different parameter settings, and the changes in Macro-F1 and Micro-F1 scores were recorded.

[0114] Step 2: Set Enhanced Node Dimensions While keeping other parameters constant, we changed the dimension of each group of augmentation nodes and observed the impact of different augmentation dimensions on model performance. The dimension of the augmentation nodes determines the number of new features introduced by each group of augmentation nodes. If the value is too small, it may lead to insufficient scalability; if the value is too large, it may increase the computational burden and introduce redundant features. Therefore, a comprehensive analysis based on experimental results is necessary.

[0115] Step 3: Determine the preferred parameters Experimental results show that: When the number of feature enhancement groups When taking 20 and when the feature enhancement dimension is 20 When the value is 25, it achieves better results in validation experiments on the DBLP, Freebase, and ACM datasets; Therefore, in the implementation of this invention, setting the number of enhancement groups to 20 and the enhancement dimension to 25 can achieve better results in relevant verification experiments. Figure 3 This illustrates the impact of varying the number of feature enhancement groups on performance. Figure 4 The impact of feature enhancement dimension changes on performance is shown.

[0116] The parameter change trend shows that increasing the number of augmentation node groups and augmentation dimensions within a certain range can usually improve model performance. This is because the wide network can more fully expand the expression space of fused features at this stage. However, when the parameters increase further, the performance improvement will tend to level off, and there may even be slight fluctuations. This indicates that the augmentation layer parameters are not necessarily better the larger they are, but a balance should be struck between expressive power and redundancy control.

[0117] Example 6: This embodiment illustrates the ablation comparison of the method of the present invention.

[0118] To further verify the contribution of each core component of this invention to the improvement of model performance, ablation experiments were conducted on three datasets: DBLP, Freebase, and ACM. In the ablation experiments, one key module was removed or replaced from the complete model structure, while the remaining network structure and training parameters remained unchanged to ensure the comparability of the experimental results.

[0119] Step 1: Remove residual connections definition This indicates the removal of residual connection terms in the convolution operation, used to verify the impact of residual structure on the stability of deep propagation.

[0120] Experimental results show that removing residual connections significantly reduces the Macro-F1 scores on the Freebase, DBLP, and ACM datasets. Specifically, the Macro-F1 score drops from 60.98 to 58.21 on the Freebase dataset, from 93.91 to 92.30 on the DBLP dataset, and from 91.83 to 86.16 on the ACM dataset.

[0121] The results demonstrate that introducing residual connections into multi-layer graph attention convolutional structures can effectively alleviate feature degradation during deep propagation, thereby improving the stability of node representations.

[0122] Step 2: Change multi-head attention to single-head attention definition This indicates that the multi-head attention mechanism has been changed to single-head attention, which is used to evaluate the contribution of the multi-head mechanism to the ability to represent neighborhood information.

[0123] Experimental results show that when multi-head attention is changed to single-head attention, the model performance also decreases. On the Freebase, DBLP, and ACM datasets, the Macro-F1 score drops to 59.34, 92.85, and 86.35, respectively.

[0124] The results demonstrate that the multi-head attention mechanism can learn neighborhood dependencies in parallel in different subspaces, enabling the model to capture more semantic information in heterogeneous graphs with finer granularity, thereby improving node representation capabilities.

[0125] Step 3: Remove width network augmentation nodes definition This indicates that the wide network is replaced with a single-layer linear classifier to test the effectiveness of the wide network feature enhancement structure.

[0126] Experimental results show that when the wide network is replaced with a single-layer linear classifier, the relevant validation results further decrease. On the Freebase, DBLP, and ACM datasets, the Macro-F1 score drops to 59.01, 92.15, and 84.65, respectively.

[0127] The results demonstrate that the enhanced node structure in the wide network can further expand the feature representation space after multi-path feature fusion, thereby improving the effectiveness of node representation and playing a positive role in subsequent task processing stages.

[0128] Step 4: Keep only single metapaths definition This indicates that convolution is not performed on multiple meta-path subgraphs, but only on a single meta-path subgraph, which is used to verify the necessity of multi-path semantic fusion.

[0129] Experimental results show that the model performance degrades most significantly when trained on only a single meta-path. On the Freebase, DBLP, and ACM datasets, the Macro-F1 score drops to 55.43, 90.12, and 84.65, respectively.

[0130] This result shows that multi-path modeling can characterize the structural relationships between nodes from different semantic perspectives. Different meta-paths have complementary information, and it is difficult to fully express the complex semantic structure in heterogeneous graphs by relying on a single path.

[0131] Step 5: Compare experimental results The experimental results above demonstrate that the residual connection structure, multi-head attention mechanism, wide network feature enhancement, and multi-path semantic modeling in this invention significantly improve the representation learning performance and related validation results of heterogeneous graphs. The modules work synergistically to enhance the model's representation learning performance and validation results in complex heterogeneous graph environments. Therefore, this invention is not a simple replacement of a single module in existing technologies, but rather a systematic design centered around the overall process of "multi-path semantic modeling—residual graph convolution extraction—efficient wide network output solution."

Claims

1. A method for learning heterogeneous graph representations in residual width networks using multi-path convolution, characterized in that, Includes the following steps: Step 1: Input heterogeneous graph data and construct corresponding semantic subgraphs based on multiple predefined meta-paths; Step 2: Perform multi-layer residual graph attention convolution on each meta-path semantic subgraph to extract the feature representation of the node in the corresponding semantic space; Step 3: Fuse the node features obtained from different meta-paths to obtain multi-path fusion features; Step 4: Use the multi-path fusion features as input to the wide network to construct enhanced nodes and form an overall feature matrix; Step 5: Based on the overall feature matrix, use ridge regression closed-form solution to solve the output weights and generate the output results of the corresponding downstream tasks.

2. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 1, characterized in that, The heterogeneous graph input in Step 1 is represented as follows: ； in, Represents a set of nodes. Denotes the set of edges. Represents a set of node types. Represents a set of edge types; The metapath set is: ； in, This represents the number of metapaths, and each metapath... Used to describe a complex semantic relationship between nodes in a heterogeneous graph; For each predefined metapath Construct its corresponding isoproton graph: ； in, This represents the set of nodes that satisfy the semantic constraints of the metapath. This represents the set of edges that satisfy the meta-path connection relationship.

3. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 1, characterized in that, The residual map attention convolution in Step 2 includes: The node features are linearly mapped to obtain subspace features; the attention scores of the node and its neighbors are calculated and normalized to obtain attention coefficients; the neighbor features are weighted and aggregated based on the attention coefficients to obtain multi-head attention output; a residual connection is added after each convolution layer to weightedly fuse the current layer output with the previous layer input.

4. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 3, characterized in that, The residual connection satisfies: ； in, Indicates the number of convolutional layers. Represents a node In metapath The second chapter Intermediate features obtained from multi-head graph attention convolution. It is a learnable scaling factor used to balance the new features learned in the current layer with the features retained in the previous layer; The attention coefficient is asymmetric to accommodate the directionality and diversity of node relationships in heterogeneous graphs.

5. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 1, characterized in that, In Step 3, the multi-path feature fusion adopts a feature dimension concatenation method, assuming a total of A single path, then the node The fusion features are represented as follows: ； in, Represents a node In the Passing through the element path Features obtained after attention convolution of layer residual maps This represents the final node embedding representation.

6. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 1, characterized in that, In Step 4, the width network eliminates the traditional random feature mapping layer and directly uses multi-path fusion features as network input. The enhanced nodes of the width network are constructed as follows: ； in, Indicates the enhanced node group number, The multi-path fusion characteristics of each node The input feature matrix is ​​formed by stacking rows. The weight matrix is ​​generated randomly. For bias terms, It is a non-linear activation function.

7. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 6, characterized in that, In Step 4, all the enhanced nodes are concatenated column by column to obtain the enhanced node matrix. ; Input feature moments Concatenate matrix with enhanced nodes By concatenating the dimensions, we obtain the overall feature matrix. .

8. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 7, characterized in that, Step 5, which involves solving for the output weights using ridge regression in a closed-form manner, specifically includes: After obtaining the overall feature matrix Then, the weight matrix is ​​output. Generate task-related outputs; output weight matrix Solving using ridge regression: ； in, This is the regularization coefficient, used to suppress overfitting; The supervision matrix corresponding to the input feature matrix X. It is an identity matrix.

9. The method for learning heterogeneous graph representations in residual width networks using multi-path convolution as described in claim 1, characterized in that, The residual graph attention convolution employs a multi-head attention mechanism to model neighborhood dependencies in parallel across multiple subspaces, thereby capturing multi-semantic information of heterogeneous graphs with fine granularity.

10. A system for learning heterogeneous graph representations of residual width networks using the multivariate path convolution method according to any one of claims 1-9, characterized in that, include: The subgraph construction module is used to decompose a heterogeneous graph into multiple semantic subgraphs based on predefined meta-paths; The residual convolution module is used to perform multi-layer residual graph attention convolutions on each semantic subgraph; The feature fusion module is used to fuse node features from different meta-paths; The width enhancement module is used to construct enhanced nodes and generate the overall feature matrix; The output solver module is used to solve for the output weights using ridge regression closed-form and generate the task output.

Images