Hypergraph representation learning method and device based on incompatibility adaptive position coding

Abstract

This application discloses a hypergraph representation learning method and apparatus based on heterogeneity adaptive positional encoding, which can be applied to the field of graph neural network technology. This application constructs a heterogeneity adaptive positional encoding module and a heterogeneity-aware distinguishable aggregation module. A triple array is input into the heterogeneity adaptive positional encoding module to capture the first output representation corresponding to long-distance semantically related neighbor relationships in the hypergraph to be analyzed. The obtained triple array is then input into the heterogeneity-aware distinguishable aggregation module to obtain a second output representation that can distinguish the importance of different neighbor nodes in the hypergraph to feature aggregation. The first and second output representations are then fused to obtain the target node embedding representation. Based on the target node embedding representation, a pre-defined hypergraph neural network model is optimized using supervised learning, thereby achieving low-dimensional feature representation learning for heterogeneous hypergraph nodes and effectively improving the representation learning performance of heterogeneous hypergraphs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of graph neural networks, and particularly relates to a hypergraph representation learning method and device based on heterophily adaptive position coding. BACKGROUND

[0002] In related technologies, hypergraph is a data structure that can represent the interaction relationship of multiple entities at the same time, and is increasingly applied to modeling complex networks with high-order relationship structure, such as social networks and biological information networks. Unlike traditional graph structures, the edges (also known as hyperedges) of hypergraph can simultaneously connect multiple nodes, thereby naturally modeling high-order dependencies. In recent years, hypergraph neural networks (HGNN) have become a popular method for hypergraph representation learning, and various HGNN variants have been proposed to improve the quality of node low-dimensional embedding. However, most existing HGNN methods generally rely on homophily, resulting in the current hypergraph representation learning having difficulties in capturing long-distance semantic relationships and serious confusion of semantic information of heterogeneous neighbors, which makes the representation learning performance of heterophily hypergraph relatively low.

[0003] In summary, the technical problems existing in related technologies need to be improved. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose a hypergraph representation learning method and device based on heterophily adaptive position coding, which can effectively improve the representation learning performance of heterophily hypergraph.

[0005] To achieve the above purpose, one aspect of the embodiments of the present application proposes a hypergraph representation learning method based on heterophily adaptive position coding, which comprises the following steps:

[0006] Obtaining a triple array corresponding to a hypergraph to be analyzed in a preset field, the triple array comprising a node set, a hyperedge set and an association matrix, the association matrix being used to represent the relationship between nodes in the node set and hyperedges in the hyperedge set;

[0007] Constructing a heterophily adaptive position coding module in a preset hypergraph neural network model, the heterophily adaptive position coding module being used to capture long-distance semantically related homogeneous neighbor relationships in the hypergraph to be analyzed;

[0008] Inputting the triple array into the heterophily adaptive position coding module to obtain a first output representation, the first output representation comprising a global position offset, semantic structure information, and a node and hyperedge hybrid representation fused with attention mechanism;

[0009] A heterogeneous sensing distinguishable aggregation module is constructed in the preset hypergraph neural network model. The heterogeneous sensing distinguishable aggregation module is used to distinguish the importance of different neighbor nodes to feature aggregation in the hypergraph to be analyzed.

[0010] The triple array is input into the heterogeneous sensing distinguishable aggregation module to obtain a second output representation, which integrates structural attention information and hyperedge sensing embedding.

[0011] The first output representation and the second output representation are fused to obtain the target node embedding representation;

[0012] Based on the target node embedding representation, the preset hypergraph neural network model is optimized using supervised learning.

[0013] The unlabeled nodes of the hypergraph to be analyzed are classified based on the optimized preset hypergraph neural network model.

[0014] In some embodiments, obtaining the ternary array corresponding to the hypergraph to be analyzed in the preset domain includes:

[0015] Obtain the hypergraph to be analyzed in the preset domain;

[0016] Extract all nodes of the hypergraph to be analyzed to form a node set;

[0017] Extract all hyperedges of the hypergraph to be analyzed to form a hyperedge set;

[0018] Construct the association matrix between the node and the hyperedge;

[0019] The ternary array is formed by the set of nodes, the set of hyperedges, and the association matrix.

[0020] In some embodiments, the heterogeneity adaptive position coding module includes a preset spatial structure and an attention mechanism unit, wherein the preset spatial structure includes a continuously differentiable spatial structure constructed by a kernel function.

[0021] In some embodiments, inputting the triple array into the heterogeneity adaptive position encoding module to obtain a first output representation includes:

[0022] Construct the initial position embedding vector of the nodes or hyperedges in the triple array;

[0023] Construct a structural deviation function for any two nodes or any two hyperedges based on the initial position embedding vector;

[0024] Generate a structural deviation vector based on the structural deviation function;

[0025] Obtain the structural perturbation matrix;

[0026] A nonlinear offset transformation function is constructed based on the structural deviation vector and the structural perturbation matrix;

[0027] A structure-aware position offset representation is constructed based on the structural deviation vector and the nonlinear offset transformation function;

[0028] The structure-aware position offset representation is fused with nodes and hyperedges to obtain the structure-enhanced input representation;

[0029] The enhanced input representation is input into the attention mechanism unit to obtain the first output representation.

[0030] In some embodiments, the attention mechanism unit includes multiple stacked deep learning layers, each of which computes the interaction process between all position encoding nodes through a self-attention mechanism.

[0031] In some embodiments, inputting the ternary array into the heterogeneous sensing distinguishable aggregation module to obtain a second output representation includes:

[0032] Obtain the preliminary embedding representation corresponding to the triple array;

[0033] The second-order variance matrix of each node or each hyperedge in the ternary array is calculated based on the preliminary embedding representation.

[0034] The second-order variance matrix is ​​vectorized;

[0035] The importance weights of each second-order variance matrix are learned based on the structural attention mechanism and the vectorized second-order variance matrix;

[0036] Structural features are aggregated based on the importance weights to obtain an enhanced embedded representation.

[0037] The second output representation is generated based on the structure-enhanced embedded representation and the initial embedded representation.

[0038] In some embodiments, generating the second output representation based on the structure-enhanced embedding representation and the preliminary embedding representation includes:

[0039] Calculate the attention weights for each pair of nodes inside each hyperedge;

[0040] The attention weights are normalized to obtain the structural attention coefficients;

[0041] Construct an attention matrix based on the structural attention coefficients;

[0042] The second output representation is calculated based on the attention matrix, the structure-enhanced embedding representation, and the initial embedding representation.

[0043] In some embodiments, fusing the first output representation and the second output representation to obtain the target node embedding representation includes:

[0044] The first output representation and the second output representation are concatenated according to the feature dimension to obtain the joint embedding representation;

[0045] Construct the projection matrix;

[0046] The target node embedding representation is calculated based on the projection matrix and the joint embedding representation.

[0047] To achieve the above objectives, another aspect of this application proposes a hypergraph representation learning device based on heterogeneity adaptive position coding, the device comprising:

[0048] The acquisition module is used to acquire a ternary array corresponding to the hypergraph to be analyzed in a preset domain. The ternary array includes a node set, a hyperedge set, and an association matrix. The association matrix is ​​used to characterize the relationship between the nodes in the node set and the hyperedges in the hyperedge set.

[0049] The first construction module is used to construct a heterogeneous adaptive position encoding module in a preset hypergraph neural network model. The heterogeneous adaptive position encoding module is used to capture long-distance semantically related neighbor relationships in the hypergraph to be analyzed.

[0050] The first processing module is used to input the triple array into the heterogeneous adaptive position encoding module to obtain a first output representation. The first output representation includes global position offset, semantic structure information, and a hybrid representation of nodes and hyperedges fused by the attention mechanism.

[0051] The second construction module is used to construct the heterogeneous perception distinguishable aggregation module in the preset hypergraph neural network model. The heterogeneous perception distinguishable aggregation module is used to distinguish the importance of different neighbor nodes to feature aggregation in the hypergraph to be analyzed.

[0052] The second processing module is used to input the triple array into the heterogeneous sensing distinguishable aggregation module to obtain a second output representation, which integrates structural attention information and hyperedge sensing embedding.

[0053] The fusion module is used to fuse the first output representation and the second output representation to obtain the target node embedding representation;

[0054] The optimization module is used to optimize the preset hypergraph neural network model based on the target node embedding representation and in conjunction with supervised learning.

[0055] The classification module is used to classify the unlabeled nodes of the hypergraph to be analyzed based on the optimized preset hypergraph neural network model.

[0056] To achieve the above objectives, another aspect of this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0057] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0058] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0059] The embodiments of this application include at least the following beneficial effects: This application provides a hypergraph representation learning method and apparatus based on heterogeneity adaptive positional encoding. This scheme constructs a heterogeneity adaptive positional encoding module and a heterogeneity-aware distinguishable aggregation module in a preset hypergraph neural network model. The obtained triple array is input into the heterogeneity adaptive positional encoding module to capture the first output representation corresponding to long-distance semantically related neighbor relationships in the hypergraph to be analyzed. The obtained triple array is then input into the heterogeneity-aware distinguishable aggregation module to obtain a second output representation that can distinguish the importance of different neighbor nodes in the hypergraph to feature aggregation. The first and second output representations are then fused to obtain the target node embedding representation. Based on the target node embedding representation, the preset hypergraph neural network model is optimized using supervised learning. This enables low-dimensional feature representation learning of heterogeneous hypergraph nodes, effectively improving the representation learning performance of heterogeneous hypergraphs and thus improving the accuracy of classifying unstandardized nodes. Attached Figure Description

[0060] Figure 1 This is a flowchart of the hypergraph representation learning method based on heteromatch adaptive position coding provided in the embodiments of this application;

[0061] Figure 2 This is a schematic diagram of the preset hypergraph neural network model provided in the embodiments of this application;

[0062] Figure 3 This is a schematic diagram of a hypergraph provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram of the structure of the hypergraph representation learning device based on heterogeneity adaptive position coding provided in the embodiments of this application;

[0064] Figure 5 This is a schematic diagram of the hardware structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0066] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0067] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0069] In related technologies, hypergraphs, as a data structure capable of simultaneously representing the interactions of multiple entities, are increasingly being used to model complex networks with high-order relational structures, such as social networks and bioinformatics networks. Unlike traditional graph structures, the edges (also called hyperedges) of a hypergraph can connect multiple nodes simultaneously, thus naturally modeling high-order dependencies. In recent years, Hypergraph Neural Networks (HGNNs) have become a popular method for hypergraph representation learning, and various HGNN variants have been proposed to improve the quality of low-dimensional node embeddings. However, most existing HGNN methods generally rely on the "homophily" assumption, which assumes that nodes within a hyperedge tend to have the same category (label). This assumption does not always hold true in reality, especially in heterophilic hypergraphs, where hyperedges often connect nodes from different categories. This leads to the following challenges for traditional HGNN methods:

[0070] First, semantic relationships between distant nodes of the same type are difficult to capture: In heterogeneous hypergraphs, nodes of the same category are often not directly connected, but rather indirectly connected through multiple dissimilar nodes as intermediaries. For example, two scholars researching the same topic may establish connections through different institutions or partners. Faced with heterogeneous hypergraphs, traditional HGNNs based on first-order or local neighbor aggregation can only capture short-distance local relationships, making it difficult to explicitly capture the relationships between nodes of the same type spanning multiple dissimilar nodes, thus limiting the ability of hypergraph neural network models to perceive semantic consistency information.

[0071] Second, semantic confusion between dissimilar neighbors: Most current hypergraph neural network models tend to assume that all neighbors have the same contribution when aggregating neighbor features, failing to effectively distinguish the information differences between nodes of the same type and nodes of different types. This approach is particularly serious in heterogeneous hypergraphs, because dissimilar nodes may have significant semantic differences. Directly fusing their features can easily introduce noise, confuse the category features of the target node itself, reduce its representational discriminative power, and affect the performance of downstream classification or clustering tasks.

[0072] Although some studies have attempted to improve existing HGNNs by introducing strategies such as attention mechanisms, adaptive aggregation, or positional encoding, there are still problems such as insufficient characterization of heterogeneous structures and insufficient utilization of higher-order information, which in turn result in low representation learning performance of hypergraph neural network models for heterogeneous hypergraphs.

[0073] In view of this, this application provides a method and apparatus for learning the representation of a hypergraph based on heterogeneity adaptive position coding, which can effectively improve the representation learning performance of heterogeneous hypergraphs.

[0074] The hypergraph representation learning method based on heterogeneous adaptive positional coding provided in this application relates to the field of graph neural network technology. This method can be applied to terminals, servers, or software running on either a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the hypergraph representation learning method based on heterogeneous adaptive positional coding, but is not limited to the above forms.

[0075] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0076] The embodiments of this application will be described in detail below with reference to the accompanying drawings:

[0077] Figure 1 This is an optional flowchart of a hypergraph representation learning method based on heteromatch adaptive position coding provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S110 to S180:

[0078] Step S110: Obtain the triple array corresponding to the hypergraph to be analyzed in the preset domain. The triple array includes a node set, a hyperedge set, and an incidence matrix. The incidence matrix is ​​used to characterize the relationship between the nodes in the node set and the hyperedges in the hyperedge set.

[0079] Step S120: Construct the heterogeneous adaptive position coding module in the preset hypergraph neural network model. The heterogeneous adaptive position coding module is used to capture the semantically related neighbor relationships at a long distance in the hypergraph to be analyzed.

[0080] Step S130: Input the triple array into the heterogeneous adaptive position encoding module to obtain the first output representation. The first output representation includes global position offset, semantic structure information, and a hybrid representation of nodes and hyperedges fused by the attention mechanism.

[0081] Step S140: Construct the heterogeneous perception distinguishable aggregation module in the preset hypergraph neural network model. The heterogeneous perception distinguishable aggregation module is used to distinguish the importance of different neighbor nodes in the hypergraph to be analyzed for feature aggregation.

[0082] Step S150: Input the triple array into the heterogeneous perception distinguishable aggregation module to obtain the second output representation, which integrates structural attention information and hyperedge perception embedding.

[0083] Step S160: Fuse the first output representation and the second output representation to obtain the target node embedding representation;

[0084] Step S170: Optimize the preset hypergraph neural network model based on the target node embedding representation and supervised learning;

[0085] Step S180: Classify the unlabeled nodes of the hypergraph to be analyzed based on the optimized preset hypergraph neural network model.

[0086] It is understood that this embodiment can obtain a hypergraph to be analyzed in a preset domain, extract all nodes of the hypergraph to form a node set, extract all hyperedges of the hypergraph to form a hyperedge set, construct an association matrix between nodes and hyperedges, and then form a triple array based on the node set, the hyperedge set, and the association matrix. Specifically, the preset domain can be a social network domain, a bioinformatics network domain, etc. For example, the triple array can be represented as... in, Let ε represent the set of nodes, where ε = {e1, e2, ..., e}. m Let H ∈ {0,1} denote the set of hyperedges. n×m This represents the incidence matrix, used to characterize the relationship between nodes and hyperedges. In the incidence matrix, if a node... Belongs to hyperedge e j If ∈ε, then H ij =1, No

[0087] Then H ij =0. The initial features of a node are represented as a feature matrix. Where the i-th row Represents node vi The initial eigenvectors.

[0088] It is understandable that, such as Figure 2 As shown, the preset hypergraph neural network model includes a Category-aware Adaptive Positional Encoding (CAPE) module and a Fine-Grained Fusion Module (FGF) module. The CAPE module includes a preset spatial structure and an attention mechanism unit. The preset spatial structure is a continuously differentiable spatial structure constructed using a kernel function. Specifically, to capture long-distance semantic structural relationships in the heterogeneous hypergraph, this embodiment introduces the CAPE module into the deep learning (Transformer) branch to model the global relative position representation between nodes and hyperedges. CAPE constructs a continuously differentiable preset spatial structure using a kernel function. In this structure, positional differences are represented by a learnable function and combined with initial features. The inputs are combined into the attention mechanism unit to achieve structure-sensitive global perception capabilities. In this embodiment, CAPE enhances semantic perception of long-distance heterogeneous structures, thereby improving the representation consistency of similar nodes across heterogeneous paths.

[0089] It is understood that the process of inputting the ternary array into the heterogeneous adaptive position encoding module to obtain the first output representation includes, but is not limited to, the following steps:

[0090] Construct the initial position embedding vector of nodes or hyperedges in a triple array;

[0091] Construct a structural deviation function for any two nodes or any two hyperedges based on the initial position embedding vector;

[0092] Generate a structural deviation vector based on the structural deviation function;

[0093] Obtain the structural perturbation matrix;

[0094] A nonlinear offset transformation function is constructed based on the structural deviation vector and the structural perturbation matrix;

[0095] A structure-aware position offset representation is constructed based on the structural deviation vector and the nonlinear offset transformation function;

[0096] The structure-aware position offset representation is fused with nodes and hyperedges to obtain the structure-enhanced input representation;

[0097] The enhanced input representation is input into the attention mechanism unit to obtain the first output representation.

[0098] Specifically, the heterogeneous adaptive position encoding module in this embodiment first constructs an initial position embedding vector for each node or hyperedge. The initial position embedding vector has the same dimension as the feature dimension. Then, the kernel function shown in Equation 1 is used to model the structural deviation between any two nodes or hyperedges.

[0099] in, These represent the initial position embedding vectors of two arbitrary nodes or hyperedges, used to learn their relative spatial positions in the hypergraph structure; `|·|` represents a learnable scalar parameter; `|·|` denotes element-wise absolute value operation. This kernel function measures the relative positional difference between any two nodes (or hyperedges) in the structure, and its output is represented as a structural deviation vector. To enhance the flexibility and nonlinear expressive power of this deviation representation, this embodiment introduces a trainable structural perturbation matrix. And the nonlinear offset transformation function shown in Equation 2 is constructed using the LeakyReLU activation function:

[0100] f(k ij ,B)=LeakyReLU(k ij +B) Formula 2;

[0101] LeakyReLU is an improved activation function used to alleviate the problem of zero gradient in the negative input region of standard ReLU. Its definition is shown in Equation 3:

[0102]

[0103] Here, α is a constant less than 1 (usually set to 0.01) used to preserve non-zero gradients in negative regions, thus avoiding the "neuron death" phenomenon. This activation function makes the structural perturbation term more flexible in encoding across structural locations, improving the ability to perceive complex topological variations. The final structure-aware position offsets of nodes and hyperedges are represented as shown in Equations 4 and 5:

[0104]

[0105]

[0106] Among them, P″ V P″ represents the structure-aware position offset of the node, respectively. EThe structure-aware position offset representation of the hyperedge is defined by σ(·), where σ(·) is the Sigmoid activation function, α(·) is the learnable attention weight function, and ⊙ represents element-wise multiplication, used to adjust the contribution strength of the structure offset to different attention channels. This position offset representation is fused with the original features of the node or hyperedge and input into the Transformer encoding module to improve the structural semantic modeling capability.

[0107] Finally, the structural offset encoding representation is element-wise added to and fused with the node and hyperedge features respectively, and this fusion serves as the input representation after structural enhancement, constituting the input to the Transformer encoding module. The structurally enhanced input representation is shown in Equation 6:

[0108]

[0109] Among them, P″ V The final result of encoding the node position, P″ E The final result of superedge position encoding, X V Let X be the initial feature of the node. E The initial features of the hyperedge are used; the concatenation operation is used to encode the nodes and hyperedges uniformly before inputting them into the Transformer structure. This input will be fed into multiple stacked deep learning layers (Transformers). Each Transformer layer calculates the interactions between nodes at all positions through a self-attention mechanism. The core attention weight calculation method is shown in Equation 7:

[0110]

[0111] Where Q, K, and V are respectively derived from input X Trans The resulting query, key, and value matrix is ​​mapped, where d is the feature dimension, and softmax(·) is used to normalize the interactive attention weights. Finally, the output of the Transformer network is shown in Equation 8:

[0112] Z Trans =Transformer(X) Trans ) Formula 8;

[0113] Among them, Z Trans A hybrid representation of nodes and hyperedges, incorporating global position offsets, semantic structure information, and attention mechanism integration, is used as the first output representation.

[0114] It is understood that, in order to further enhance the ability of the preset hypergraph neural network model to perceive the differences in local patterns of dissimilar neighbors, this embodiment designs a dissimilar perception distinguishable aggregation module (FGF) in the GCN branch. FGF can effectively distinguish the semantic contributions of similar and dissimilar neighbors through fine-grained association modeling and attention reweighting strategy, alleviate the problem of feature confusion of dissimilar nodes, and improve the discriminative power of node embedding.

[0115] Specifically, the process of inputting a ternary array into the heterogeneous sensing and distinguishable aggregation module to obtain a second output representation in this embodiment includes, but is not limited to, the following steps:

[0116] Obtain the preliminary embedding representation corresponding to the triple array;

[0117] Calculate the second-order variance matrix of each node or each hyperedge in the ternary array based on the preliminary embedding representation;

[0118] Vectorize the second-order variance matrix;

[0119] The importance weights of each second-order variance matrix are learned based on the structural attention mechanism and the vectorized second-order variance matrix.

[0120] Structural features are aggregated based on importance weights to obtain an embedded representation with enhanced structure.

[0121] The second output representation is generated based on the structurally enhanced embedding representation and the initial embedding representation.

[0122] It is understandable that the heterogeneous perception distinguishable aggregation module in this embodiment first obtains a preliminary embedded representation based on the convolution of the node and hyperedge hybrid graph. For each of these nodes or hyperedges, Z represents... i The second-order covariance matrix is ​​calculated using Formula 9:

[0123]

[0124] Here, the second-order covariance matrix represents the autocorrelation feature information of the current node or hyperedge in the embedding space. To incorporate this higher-order matrix into the attention mechanism, this embodiment vectorizes the second-order covariance matrix using Equation 10:

[0125]

[0126] Subsequently, a structural attention mechanism is introduced to learn the importance weights of each covariance matrix using Equation 11:

[0127]

[0128] in, For learnable parameters of structural attention, This is the bias term. Structural features are aggregated based on importance weights using Equation 12 to obtain the structurally enhanced embedding representation:

[0129]

[0130] It is understood that, in this embodiment, after obtaining the structurally enhanced embedding representation, a second output representation is generated based on the structurally enhanced embedding representation and the preliminary embedding representation. Specifically, this step includes, but is not limited to, the following steps:

[0131] Calculate the attention weights for each pair of nodes inside each hyperedge;

[0132] Normalize the attention weights to obtain the structural attention coefficient;

[0133] Construct an attention matrix based on structural attention coefficients;

[0134] The second output representation is calculated based on the attention matrix, the structure-enhanced embedding representation, and the initial embedding representation.

[0135] Specifically, to enhance the model's ability to characterize heterogeneous structures within hyperedges, this embodiment constructs a heterogeneous sensitivity attention mechanism on node pairs (i,j) within each hyperedge. The formula for calculating the attention weights between node pairs within hyperedges is shown in Equation 13:

[0136]

[0137] in, They are nodes v i ,v j Embedded representation of (or hyperedge), This indicates a feature-weighted operation. Here are the learnable linear weight matrix parameters, σ(·) is the nonlinear activation function (such as ReLU or GELU), and softplus(x) = log(1 + e^(-1 / 2)). x The activation function used for smoothing and normalization ensures that the attention value is positive and possesses differentiability and numerical stability. This attention mechanism measures the degree of structural coupling between node pairs in the current hyperedge. In this embodiment, to suppress the influence of scale differences, Equation 14 is used to adjust all attention weights. After normalization, the normalized structural attention coefficient β is obtained. ij :

[0138]

[0139] The normalized structural attention coefficient β ij Construct an attention matrix between nodes inside the hyperedge The node features are then weighted and aggregated using Formula 15 to fuse structural attention information and hyperedge-aware embedding, ultimately yielding the output of the GCN branch as the second output representation.

[0140]

[0141] in, This represents the feature weighting operation, A E This indicates that the structural attention mechanism is governed by the normalized coefficient β. ij The attention matrix is ​​formed.

[0142] Specifically, the heteromatch perception distinguishable aggregation module combines the second-order structure covariance matrix, node pair perception mechanism and attention normalization strategy to effectively enhance the ability of the preset hypergraph neural network model to perceive the differences within the heteromatch structure, thereby achieving distinguishable aggregation of node features under heteromatch structure.

[0143] It is understandable that in this embodiment, after obtaining the first and second output representations of the two branches, the first and second output representations are fused to obtain the target node embedding representation. The fusion process can involve concatenating the first and second output representations according to the feature dimensions to obtain a joint embedding representation, constructing a projection matrix, and then calculating the target node embedding representation based on the projection matrix and the joint embedding representation. Specifically, after completing the structural encoding of the Transformer branch and the GCN branch respectively, this embodiment further fuses their output representations to generate a unified node embedding representation, and then restores it to the original node space through an inverse mapping mechanism. Since the two representations are the same in dimension, feature-level concatenation or superposition fusion is allowed. Therefore, this embodiment adopts a concatenation method to generate the joint embedding representation, and the concatenation process is shown in Equation 16:

[0144]

[0145] in, It is the global embedded representation (first output representation) of the Transformer branch output. The local fine-grained embeddings (second output representations) generated for the GCN branches are concatenated along the feature dimension using `concat(·)`, preserving complementary information between the two branches. However, since the joint embedding representation contains mixed information from nodes and hyperedges, it is necessary to remap the mixed embeddings back to the pure node space. This embodiment uses a reverse mapping mechanism based on the hypergraph association matrix for projection. A projection matrix is ​​constructed. Each row corresponds to a node. If the node is the i-th node, then... Otherwise, the value is 0, meaning the matrix retains the node positions and discards the hyperedge portion to obtain the final node embedding representation as the target node embedding representation. The target node embedding representation is shown in Equation 17:

[0146]

[0147] It is understandable that, after obtaining the target node embedding representation, this embodiment optimizes the preset hypergraph neural network model based on supervised learning. Specifically, when some nodes in the hypergraph have known labels, the cross-entropy loss function shown in Equation 18 is constructed based on the fused node representation:

[0148]

[0149] in, In the predicted probability of node v, the y-th node... v The values ​​of each component correspond to their actual labels y. v The predicted probability. This indicates the number of nodes with labels.

[0150] This embodiment learns and determines all parameters of the model by iteratively optimizing the loss function. Then, based on the pre-defined hypergraph neural network model with determined parameters, it can classify the unlabeled nodes of the hypergraph to be analyzed.

[0151] In some embodiments, the methods of the present application can be applied to Figure 3 When illustrating the application using the hypergraph to be analyzed as an example, the steps include, but are not limited to:

[0152] Step 1: Hypergraph topology and node feature modeling. Figure 3 Represented as a hypergraph This hypergraph contains 9 nodes and 5 hyperedges, meaning the node set is as follows: The node categories include A (v1, v2, v6), B (v3, v4, v7), and C (v5, v8, v9). The hyperedge set consists of e1 = {v1, v3, v5, v7} (strongly heterogeneous structure), e2 = {v2, v4, v6}, e3 = {v3, v5, v9}, e4 = {v1, v2, v6} (internal edges of A-class nodes), and e5 = {v7, v8, v9} (connecting B and C-class nodes). Construct the node-hyperedge incidence matrix. Indicates the connection relationship between a node and its hyperedge:

[0153]

[0154] Step 2, Figure 3The hypergraph input heterogeneity adaptive position coding (CAPE) module, as shown, processes the data to obtain the first output representation. Specifically, CAPE first constructs an initial position embedding vector for each node or hyperedge. Its dimension is consistent with the feature dimension. Then, the structural deviation between any two nodes or hyperedges is modeled using the following kernel function:

[0155]

[0156] After introducing the offset term and applying LeakyReLU, we obtain: f(k) ij ,B)=LeakyReLU(k ij +B);

[0157] The position offset code P″ is obtained from the structural modeling. V With P″ E , which correspond to the spatial location information of the node and the hyperedge, respectively, and their characteristics are as follows:

[0158]

[0159]

[0160] Then, the positional encoding is added to the original feature:

[0161] The global representation Z is obtained by inputting a multi-layer Transformer. Trans (First output representation). For example, although v1 and v6 are not directly connected, they coexist in e4, and CAPE can compress their relative positions.

[0162] Step 3, Figure 3 The hypergraph input heterogeneous sensing and differentiation aggregation module processes the data to obtain a second output representation. Specifically, in this embodiment, each node is embedded with a hyperedge Z. i Generate the covariance matrix: After vectorizing the covariance matrix, it is aggregated using structural attention:

[0163]

[0164] And form the enhanced embedding representation Z struct Meanwhile, in the heterogeneous hyperedge e1, node pairs such as v1,v5 and v3,v7 belong to different categories, affecting the attention weight β. ij Suppression is performed to prevent erroneous feature aggregation, thereby obtaining the final output result as the second output representation.

[0165] Step 4: Global-Local Feature Collaborative Fusion. First, concatenate the first output representation corresponding to the Transformer branch and the second output representation corresponding to the GCN branch:

[0166] Z fusion =concat(Z) Trans Z GCN );

[0167] Among them, Z Trans Z represents the embedding feature matrix (first output representation) of the Transformer branch. GCN The embedding feature matrix (second output representation) corresponding to the GCN branch is shown below:

[0168]

[0169] Finally, through the mapping matrix The decoded final node representation (target node embedding representation) is as follows:

[0170]

[0171] Step 5: Optimize the pre-defined hypergraph neural network model based on supervised learning. Given some known node labels (e.g., v1, v3, v5, v7, v9), construct the cross-entropy loss function:

[0172]

[0173] The parameters of the entire model can be optimized through backpropagation, and after the model optimization is completed, the unlabeled nodes such as v2, v4, v6, and v8 can be accurately classified based on the learned node embeddings.

[0174] As described above, this embodiment provides a hypergraph representation learning method based on heterogeneity adaptive positional encoding. This method has a pre-defined hypergraph neural network model with a parallel structure of Transformer and GCN (Graph Convolution Networks). A heterogeneity adaptive positional encoding module is introduced into the Transformer branch to capture semantically related neighbor relationships at long distances. A heterogeneity-aware distinguishable aggregation module is designed in the GCN branch to distinguish the importance of different neighbor nodes in heterogeneous hyperedges for feature aggregation. Thus, by jointly learning through the Transformer and GCN branches, low-dimensional feature representation learning of heterogeneous hypergraph nodes can be achieved, thereby effectively improving the representation learning performance of heterogeneous hypergraphs.

[0175] Please see Figure 4 This application also provides a hypergraph representation learning device based on heterogeneity adaptive position coding, the device comprising:

[0176] The acquisition module 410 is used to acquire the ternary array corresponding to the hypergraph to be analyzed in the preset domain. The ternary array includes a node set, a hyperedge set, and an incidence matrix. The incidence matrix is ​​used to represent the relationship between the nodes in the node set and the hyperedges in the hyperedge set.

[0177] The first construction module 420 is used to construct the heterogeneous adaptive position coding module in the preset hypergraph neural network model. The heterogeneous adaptive position coding module is used to capture the semantically related neighbor relationships at a long distance in the hypergraph to be analyzed.

[0178] The first processing module 430 is used to input the triple array into the heterogeneous adaptive position encoding module to obtain a first output representation. The first output representation includes global position offset, semantic structure information, and a hybrid representation of nodes and hyperedges fused by the attention mechanism.

[0179] The second construction module 440 is used to construct a heterogeneous perception distinguishable aggregation module in a preset hypergraph neural network model. The heterogeneous perception distinguishable aggregation module is used to distinguish the importance of different neighbor nodes to feature aggregation in the hypergraph to be analyzed.

[0180] The second processing module 450 is used to input the ternary array into the heterogeneous sensing distinguishable aggregation module to obtain a second output representation, which integrates structural attention information and hyperedge sensing embedding.

[0181] The fusion module 460 is used to fuse the first output representation and the second output representation to obtain the target node embedding representation;

[0182] The optimization module 470 is used to optimize the preset hypergraph neural network model based on the target node embedding representation and in conjunction with supervised learning.

[0183] The classification module 480 is used to classify the unlabeled nodes of the hypergraph to be analyzed based on the optimized preset hypergraph neural network model.

[0184] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0185] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This computer device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0186] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0187] Please see Figure 5 , Figure 5 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes:

[0188] The processor 510 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0189] The memory 520 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 520 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510 using the methods described in the embodiments of this application.

[0190] The input / output interface 530 is used to implement information input and output;

[0191] The communication interface 540 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0192] Bus 550 transmits information between various components of the device (e.g., processor 510, memory 520, input / output interface 530, and communication interface 540);

[0193] The processor 510, memory 520, input / output interface 530 and communication interface 540 are connected to each other within the device via bus 550.

[0194] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0195] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0196] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0197] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0198] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0199] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0200] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0201] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0202] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0203] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0204] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0205] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0206] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0208] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0209] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A hypergraph representation learning method based on heteromatch adaptive positional coding, characterized in that, The method includes the following steps: Obtain a ternary array corresponding to the hypergraph to be analyzed in a preset domain. The ternary array includes a node set, a hyperedge set, and an association matrix. The association matrix is ​​used to characterize the relationship between the nodes in the node set and the hyperedges in the hyperedge set. A heterogeneous adaptive position encoding module is constructed in a preset hypergraph neural network model. The heterogeneous adaptive position encoding module is used to capture the semantically related neighbor relationships at a long distance in the hypergraph to be analyzed. The heterogeneous adaptive position encoding module includes a preset spatial structure and an attention mechanism unit. The preset spatial structure includes a continuous and differentiable spatial structure constructed by a kernel function. The triple array is input into the heterogeneous adaptive position encoding module to obtain a first output representation, which includes global position offset, semantic structure information, and a hybrid representation of nodes and hyperedges fused by the attention mechanism. A heterogeneous sensing distinguishable aggregation module is constructed in the preset hypergraph neural network model. The heterogeneous sensing distinguishable aggregation module is used to distinguish the importance of different neighbor nodes to feature aggregation in the hypergraph to be analyzed. The triple array is input into the heterogeneous sensing distinguishable aggregation module to obtain a second output representation, which integrates structural attention information and hyperedge sensing embedding. The first output representation and the second output representation are fused to obtain the target node embedding representation; Based on the target node embedding representation, the preset hypergraph neural network model is optimized using supervised learning. The unlabeled nodes of the hypergraph to be analyzed are classified based on the optimized preset hypergraph neural network model. The first output representation is obtained by inputting the triple array into the heterogeneity adaptive position encoding module, including: Construct the initial position embedding vector of the nodes or hyperedges in the triple array; Construct a structural deviation function for any two nodes or any two hyperedges based on the initial position embedding vector; Generate a structural deviation vector based on the structural deviation function; Obtain the structural perturbation matrix; A nonlinear offset transformation function is constructed based on the structural deviation vector and the structural perturbation matrix; A structure-aware position offset representation is constructed based on the structural deviation vector and the nonlinear offset transformation function; The structure-aware position offset representation is fused with nodes and hyperedges to obtain the structure-enhanced input representation; The enhanced input representation is input into the attention mechanism unit to obtain the first output representation.

2. The method according to claim 1, characterized in that, The step of obtaining the ternary array corresponding to the hypergraph to be analyzed in the preset domain includes: Obtain the hypergraph to be analyzed in the preset domain; Extract all nodes of the hypergraph to be analyzed to form a node set; Extract all hyperedges of the hypergraph to be analyzed to form a hyperedge set; Construct the association matrix between the node and the hyperedge; The ternary array is formed by the set of nodes, the set of hyperedges, and the association matrix.

3. The method according to claim 1, characterized in that, The attention mechanism unit includes multiple stacked deep learning layers, each of which calculates the interaction process between all position encoding nodes through a self-attention mechanism.

4. The method according to claim 1, characterized in that, The step of inputting the ternary array into the heterogeneous sensing and distinguishable aggregation module to obtain the second output representation includes: Obtain the preliminary embedding representation corresponding to the triple array; The second-order variance matrix of each node or each hyperedge in the ternary array is calculated based on the preliminary embedding representation. The second-order variance matrix is ​​vectorized; The importance weights of each second-order variance matrix are learned based on the structural attention mechanism and the vectorized second-order variance matrix; Structural features are aggregated based on the importance weights to obtain an enhanced embedded representation. The second output representation is generated based on the structure-enhanced embedded representation and the initial embedded representation.

5. The method according to claim 4, characterized in that, The step of generating the second output representation based on the structure-enhanced embedding representation and the preliminary embedding representation includes: Calculate the attention weights for each pair of nodes inside each hyperedge; The attention weights are normalized to obtain the structural attention coefficients; Construct an attention matrix based on the structural attention coefficients; The second output representation is calculated based on the attention matrix, the structure-enhanced embedding representation, and the initial embedding representation.

6. The method according to claim 1, characterized in that, The step of fusing the first output representation and the second output representation to obtain the target node embedding representation includes: The first output representation and the second output representation are concatenated according to the feature dimension to obtain the joint embedding representation; Construct the projection matrix; The target node embedding representation is calculated based on the projection matrix and the joint embedding representation.

7. A hypergraph representation learning device based on heteromatch adaptive positional coding, characterized in that, The device includes: The acquisition module is used to acquire a ternary array corresponding to the hypergraph to be analyzed in a preset domain. The ternary array includes a node set, a hyperedge set, and an association matrix. The association matrix is ​​used to characterize the relationship between the nodes in the node set and the hyperedges in the hyperedge set. The first construction module is used to construct a heterogeneous adaptive position encoding module in a preset hypergraph neural network model. The heterogeneous adaptive position encoding module is used to capture long-distance semantically related homogeneous neighbor relationships in the hypergraph to be analyzed. The heterogeneous adaptive position encoding module includes a preset spatial structure and an attention mechanism unit. The preset spatial structure includes a continuous and differentiable spatial structure constructed by a kernel function. The first processing module is used to input the triple array into the heterogeneous adaptive position encoding module to obtain a first output representation. The first output representation includes global position offset, semantic structure information, and a hybrid representation of nodes and hyperedges fused by the attention mechanism. The second construction module is used to construct the heterogeneous perception distinguishable aggregation module in the preset hypergraph neural network model. The heterogeneous perception distinguishable aggregation module is used to distinguish the importance of different neighbor nodes to feature aggregation in the hypergraph to be analyzed. The second processing module is used to input the triple array into the heterogeneous sensing distinguishable aggregation module to obtain a second output representation, which integrates structural attention information and hyperedge sensing embedding. The fusion module is used to fuse the first output representation and the second output representation to obtain the target node embedding representation; The optimization module is used to optimize the preset hypergraph neural network model based on the target node embedding representation and in conjunction with supervised learning. The classification module is used to classify the unlabeled nodes of the hypergraph to be analyzed based on the optimized preset hypergraph neural network model; The first output representation is obtained by inputting the triple array into the heterogeneity adaptive position encoding module, including: Construct the initial position embedding vector of the nodes or hyperedges in the triple array; Construct a structural deviation function for any two nodes or any two hyperedges based on the initial position embedding vector; Generate a structural deviation vector based on the structural deviation function; Obtain the structural perturbation matrix; A nonlinear offset transformation function is constructed based on the structural deviation vector and the structural perturbation matrix; A structure-aware position offset representation is constructed based on the structural deviation vector and the nonlinear offset transformation function; The structure-aware position offset representation is fused with nodes and hyperedges to obtain the structure-enhanced input representation; The enhanced input representation is input into the attention mechanism unit to obtain the first output representation.

8. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-6.

Images