A semantic search method based on node dependence of HGNN

Abstract

This invention discloses a node-dependent semantic search method based on HGNN. Unlike previous works, this invention considers the diversity of nodes in HG and, guided by a node-dependent semantic structure, attempts to perform node-dependent semantic search on HGNN for the first time. It constructs an expressive semantic structure comprising relation selection and connection selection components, transforming the semantic search on HGNN into searching for selection strategies within these two key components. Furthermore, to improve the adaptability of this invention, a predictor is designed to achieve node-dependent semantic search, guiding the information flow of HGNN through adaptive relation selection and connection selection. Extensive experimental results demonstrate that this invention generally outperforms all existing benchmark methods in node classification and connection prediction tasks, exhibiting high inference efficiency and effectively capturing task-relevant semantics.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and understanding, and in particular to a node-dependent semantic search method based on HGNN. Background Technology

[0002] Currently, Graph Neural Networks (GNNs) have become state-of-the-art techniques for many graph learning tasks. They typically employ a message-passing approach, iteratively aggregating neighbor features to update the representation of the central node. While GNNs have achieved success on homogeneous graphs, we note that on heterogeneous graphs (HGs), the naive message-passing mechanism ignores relational diversity, thus failing to capture the rich semantics of HGs and often leading to suboptimal performance in downstream tasks.

[0003] To address this issue, many researchers have focused on modifying GNNs to leverage the rich semantics of HGs to guide message passing, designing various heterogeneous graph neural networks (HGNNs). For example, heterogeneous graph attention networks (HANs) and metapath-aggregated graph neural networks (MAGNNs) incorporate human-defined metapaths into message passing. However, existing HGNNs primarily rely on human-defined metapaths, requiring significant human effort and expert knowledge.

[0004] To save manpower in designing semantic structures, Graph Transformer Networks (GTNs) and Heterogeneous Graph Transformers (HGTs) implicitly learn meta-paths through relation fusion based on attention mechanisms. However, this approach lacks explicit relation selection, and fusing all available relations remains noisy and computationally inefficient. Therefore, leveraging task-relevant semantics becomes a crucial issue in HGNN design.

[0005] A recently proposed method, Differentiable Meta Graph Search (DiffMG), explicitly searches for task-related semantics using Differentiable Neural Architecture Search (NAS). However, this method of message passing between layers, which preserves only one relation, inevitably leads to unnecessary information loss, thereby impairing the expressiveness of the searched semantics.

[0006] Given that all these methods have problems from different perspectives, effectively designing HGNNs under the guidance of task-related semantics remains an open question. Furthermore, existing semantic search methods for HGNNs yield a globally shared task-level semantic structure across all nodes, neglecting the diversity of different nodes. Therefore, how to conduct node-dependent semantic search based on HGNNs still requires further investigation. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a node-dependent semantic search framework (NDS) based on HGNN.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] This invention provides a node-dependent semantic search method based on HGNN, comprising the following steps:

[0010] S1. Edge assignment relationship selection operation between continuous blocks in a directed acyclic graph enables the HGNN layer to retain task-related relationships and perform neighborhood aggregation along the information propagation path related to the task to construct a local semantic structure.

[0011] S2. Edge assignment connection selection operation is performed between discontinuous blocks in a directed acyclic graph. Residual connection selection is used to fuse the semantic information captured by information propagation paths of different lengths to construct the higher-order part of the semantic structure.

[0012] S3. Sum the results of relation selection and join selection operations to obtain the final output representation of the HGNN layer.

[0013] Furthermore, a binary mask matrix is ​​used to represent the selection strategy and the connection strategy.

[0014] Furthermore, the relation selection process in step S1 is as follows:

[0015] In the l-th block, after relation-aware neighborhood aggregation, the node representation is updated as follows:

[0016]

[0017] in, A represents the representation of the relation selection component from the l-th block. l A controls the convergence process of perceived relationships. l It is also the output of the relational selection operation;

[0018] A l The generation process is as follows:

[0019]

[0020] Where ⊙ denotes element-wise matrix multiplication. This represents the binary mask matrix of the relation r obtained from the search on the l-th block.

[0021] Furthermore, the connection selection operation in step S2 is as follows:

[0022] The connection selection process for the l-th block is performed between it and all its non-contiguous predecessor blocks:

[0023]

[0024] in This represents the representation obtained from the connection selection component of the l-th block. This represents the binary connection mask matrix obtained from the search.

[0025] Furthermore, the final output of the HGNN layer in step S3 is represented as follows:

[0026]

[0027] The set of relation mask matrices and connection mask matrices is represented as:

[0028]

[0029] in This represents the binary mask matrix of the relation r obtained from the search on the l-th block. This represents the binary connection mask matrix obtained from the search.

[0030] Furthermore, in step S1, given the target node representation of the l-th layer input... And relation r, using a predictor that selects relations based on the dependencies of the predicted nodes. Predict the relationship selection probability of node i in This represents the selection strategy of relation r when node i performs neighborhood aggregation at level l. This indicates whether to retain or reject; it provides an approximation of the hard-choice probability. Then, the binary relation mask matrix Calculated as: in express The i-th column vector, where 1 represents an N-dimensional column vector with all elements having the value 1.

[0031] Furthermore, predictor by As input, the relation selection strategy is derived using a two-layer temperature-sensing softmax algorithm, as shown below:

[0032]

[0033]

[0034] in Predictor The output vector of softmax is τ, which is a temperature parameter. When τ→0, the output distribution of softmax becomes monothermic.

[0035] Further, in step S2, a predictor for the relational selection probability of node i is predicted using a predictor that predicts the connection selection dependent on the node, thus obtaining the binary connection mask matrix. The connection selection strategy is derived using a two-layer temperature-sensing softmax algorithm.

[0036] Furthermore, in step S2, a predictor is selected using the relational selection of the predicted node dependencies. Represented by the target node of the l-th layer input As input, the connection selection strategy is derived using a two-layer temperature-sensing softmax algorithm, as shown below:

[0037]

[0038]

[0039] in Predictor The output vector, The connection selection strategy between layer k and layer l of node i is represented by the connection mask matrix. in express The i-th row vector, 1′ represents a d-dimensional column vector with all elements having the value 1.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] This invention provides a node-dependent semantic search method based on HGNN, representing the first attempt to perform node-dependent semantic search on HGNN. It constructs an expressive semantic structure comprising relation selection and connection selection components, transforming the semantic search on HGNN into searching for selection strategies within these two key components. Furthermore, to enhance the adaptability of this invention, a predictor is designed to achieve node-dependent semantic search. Extensive experimental results demonstrate that this invention generally outperforms all existing benchmark methods on node classification and connection prediction tasks, exhibiting high inference efficiency and effectively capturing task-relevant semantics. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0043] Figure 1 This is an architecture diagram of a node-dependent semantic search method based on HGNN provided in an embodiment of the present invention. Detailed Implementation

[0044] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] The objective of this invention is to effectively design Hidden Generative Neural Networks (HGNNs) through task-related semantic search. To achieve this goal, we propose searching semantic structures and constructing an HGNN architecture guided by these structures. Different semantic structures can convey different semantics within the Hidden Generic Neural Network (HG) and can be seamlessly integrated into the heterogeneous message passing process of the HGNN. Therefore, from the perspective of searching semantic structures, constructing the optimal HGNN architecture naturally enables semantic search on the HGNN. Furthermore, considering the diversity of node samples in the HG, task-related semantics should adapt to different nodes. To improve the adaptability of this framework, we further design a predictor to perform node-dependent semantic search.

[0046] Regarding semantic structures on HGNNs, artificially defined "semantic structures" (meta-paths or meta-graphs) aim to describe the proximity between nodes. In contrast, semantic structures in the HGNN context tend to guide task-related message passing flows through the efficient selection and fusion of relations. Different combinations of such relation selection and fusion can be interpreted as different semantic information captured by the HGNN. Therefore, this invention transforms semantic search on HGNNs into designing the HGNN architecture from the perspective of semantic structure search.

[0047] The HGNN architecture of this invention is as follows: Figure 1 As shown, an HGNN architecture can be represented as a Directed Acyclic Graph (DAG). A DAG is an ordered sequence of (L+1) blocks, containing a start block and an end block. The l-th block represents the node representation H of the output of the l-th layer HGNN. l The directed edge (k, l) and the edge applied to H k An operation is associated with the endpoint block. The final output H of the HGNN corresponds to the endpoint block and is then applied to downstream tasks. By assigning operations to the directed edges of the DAG, the corresponding HGNN architecture can possess semantic expressive capabilities. In our design, operations can be categorized into two types: relation selection and connection selection, which respectively contribute to the local and higher-order construction of semantic structures.

[0048] Between consecutive blocks ( Figure 1 The edge selection operation assigned to edges (0,1) and (1,2) in (b) enables the HGNN layer to preserve task-related relations and perform neighborhood aggregation along them, which helps construct local semantic structures. Relation selection is performed in each block, and message passing is completed along the information propagation path related to the task. Such paths can convey meaningful semantic information.

[0049] On the other hand, between non-contiguous blocks ( Figure 1 The connection selection operation is assigned between edges (0,2) in (b) to flexibly integrate the semantic information captured by information propagation paths of different lengths, which helps to construct the higher-order part of the semantic structure.

[0050] Furthermore, we transform the semantic search on HGNN into an adaptive relation selection and connection selection strategy to generate task-relevant semantic structures. The technical details of these two selection operations will be described below.

[0051] This invention proposes a node-dependent semantic search method based on HGNN, comprising the following steps:

[0052] S1. Edge assignment relationship selection operation between continuous blocks in a directed acyclic graph enables the HGNN layer to retain task-related relationships and perform neighborhood aggregation along the information propagation path related to the task to construct a local semantic structure.

[0053] S2. Edge assignment connection selection operation is performed between discontinuous blocks in a directed acyclic graph. Residual connection selection is used to fuse the semantic information captured by information propagation paths of different lengths to construct the higher-order part of the semantic structure.

[0054] S3. Sum the results of relation selection and join selection operations to obtain the final output representation of the HGNN layer.

[0055] Regarding relationship selection:

[0056] The various relationships on an HG (Hypergraph) provide rich information that helps improve the performance of downstream tasks. However, task-irrelevant relationship information can also introduce noise. For example, in a citation network, the institution to which an author belongs can be considered noise when predicting an author's research field. Therefore, when HGNNs perform neighborhood aggregation, they need to selectively utilize heterogeneous information to attenuate noise and generate a task-relevant message flow. Based on the above discussion, we perform a relation selection operation on the edges between two contiguous blocks in a DAG. The selected relation guides neighbor aggregation to construct a local semantic structure.

[0057] Specifically, such as Figure 1 As shown in (c), in the l-th block, the node representation is updated after relation-aware neighborhood aggregation:

[0058]

[0059] in, A represents the representation of the relation selection component from the l-th block. l It controls the aggregation process of perceived relations, and it is also the output of relation selection operation.

[0060] Relation selection involves filtering out task-irrelevant relations and retaining task-relevant ones. We use a binary mask matrix to represent the selection strategy. A l The generation process is as follows:

[0061]

[0062] Where ⊙ denotes element-wise matrix multiplication. This represents the binary mask matrix of the relation r obtained from the search on the l-th block. The detailed calculation process will be described in detail later.

[0063] The information propagation path of HGNN established under the guidance of relation selection can convey rich semantics.

[0064] Regarding connection selection:

[0065] After exploring relation-selection-guided local neighborhood aggregation, we next introduce a higher-order part of the semantic structure constructed through connection selection to further enhance its expressive power. Information propagation paths of different lengths can convey semantic information from different perspectives. Longer propagation paths may introduce unnecessary noise, while shorter paths cannot carry sufficient semantic information. We further enable HGNN to flexibly fuse the semantics captured by information propagation paths of different lengths. We employ residual connection selection as the operation on the edges between two non-contiguous blocks of a DAG.

[0066] Specifically, such as Figure 1 As shown in (d), the connection selection process for the l-th block is performed between it and all its non-contiguous predecessor blocks:

[0067]

[0068] in This represents the representation obtained from the connection selection component of the l-th block. This represents the binary connection mask matrix obtained from the search, and its calculation process will be described in detail later.

[0069] By constructing the HGNN architecture from two parts—relation selection and connection selection—we can obtain... and These two representations, when combined, yield the final output representation of the l-th HGNN layer. Here, we choose summation as the combination function:

[0070]

[0071] The set of relation and connection mask matrices, i.e. This can be interpreted as a semantic structure because it can guide the message passing process of HGNN with the captured semantic information. To improve the adaptability of our proposed framework, we further refine the semantic structure search process to make it node-dependent.

[0072] Regarding the semantic structure of node dependencies:

[0073] Existing methods rely on searching globally shared static semantic structures, which is insufficient to capture the dynamic semantic information of different nodes. To overcome this limitation, we propose an adaptive method for searching node-related semantic structures. Specifically, we design a predictor module to predict the relation selection and connection selection strategies that nodes depend on.

[0074] We first introduce predictors for node dependency selection. Specifically, such as... Figure 1 As shown in (c), given the target node representation of the l-th layer input. Given relation r, our goal is to predict the relation selection probability of node i, i.e. in This represents the selection strategy for relation r when node i performs neighborhood aggregation at level l. Indicates to retain, and vice versa. Predictor by As input, the relationship selection strategy is derived using temperature-sensing softmax, as shown below:

[0075]

[0076]

[0077] in Predictor The output vector.

[0078] In this invention, we use a two-layer Multi-Layer Perceptron (MLP) as the predictor. Architecture. τ is a temperature parameter that controls the "sharpness" of the output distribution: when τ→0, the softmax output distribution becomes monothermic.

[0079] To obtain an approximate value of the hard-choice probability Then, the binary relation mask matrix It can be calculated as: in express The i-th column vector, where 1 represents an N-dimensional column vector with all elements having the value 1.

[0080] Similarly, connect the mask matrix It can also be obtained using a similar method:

[0081]

[0082]

[0083] in Predictor The output vector, The connection mask matrix represents the connection selection strategy between the k-th and l-th layers of node i. in express The i-th row vector, 1′ represents a d-dimensional column vector with all elements having the value 1.

[0084] With the help of the predictor, we can conduct a search for node-dependent semantic structures, enabling HGNN to dynamically capture the task-related semantics of different nodes.

[0085] Regarding the optimization of the predictor and HGNN:

[0086] Guided by the node-dependent semantic structure, HGNN performs L message passing steps to obtain the final representation matrix H. L , used for downstream tasks.

[0087] The optimization of the predictor and the GNN module is interdependent; jointly training them from scratch may result in poor task performance. Let ω be the parameter set of the GNN module, and λ be the parameter set of the predictor. We use an alternating optimization approach to iteratively update ω and λ.

[0088] This invention provides a node-dependent semantic search method based on HGNN, which differs from previous works. Taking into account the diversity of nodes in HG, and guided by a node-dependent semantic structure, this invention is the first to attempt to perform node-dependent semantic search on HGNN, constructing an expressive semantic structure comprising two components: relation selection and connection selection. The semantic search on HGNN is then transformed into searching for selection strategies within these two key components. Furthermore, to improve the adaptability of this invention, a predictor is designed to achieve node-dependent semantic search, guiding the information flow of HGNN through adaptive relation and connection selection.

[0089] Extensive experimental results demonstrate that this invention generally outperforms all existing benchmark methods (such as NAS) in node classification and connection prediction tasks, exhibiting high inference efficiency and effectively capturing task-related semantics. Furthermore, while NAS-based methods require a two-stage training process—search and retraining—this invention only requires a single-stage training phase, further improving operational efficiency.

[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A node-dependent semantic search method based on HGNN, characterized in that, When applied in citation networks to predict an author's research area, the process includes the following steps: S1. Edge assignment relation selection operation between continuous blocks in a directed acyclic graph (DAG) enables the HGNN layer to retain task-related relations and perform neighborhood aggregation along task-related information propagation paths to construct a local semantic structure; the relation selection operation process is as follows: In the l-th block, after relation-aware neighborhood aggregation, the node representation is updated as follows: in, A represents the representation of the relation selection component from the l-th block. l A controls the convergence process of perceived relationships. l It is also the output of the relational selection operation; A l The generation process is as follows: Where ⊙ denotes element-wise matrix multiplication. This represents the binary mask matrix of the relation r obtained from the search on the l-th block; S2. Assigning edge selection operations between discontinuous blocks in a directed acyclic graph, using residual connection selection to fuse semantic information captured by information propagation paths of different lengths to construct the higher-order part of the semantic structure; the connection selection operation process is as follows: The connection selection process for the l-th block is performed between it and all its non-contiguous predecessor blocks: in This represents the representation obtained from the connection selection component of the l-th block. This represents the binary connectivity mask matrix obtained from the search; S3. Sum the results of relation selection and join selection operations to obtain the final output representation of the HGNN layer.

2. The node-dependent semantic search method based on HGNN according to claim 1, characterized in that, A binary mask matrix is ​​used to represent the selection strategy and the connection strategy.

3. The node-dependent semantic search method based on HGNN according to claim 1, characterized in that, The final output of the HGNN layer in step S3 is represented as follows: The set of relation mask matrices and connection mask matrices is represented as: in This represents the binary mask matrix of the relation r obtained from the search on the l-th block. This represents the binary connection mask matrix obtained from the search.

4. The node-dependent semantic search method based on HGNN according to claim 1, characterized in that, In step S1, given the target node representation of the l-th layer input... And relation r, using a predictor that selects relations based on the dependencies of the predicted nodes. Predict the relationship selection probability of node i in This represents the selection strategy of relation r when node i performs neighborhood aggregation at level l. This indicates reservation, and vice versa; To obtain an approximate value of the hard-choice probability Then, the binary relation mask matrix Calculated as: in express The i-th column vector, where 1 represents an N-dimensional column vector with all elements having the value 1.

5. The node-dependent semantic search method based on HGNN according to claim 4, characterized in that, Predictor by As input, the relation selection strategy is derived using a two-layer temperature-sensing softmax algorithm, as shown below: in Predictor The output vector of softmax is τ, which is a temperature parameter. When τ→0, the output distribution of softmax becomes monothermic.

6. The node-dependent semantic search method based on HGNN according to claim 1, characterized in that, In step S2, a predictor is selected using the relationship that predicts the dependencies of the nodes. Represented by the target node of the l-th layer input As input, the connection selection strategy is derived using a two-layer temperature-sensing softmax algorithm, as shown below: in Predictor The output vector, The connection selection strategy between layer k and layer l of node i is represented by the connection mask matrix. in express The i-th row vector, 1 ′ This represents a d-dimensional column vector whose elements are all 1.

Images