A network representation learning method, device, apparatus and storage medium

Abstract

The present disclosure provides a network representation learning method, device and equipment and a storage medium, wherein the network representation learning method utilizes a target neural network to perform network representation learning in a target field; the target neural network comprises N-level first neural networks and N-1-level second neural networks; the output of the t-level first neural network is the input of the t-level second neural network; the output of the (t-1)-level second neural network is the input of the t-level first neural network and the input of the t-level second neural network; the network representation learning method comprises: obtaining heterogeneous network data; inputting the heterogeneous network data into the target neural network to obtain a first network representation; determining training data based on the heterogeneous network data and the first network representation; training a logistic regression model and the target neural network based on the training data; and taking the second network representation output by the target neural network when the training of the logistic regression model and the target neural network is completed as a target network representation of a target field task.

Description

Technical Field

[0001] This disclosure belongs to the field of network representation learning and graph neural network technology, specifically relating to a network representation learning method, apparatus, device and storage medium. Background Technology

[0002] Biological networks are a way of representing biological systems using graphs. Nodes in a biological network are elements within the biological system, and edges represent the relationships between these elements. For example, in a protein-protein interaction network, proteins constitute the nodes, and the interactions between proteins form the edges. Biological networks reflect the intricate relationships within biological systems, which is crucial for understanding these systems and can be used to solve various tasks in the life sciences. Therefore, the research and analysis of biological networks have high application value. A key aspect of studying biological networks lies in extracting their feature information, and network representation learning holds great potential in this area. Summary of the Invention

[0003] This disclosure aims to at least solve one of the technical problems existing in the prior art, and to provide a network representation learning method, apparatus, device and storage medium.

[0004] Firstly, the technical solution adopted to solve the technical problem of this disclosure is a network representation learning method, which uses a pre-built target neural network to learn the network representation in the target domain; the target neural network includes an N-level first neural network and an N-1-level second neural network; wherein, the intermediate node output by the t-th level first neural network is represented as the input of the corresponding t-th level second neural network; the target hidden state information output by the (t-1)-th level second neural network is the input of the t-th level first neural network and the input of the t-th level second neural network, respectively; N≥2, 2≤t≤N-1, and N and t are both rounded down;

[0005] The network representation learning method includes:

[0006] Acquire heterogeneous network data within the target domain; the heterogeneous network data includes data from each node in the heterogeneous network;

[0007] The heterogeneous network data is input into the target neural network, and after processing by the first neural network at each level and the second neural network at each level, the first network representation output by the target neural network is obtained;

[0008] Based on the heterogeneous network data and the first network representation, training data is determined;

[0009] Based on the training data, the logistic regression model and the target neural network are trained. When the training of the logistic regression model and the target neural network is completed, the second network representation output by the target neural network is used as the target network representation of the target domain task.

[0010] In some embodiments, acquiring heterogeneous network data in the target domain includes:

[0011] Retrieve at least one network from a pre-set database, wherein the network in the target domain contains multiple different types of nodes;

[0012] Based on the type of each node in the network under each target domain, the obtained networks under at least one target domain are integrated to build a heterogeneous network corresponding to the target domain task, and the heterogeneous network data is obtained.

[0013] In some embodiments, determining the intermediate node representation of the output of the first neural network at level t includes:

[0014] Based on the heterogeneous network data, a two-layer graph attention mechanism is used to determine the type-level information corresponding to the type of each node in the heterogeneous network, and based on the type-level information corresponding to the type of each node, the node-level information is determined.

[0015] Obtain the pre-determined structural information of each node; the structural information includes the association information between each node;

[0016] Based on the node-level information and the structural information, the target attention information between each node is determined to determine the intermediate node representation of the output of the first neural network at level t.

[0017] In some embodiments, determining type-level information corresponding to a target type includes:

[0018] Based on the target hidden state information output by the second neural network at level (t-1), the first node representation of each neighboring node adjacent to the specific node is determined;

[0019] Based on the first node representation of each neighboring node adjacent to a specific node, the type of each neighboring node, and the adjacency matrix of the heterogeneous network data, the target type representation of the target type is determined.

[0020] Based on the target type representation and the second node representation of the specific node, determine the type-level attention score corresponding to the target type;

[0021] Based on the type-level attention scores corresponding to each type and the type-level attention score corresponding to the target type, the type-level information corresponding to the target type is determined.

[0022] In some embodiments, the node-level information for determining a target neighbor node adjacent to the specific node includes:

[0023] Based on the first node representation of each neighboring node adjacent to the specific node, the second node representation of the specific node, and the type-level information corresponding to the type of each neighboring node, the node-level attention score of the target neighboring node is determined;

[0024] The node-level information of the target neighbor node is determined based on the node-level attention scores of each neighbor node adjacent to the specific node and the node-level attention score of the target neighbor node.

[0025] In some embodiments, the steps for determining the structural information of each node include:

[0026] The heterogeneous network is homogenized to obtain a homogeneous network;

[0027] Based on the third node representation of each node in the homogeneous network, the structural similarity between each node in the homogeneous network is determined by a preset similarity algorithm to obtain the structural-level information of each node.

[0028] In some embodiments, determining the target attention information between nodes based on the node-level information and the structure-level information includes:

[0029] Obtain the first fusion weight of the node-level information and the second fusion weight of the structure-level information;

[0030] Based on the node-level information, the first fusion weight of the node-level information, the structure-level information, and the second fusion weight of the structure-level information, the target attention information between each node is determined.

[0031] In some embodiments, determining the intermediate node representation of the output of the first neural network at level t includes:

[0032] Based on the target hidden state information output by the second neural network at level (t-1), determine the node representation matrix corresponding to each type;

[0033] Based on the target attention information between nodes and the node representation matrix corresponding to each type, the intermediate node representation of the output of the first neural network at level t is determined; the node representation matrix includes the node representation of each node under the corresponding type.

[0034] In some embodiments, the second neural network is a gated recurrent unit (GRU).

[0035] Determine the target hidden state information output by the second neural network at level t, including:

[0036] Based on the intermediate node information received by the GRU at level t from the output of the first neural network at level t and the target hidden state information output by the GRU at level (t-1), the update gate data and reset gate data of the GRU at level t are determined.

[0037] Based on the reset gate data, the intermediate node information output by the first neural network at level t, and the target hidden state information output by the GRU at level (t-1), the candidate hidden state information of the GRU at level t is determined.

[0038] Based on the candidate hidden state information, the update gate data, and the target hidden state information output by the GRU at level (t-1), the target hidden state information output by the GRU at level t is determined.

[0039] In some embodiments, determining the training data based on the heterogeneous network data and the first network representation includes:

[0040] A preset number of node pairs are selected from the heterogeneous network, and based on the heterogeneous network data, it is determined whether there is a correlation between the nodes in the node pairs, and a preset label is set for the node pairs with the correlation.

[0041] Based on the first network representation, labeled node pairs are determined and used as training data.

[0042] In some embodiments, the step of training the logistic regression model and the target neural network based on the training data, and then using the second network representation output by the target neural network as the target network representation for the target domain task after the training of the logistic regression model and the target neural network is completed, includes:

[0043] The training data is input into the logistic regression model to perform link prediction, and the link prediction results are obtained.

[0044] Based on the link prediction results, a weighted loss value is constructed, and the logistic regression model and the target neural network are trained by weighted backpropagation of the weighted loss value until the weighted loss value converges.

[0045] When the logistic regression model and the target neural network are trained, the second network representation output by the target neural network is used as the target network representation for the target domain task.

[0046] Secondly, this disclosure also provides a network representation learning device, including: a data acquisition module, a target neural network, a sample determination module, and a logistic regression model; the target neural network includes an N-level first neural network and an N-1-level second neural network; wherein, the intermediate node output by the t-th level first neural network is represented as the input of the corresponding t-th level second neural network; the target hidden state information output by the (t-1)-th level second neural network is the input of the t-th level first neural network and the input of the t-th level second neural network, respectively; N≥2, 2≤t≤N-1, and N and t are both rounded down;

[0047] The data acquisition module is configured to acquire heterogeneous network data in the target domain; the heterogeneous network data includes data from each node in the heterogeneous network.

[0048] The target neural network is configured to receive the heterogeneous network data, process it through the first neural network at each level and the second neural network at each level, and output a first network representation;

[0049] The sample determination module is configured to determine training data based on the heterogeneous network data and the first network representation;

[0050] The logistic regression model is configured to train itself and the target neural network based on the training data. When the training of itself and the target neural network is completed, the second network representation output by the target neural network is used as the target network representation of the target domain task.

[0051] Thirdly, embodiments of this disclosure also provide a computer device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the network representation learning method as described in any of the above embodiments are performed.

[0052] Fourthly, embodiments of this disclosure also provide a computer non-transient readable storage medium, wherein a computer program is stored on the computer non-transient readable storage medium, and the computer program is executed by a processor to perform the steps of the network representation learning method as described in any of the above embodiments. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the network architecture of the target neural network provided in an embodiment of the present disclosure;

[0054] Figure 2A schematic flowchart illustrating a network representation learning method provided in an embodiment of this disclosure;

[0055] Figure 3 An exemplary network integration diagram provided for embodiments of this disclosure;

[0056] Figure 4 A schematic diagram of a network representation learning device provided in an embodiment of this disclosure;

[0057] Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present disclosure. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0059] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0060] In this disclosure, "multiple or several" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0061] In related technologies, existing heterogeneous network representation learning methods can be mainly divided into two categories: one is based on random walk methods, such as machine learning methods using algorithms like matappath2vec, HERec, and HIN2vec; the other is based on graph neural networks, mainly methods based on heterogeneous graph attention networks, such as methods combining graph attention mechanisms with HAN, HGAT, HetGNN, and MAGNN. Methods based on heterogeneous graph attention networks follow the message-passing paradigm, that is, the paradigm of aggregating neighbor node information to update the central node information. Generally, it is set to T iterations, with each iteration performing information aggregation and updating on all nodes once.

[0062] However, the inventors discovered at least the following problems in the related technologies: 1) Existing graph attention networks only consider the first-order neighborhood of a node when aggregating node information, failing to fully utilize the structural information in the graph, as the structural information of the nodes in the neighborhood is lost during the node information aggregation process. 2) If higher-order neighborhoods are considered for deep stacking, it will lead to oversmoothing. The reason is that when graph attention networks aggregate node information, they are essentially aggregating the information of neighboring nodes. For any node in the graph, each time the node's features are updated, the information of higher-order neighboring nodes is aggregated. If the order of the highest-order neighboring node is called the aggregation radius of the node, it can be found that as the number of layers in the graph attention network increases, the aggregation radius of the node also increases. Once a certain threshold is reached, the nodes covered by the node are almost identical to the nodes in the entire graph, which is oversmoothing. 3) In related technologies, only the result of information aggregation in the previous layer can be used as the input of the next layer to obtain a new aggregation result, without considering the fusion of graph features extracted by different information aggregation layers.

[0063] Based on this, this disclosure specifically provides a network representation learning method that essentially eliminates one or more of the problems caused by the limitations and defects of related technologies. Specifically, this network representation learning method utilizes a pre-built target neural network to learn network representations for the target domain. Figure 1 This is a schematic diagram of the network architecture of the target neural network provided in the embodiments of this disclosure, such as... Figure 1As shown, each of the first N-1 neural networks (excluding the Nth level first neural network) corresponds one-to-one with each of the first N-1 level second neural networks; the intermediate node output by the t-th level first neural network represents the input of its corresponding t-th level second neural network; the target hidden state information output by the (t-1)-th level second neural network is the input of the t-th level first neural network and the input of the t-th level second neural network, respectively; N≥2, 2≤t≤N-1, and N and t are both rounded.

[0064] This disclosure provides a network representation learning method, which will be described in detail below. Figure 2 This is a flowchart illustrating a network representation learning method provided in an embodiment of the present disclosure, as shown below. Figure 2 As shown, steps S11 to S14 are included:

[0065] S11. Obtain heterogeneous network data in the target domain.

[0066] Heterogeneous network data includes data from each node in the heterogeneous network.

[0067] In this step, the target domain can be, for example, the biological domain. The target domain task can include link prediction tasks in the biological domain, such as predicting co-pathological associations between disease nodes, mining disease-related genes, and mining potential associations between diseases and miRNAs. Alternatively, the target domain task can also include link prediction tasks in other domains, such as classifying and predicting nodes in heterogeneous networks, classifying and predicting heterogeneous networks, and predicting association relationships between nodes in heterogeneous networks. This disclosure does not specifically limit these tasks, nor will it list them all.

[0068] For example, the embodiments disclosed herein can be applied to biological network representation learning in the biological field. For ease of understanding, the following description uses a heterogeneous network as an example, which is a heterogeneous network integrated from multiple existing biological networks in the biological field.

[0069] Heterogeneous network data includes data from each node in the heterogeneous network, mainly including the content information and structural information of the nodes. The content information includes the feature vector of the node itself, and the structural information includes the relationship information between the nodes.

[0070] S12. Input the heterogeneous network data into the target neural network. After processing by the first neural network at each level and the second neural network at each level, the first network representation output by the target neural network is obtained.

[0071] like Figure 1As shown, heterogeneous network data G is input into the target neural network. Specifically, the heterogeneous network data is input into the first-level first neural network and the first-level second neural network, respectively. The first-level first neural network performs logical processing based on the heterogeneous network data G and outputs the intermediate node representation H of the heterogeneous network data G. 1 After that, the intermediate node represents H. 1 As the input to the first-level second neural network, the first-level second neural network is based on the intermediate node representation H. 1 Perform logical processing and output the target hidden state information h. 1 And so on, the first neural network of level t is based on the target hidden state information h. t-1 Perform logical processing and output the intermediate node representation H. t The second neural network at level t is based on the intermediate node representation H. t The target hidden state information h output by the second neural network at level (t-1) t-1 Perform logical processing and output the target hidden state information h. t Finally, the Nth-level first neural network is based on the target hidden state information h. N-1 Perform logical processing and output the first network representation H. N The first network represents H. N It is calculated based on the following formula Y. The first network representation H... N This includes the node representation of each node.

[0072] S13. Based on heterogeneous network data and the first network representation, determine the training data.

[0073] In this step, a subset of node pairs are selected from the heterogeneous network data G. For a given pair of nodes P... (i,j) According to P (i,j) Determine if an edge exists between nodes, and label it with 1 or 0 to obtain the label of the corresponding node pair, based on the first network representation H. N Labeled node pairs are used as training data.

[0074] S14. Based on the training data, train the logistic regression model and the target neural network. When the logistic regression model and the target neural network are trained, the second network representation output by the target neural network is used as the target network representation for the target domain task.

[0075] Specifically, by using the training data as input to the logistic regression model, the cross-entropy loss function can be used to calculate the loss of the entire model, including the logistic regression model and the target neural network. Based on the calculated loss, the parameters are updated through backpropagation to obtain the optimal result.

[0076] It should be noted that the above S12 to S14 processes are cyclical processes, that is, the logistic regression model and the target neural network are trained based on the training data; then, based on the calculated loss, the learnable parameters in the logistic regression model and the target neural network are updated through backpropagation; then, a new round of cyclic process is carried out, that is, based on the updated logistic regression model and the target neural network, S12 is executed to obtain a new first network representation, S13 is executed to obtain new training data, and S14 is executed based on the new training data to continue training the logistic regression model and the target neural network, and so on, until the logistic regression model and the target neural network are trained.

[0077] This embodiment applies a second neural network to the layer surface. By setting up a one-to-one correspondence between a first neural network and a second neural network, the intermediate node representation output by the t-th level first neural network is used as the input to its corresponding t-th level second neural network. The target hidden state information output by the (t-1)-th level second neural network is the input to both the t-th level first neural network and the t-th level second neural network, respectively. This achieves the fusion of multi-level graph features (i.e., the intermediate node representation output by each level of the first neural network), and uses the fused multi-level graph features to update the node representation of the entire graph. This complements the advantages of graph features from different levels, overcoming the shortcomings of related technologies that do not consider the fusion of graph features from different levels. Simultaneously, maintaining the graph features obtained by the low-level second neural network in the high-level graph neural network can alleviate the oversmoothing problem of the second neural network to some extent.

[0078] In this embodiment, the first neural network can be a Heterogeneous Graph Attention Networks Introducing Structural Information (ISHGAT). The second neural network can be a Gated Recurrent Unit (GRU).

[0079] The following is a detailed description of each step in steps S11 to S14 above.

[0080] Specifically, for step S11, at least one network under a target domain can be obtained from a pre-set database. The network under the target domain contains various types of nodes. Based on the type of each node in the network under each target domain, the obtained networks under at least one target domain are integrated to construct a heterogeneous network corresponding to the target domain task, and heterogeneous network data is obtained.

[0081] Taking the biological field as an example, the pre-set database here can be a public database containing various biological networks, such as the Pharmacogenetics and Pharmacogenomics Knowledge Base (PharmGKB), the Human Protein Resource Database (HPRD), the experimentally validated miRNA target gene database (MicroRNA-Target Interactions (MTI), also known as the miRTarBase database and the miR2Disease database.

[0082] Biological networks can include at least four types of networks, such as disease-gene association networks, gene-gene interaction networks, gene-microRNA association networks, and miRNA-disease association networks.

[0083] Specifically, disease-gene association networks can be obtained from the PharmGKB database; protein-protein interaction networks can be obtained from the HPRD database, and since the nodes in the protein-protein interaction network essentially correspond to genes, the protein-protein interaction network can be transformed into a gene-gene interaction network; gene-miRNA association networks can be obtained from the miRTarBase database; and miRNA-disease association networks can be obtained from the miR2Disease database. These four biological networks contain three types of nodes: disease, gene, and miRNA nodes. Then, using nodes of the same type as links, the four biological networks are integrated to form a heterogeneous network.

[0084] For example, Figure 3 An exemplary network integration diagram provided for embodiments of this disclosure, such as... Figure 3 As shown, A represents a disease, B represents a gene, and C represents a miRNA. A disease type may contain multiple different nodes, such as nodes A1, A2, and A3; a gene type may contain multiple different nodes B1 and B2; and a miRNA type may contain multiple different nodes C1 and C2. By integrating identical nodes from disease-gene association networks, gene-gene interaction networks, gene-miRNA association networks, and miRNA-disease association networks into a single node, connections between different biological networks can be achieved.

[0085] The aforementioned biological network includes three types of nodes: diseases, genes, and miRNAs. In one possible implementation, this embodiment of the disclosure can use the word embedding vector of a disease in a medical encyclopedia as the feature vector of the disease node. The sequence information of a gene is encoded into a vector using Kerr as the basic unit, serving as the feature vector of the gene node. Similarly, the sequence information of miRNAs is encoded into a vector using Kerr as the basic unit, serving as the feature vector of the miRNA node. The feature vectors of each node in the integrated heterogeneous network are used as the feature vectors of the nodes in the heterogeneous network data G.

[0086] Regarding step S12, for heterogeneous network data G, given a specific node, denoted as node v, neighboring nodes of different types may have different effects on this specific node v. Neighboring nodes of the same type may also have different importance; therefore, it is necessary to capture the different importance at both the node and type levels. This embodiment employs a two-layer graph attention mechanism, namely type-level attention and node-level attention, to calculate the final attention score of the first neural network, i.e., the target attention information θ. ij .

[0087] In specific implementation, we will take the determination of the intermediate node representation of the output of one level of the first neural network in a multi-level first neural network as an example, for instance, determining the intermediate node representation of the output of the t-th level first neural network. It should be noted that for the processing of the first-level first neural network, since there is no preceding second neural network, the input data of the first-level first neural network is heterogeneous network data G. The processing principle of each level of the first neural network is the same; for the processing of the first-level first neural network, please refer to the following explanation of the processing of the t-th level first neural network, and repeated parts will not be elaborated upon.

[0088] The following explanation uses the processing of the first neural network at level t as an example. The input data of the first neural network at level t is the output data of the second neural network at level (t-1). See [link to documentation] for details. Figure 1 The data transmission process is shown. Determining the intermediate node representation of the output of the first neural network at level t specifically includes the following steps S12-1 to S12-3:

[0089] S12-1. Based on heterogeneous network data, a two-layer graph attention mechanism is adopted to determine the type-level information corresponding to the type of each node in the heterogeneous network, and based on the type-level information corresponding to the type of each node, the node-level information of each node is determined.

[0090] In this step, the first neural network at level t receives the target hidden state information h output by the second neural network at level (t-1). t-1It should be noted that the inputs to both the first and second neural networks at each level are essentially derived from the inputs in the target neural network through logical processing. Therefore, the target hidden state information h output by the (t-1)th level second neural network is obtained based on heterogeneous network data. t-1 The first neural network at level t receives the target's hidden state information h. t-1 It employs a two-layer graph attention mechanism. One layer is used to learn the weights of different types of neighbor nodes, that is, to determine type-level information; the other layer is used to capture the importance of different neighbor nodes, that is, to determine node-level information based at least on the determined type-level information.

[0091] Type-level information represents the influence of the types of neighboring nodes u adjacent to a specific node v on that specific node v. Node-level information represents the influence of a neighboring node u adjacent to a specific node v on that specific node.

[0092] Nodes in a heterogeneous network include various different types, denoted as E. Determining the type-level information corresponding to a target type e specifically includes steps S201–S204:

[0093] S201. Based on the target hidden state information output by the (t-1)th level second neural network, determine the first node representation of each neighboring node adjacent to the specific node.

[0094] In this step, the target's hidden state information h t-1 This includes the node representation of each node. For a given node v, determine its neighboring nodes m, forming a neighbor set N. m .

[0095] Target hides hidden state information h t-1 The first node representation of neighbor node m is included, denoted as h. m .

[0096] S202. Based on the first node representation of each neighboring node adjacent to the specific node, the type of each neighboring node, and the adjacency matrix of the heterogeneous network data, determine the target type representation of the target type.

[0097] Based on the types of each neighbor node m of a specific node v, select neighbor nodes of type e from each neighbor node m of the specific node v to form a set of neighbor nodes of type e. Neighbor node set The first node of a neighbor node u in the algorithm is represented by h. u The target type e is represented by the first node of the neighbor node u of a specific node v, which is of type e. uThe sum. The target type representation h of target type e is determined according to the following formula. e , specifically:

[0098]

[0099] in, It is a normalized adjacency matrix, where A′=A+I represents the adjacency matrix containing self-connections of a specific node v, A represents the adjacency matrix of heterogeneous network data G, and I is the identity matrix. M represents the degree matrix of all nodes.

[0100] S203. Based on the target type representation and the second node representation of a specific node, determine the type-level attention score corresponding to the target type.

[0101] Target hidden state information h t-1 This includes the node representations of each node, specifically the second node representation h of a particular node v. v The type-level attention score a corresponding to the target type e is determined according to Formula 2 below. e :

[0102] a e =σ(μ) e ·[h v ||h e Formula 2

[0103] Where || denotes the connection between two representation vectors; μ e σ represents the learnable parameters corresponding to type-level attention, i.e., the attention vector parameters for target type e; σ represents the activation function. Formula 2 essentially uses the learnable parameters μ... e And the activation function σ, calculate two representation vectors (i.e., the second node representation h). v With target type representation h e The similarity between ).

[0104] S204. Based on the type-level attention scores corresponding to each type and the type-level attention scores corresponding to the target type, determine the type-level information corresponding to the target type.

[0105] The type-level attention score a for each type (i.e., each type in e′∈E) e′ The type-level attention score a corresponding to target type e e In practice, the normalized exponential function softmax can be used to normalize the type-level attention scores for all types, thus determining the type-level information corresponding to each type. Formula 3 below determines the type-level information α corresponding to the target type e. e The same method applies to determining the type-level information for other types.

[0106]

[0107] To determine the node-level information of a target neighbor node adjacent to a specific node v, the specific steps include S301 to S302:

[0108] S301. Based on the first node representation of each neighboring node adjacent to the specific node, the second node representation of the specific node, and the type-level information corresponding to the type of each neighboring node, determine the node-level attention score of the target neighboring node.

[0109] The target neighbor node is any one of the neighbor nodes adjacent to the specific node. This embodiment of the disclosure will be described in detail by taking the determination of the node-level information of one of the neighbor nodes adjacent to the specific node as an example. The determination of the node-level information of each other neighbor node among the neighbor nodes adjacent to the specific node is the same as in steps S301 to S302, and the process is repeated and will not be described again.

[0110] Specifically, given a specific node v of type e, the first node representation h of each neighboring node m adjacent to the specific node v can be determined according to S201. m Forming a set of neighboring nodes N m Based on the type-level information corresponding to the types of each neighbor node, determine the type-level information corresponding to the type of the target neighbor node. Based on the type-level information corresponding to the type of the target neighbor node, the first node representation of the target neighbor node, and the second node representation of the specific node, determine the node-level attention score of the target neighbor node.

[0111] For a target neighbor node m′ adjacent to a specific node, the node-level attention score b of the target neighbor node m′ can be determined according to the following formula 4. vm′ :

[0112] b vm′ =σ(γ·α) e″ [h v ||h m′ Formula 4

[0113] Among them, h v The second node representing a specific node v is h. v h m′ Let α represent the first node representation of the target neighbor node m′. || denotes the connection between two representation vectors. e″The type-level information corresponding to the type e″ of the target neighbor node m′ is represented. γ is the learnable parameter corresponding to the node-level attention, that is, the attention vector parameter. σ is the activation function. The above process is essentially achieved through the learnable parameters γ and μ. e The similarity between two nodes is calculated using the function σ.

[0114] S302. Based on the node-level attention scores of each neighboring node adjacent to the specific node and the node-level attention score of the target neighboring node, determine the node-level information of the target neighboring node.

[0115] Given the set N of neighboring nodes that are adjacent to a specific node v. m In practice, the softmax function can be used to normalize the node-level attention scores of each neighboring node to determine the node-level information corresponding to each neighboring node.

[0116] Taking the determination of the node-level information of the target neighbor node as an example, the following formula (5) determines the node-level information β of the target neighbor node m′. vm′ The same method applies to determining the type-level information for other neighboring nodes.

[0117]

[0118] The above step S12-1 calculates the content information of each node based on the two-layer graph attention mechanism. This embodiment of the present disclosure further introduces the structural information of the nodes based on the above-described determination of their content information, as shown in step S12-2.

[0119] S12-2. Obtain the pre-determined structural information of each node; the structural information includes the relationship information between each node.

[0120] Specifically, the heterogeneous network is first homogenized to obtain a homogeneous network. Based on the third-node representation of each node in the homogeneous network, a preset similarity algorithm is used to determine the structural similarity between nodes in the homogeneous network, thereby obtaining the structural-level information of each node.

[0121] In one possible implementation, a heterogeneous network is homogenized to obtain a homogeneous network. Then, the node2vec model can be used to obtain a representation of each node containing structural information, i.e., a third-node representation, on the homogenized network. Here, the node2vec model has two parameters, p and q, which can be used to control the tendency of random walks. Specifically, a larger p makes it less likely for the node2vec model to return to the previously visited node, encouraging it to walk further; a smaller p makes it easier to return to the previous node, causing it to tend to explore around the starting point. Similarly, a larger q makes the node2vec model more inclined to explore around the previous node, while a smaller q makes it more inclined to explore nodes farther away from the previous node. Therefore, using the node2vec model allows it to capture structural similarities between nodes in a wider area. Then, the similarity of the third node representation (i.e., the representation containing structural information) of each node can be calculated using the cosine similarity function, resulting in the structural similarity matrix S, which represents the structural-level information of each node. Each element s in the matrix... ij This represents the structural information similarity between node i and node j.

[0122] S12-3. Based on node-level and structure-level information, determine the target attention information between each node to determine the intermediate node representation of the output of the first neural network at level t.

[0123] Determine the target attention information between nodes; specifically, this can be achieved by using node-level information β representing content information similarity. ij and structural-level information s representing structural information similarity. ij Perform a weighted summation, and use the result as the final target attention information θ between node i and node j. ij .

[0124] In one possible implementation, a first fusion weight of node-level information and a second fusion weight of structural information are obtained; based on node-level information β... ij The first fusion weight λ for node-level information and the structural information s ij The second fusion weight (1-λ) of the structural information is used to determine the target attention information θ between nodes. ij For details, please refer to Formula Six below:

[0125] θ ij =λβ ij +(1-λ)s ij Formula Six

[0126] Here, the first fusion weight and the second fusion weight can be learned based on the prediction model, or they can be obtained based on experience in actual application scenarios. In this embodiment, the specific fusion weight values ​​are not limited.

[0127] Based at least on the target attention information between nodes determined in S12-3, the intermediate node representations of the first neural network output are determined. The following detailed explanation uses the determination of the intermediate node representations of the first neural network output at level t as an example. Specifically, based on the target hidden state information output by the second neural network at level (t-1), the node representation matrices corresponding to each type are determined. Here, the node representation matrix includes the node representations of each node under the corresponding type. Target hidden state information h t-1 The matrix includes node representations of each node, from which we can obtain the node representation matrices corresponding to each type of the node (i.e., each type in e′∈E). This includes node representations (i.e., feature vectors) of each node of type e′, where each row of data represents the feature vector of a node of type e′. Then, based on the target attention information θ between nodes... ij And the node representation matrices corresponding to each type. Determine the intermediate node representation H of the output of the first neural network at level t. t See Formula 7 below for details; intermediate nodes represent H. t The node representation matrix is ​​obtained by aggregating the corresponding node representation matrices using learnable transformation matrices corresponding to different types.

[0128]

[0129] Among them, the target attention information θ between each node ij Attention matrices can be formed θ represents the target attention information between nodes of type e′. ij The attention matrix formed by these components. This is the learnable transformation matrix corresponding to type e′. It should be noted that when t=1, the initial value... The node representation matrix, also known as the node feature vector matrix, represents the node of type e′ in the heterogeneous network data G, containing the feature vectors of each node.

[0130] This embodiment employs a two-layer graph attention mechanism to determine the content information of nodes, and introduces structural information into the nodes based on the content information. That is, structural information similarity is added to the content information similarity as the final target attention information. In this embodiment, the first neural network can be a heterogeneous graph attention network (HGAT) with added structural information, i.e., a graph neural network. By using the heterogeneous graph attention network with added structural information as each level in a multi-level graph neural network to aggregate node information at each layer, the problem of losing node structural information in the neighborhood is solved in related technologies.

[0131] In this embodiment, the first neural network is a heterogeneous graph attention network with added structural information, i.e., a graph neural network, and the second neural network is a gated recurrent unit (GRU). This embodiment introduces GRU units between different levels of the graph neural network to form a multi-level graph neural network (i.e., a target neural network), achieving the fusion of graph features from different levels. The GRU unit receives intermediate node information output from the graph neural network at its corresponding level, as well as target hidden state information output from the GRU unit at the previous level, and outputs the target hidden state information at the current level. The specific execution process is described in steps S401-S403.

[0132] S401. Based on the intermediate node information received by the t-th level GRU from the output of the first neural network at level t and the target hidden state information output by the (t-1)-th level GRU, determine the update gate data and reset gate data of the t-th level GRU.

[0133] Update gate u in the t-th level GRU unit t and reset door r t The calculation process is shown in Formulas 8 and 9:

[0134] u t =ρ(W u [H t ,h t-1 ]+b u Formula Eight

[0135] r t =ρ(W r [H t ,h t-1 ]+b r Formula Nine

[0136] Among them, u t This indicates the update gate data, controlling how much information from level (t-1) and level t should be passed to future levels, such as subsequent levels (i.e., levels (t+1) to N); r tThis indicates resetting the gate data, controlling how much past information to forget, such as information from previous levels; [] indicates concatenating two vectors; W u W represents a learnable parameter in the update gate computation process within a GRU unit. r H represents a learnable parameter representing the reset gate computation process in a GRU unit; t h represents the intermediate node representation of the output of the first neural network at level t; t-1 ρ represents the target hidden state information output by the (t-1)th level GRU; ρ represents the simoid function, which can transform data into values ​​in the range of 0-1, thus serving as a gating signal; b u This indicates the bias in the update gate calculation process; b r This indicates the bias during the gate calculation process.

[0137] Here, the t-th level GRU unit obtains the control states of the update gate and the reset gate by acquiring the intermediate node information of the output of the first neural network at the t-th level and the target hidden state information of the output of the (t-1)-th level GRU.

[0138] It should be noted that if t=1, then the initial value h0 is the heterogeneous network data G, which includes the node representation of each node.

[0139] S402. Based on the reset gate data, the intermediate node information output by the first neural network at level t, and the target hidden state information output by the GRU at level (t-1), determine the candidate hidden state information of the GRU at level t.

[0140] The candidate hidden state information c of the t-th level GRU t For the calculation process, please refer to Formula 10:

[0141] c t =tanh(W c [H t ,(r t ×h t-1 )]+b c Formula 10

[0142] Among them, c t This represents the candidate hidden state information, containing the intermediate node representation H of the input to the first neural network at level t. t And targeted processing of the target hidden state information h input to the (t-1)th level second neural network t-1 The is retained; tanh is the hyperbolic tangent function; × represents the product of two vectors; W c b represents a learnable parameter used in the computation of candidate hidden state information within a GRU unit; cThis represents the bias used when calculating candidate hidden state information.

[0143] S403. Based on the candidate hidden state information, the updated gate data, and the target hidden state information output by the (t-1)th level GRU, determine the target hidden state information output by the t-th level GRU.

[0144] The target hidden state information h output by the t-th level GRU t For the calculation process, please refer to Formula 11:

[0145] h t =(1-u t )×h t-1 +u t ×c t Formula 11, following the steps outlined above, processes the first and second neural networks at each level of the target neural network. Ultimately, the entire target neural network can be represented as follows: Among them, the normalized adjacency matrix The initial feature vector matrix X of all nodes is used as the input to the target neural network model; μ e γ and W are the learnable parameters corresponding to type-level and node-level attention, respectively. t W represents the transformation matrix parameters of the learnable first-level neural network. u W r W c These are the learnable parameters of the GRU unit.

[0146] Specifically, for step S13, a predetermined number of node pairs can be selected from the heterogeneous network. Based on the heterogeneous network data, it can be determined whether there is a correlation between the nodes in the node pairs, and the node pairs with correlation can be assigned a predetermined label. Based on the first network representation, labeled node pair data is determined, and the labeled node pair data is used as training data.

[0147] After obtaining the first network representation H of the target neural network output... N In the case of randomly selecting a certain number of node pairs from heterogeneous network data G, for a given pair of nodes P... (i,j) According to P (i,j) Determine if an edge exists between nodes, and label it with 1 or 0 to obtain the label of the corresponding node pair, based on the first network representation H. N The node representations of each node are used as training data, that is, the node representations of labeled node pairs are used as training data and input into the logistic regression model to train the logistic regression model and the target neural network.

[0148] The entire large model, consisting of the logistic regression model and the target neural network, can be specifically represented as follows: Where ω represents the learnable parameters of the logistic regression model, and Z represents the prediction result of the entire model.

[0149] For step S14, model training, specifically, the training data is input into the logistic regression model to perform link prediction, and the link prediction result Z is obtained; based on the link prediction result, a weighted loss value is constructed, and the logistic regression model and the target neural network are trained by weighted backpropagation on the weighted loss value until the weighted loss value converges; when the logistic regression model and the target neural network are trained, the second network representation output by the target neural network is used as the target network representation of the target domain task.

[0150] For example, during training, the cross-entropy loss function Loss can be used to construct a weighted loss value based on the link prediction result Z, for example... Where n is the number of training samples, t lk Z is a sign function; it takes the value 1 if the true type of the l-th sample is k, and 0 otherwise. lk This represents the output of the logistic regression model, which is the probability that the l-th sample belongs to type k.

[0151] Here, the entire model, composed of the logistic regression model and the target neural network, constructs a weighted loss value Loss based on the calculated link prediction result Z, and updates the parameters through backpropagation, specifically updating μ. e , γ, W t W u W r W c The weighted loss values ​​are calculated and adjusted until they converge, yielding the optimal model output Z. At this point, the second network representation output by the target neural network is the target network representation for the target domain task.

[0152] For example, taking link prediction in the biological domain as the target task, the process begins with learning biological network representations. Specifically, this involves: first, acquiring and constructing a heterogeneous biological network; then, designing a multi-level graph neural network model, i.e., the target neural network (multi-level heterogeneous graph attention networks with added structural information + multi-level GRU units); next, training the target neural network and a logistic regression model using training data. The second network representation output by the target neural network after training is complete is then used as the target network representation for the target domain task. After obtaining the target network representation for the target domain task, this representation can be applied to the task.

[0153] Based on the same inventive concept, this disclosure also provides a network representation learning device corresponding to the network representation learning method. Since the principle of the network representation learning device in this disclosure is similar to that of the network learning method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0154] Figure 4 This is a schematic diagram of a network representation learning device provided in an embodiment of the present disclosure. In the above embodiment, the large model composed of the target neural network 42 and the logistic regression model 44 is integrated on the network representation learning device, such as... Figure 4 As shown, the network representation learning device includes a data acquisition module 41, a target neural network 42, a sample determination module 43, and a logistic regression model 44. The target neural network 42 includes an N-level first neural network and an N-1-level second neural network; the intermediate node output by the t-th level first neural network represents the input of its corresponding t-th level second neural network; the target hidden state information output by the (t-1)-th level second neural network is the input of the t-th level first neural network and the input of the t-th level second neural network, respectively; N≥2, 2≤t≤N-1, and N and t are both rounded down.

[0155] The data acquisition module 41 is configured to acquire heterogeneous network data in the target domain; the heterogeneous network data includes the data of each node in the heterogeneous network; the target neural network 42 is configured to receive the heterogeneous network data, process it through the first neural network at each level and the second neural network at each level, and output the first network representation; the sample determination module 43 is configured to determine the training data based on the heterogeneous network data and the first network representation; the logistic regression model 44 is configured to train itself and the target neural network 42 based on the training data, and use the second network representation output by the target neural network 42 when it is trained as the target network representation of the target domain task.

[0156] The network representation learning apparatus provided in this disclosure is used for learning network representations in a target domain. The learned target network representation can be applied to target neighborhood tasks. Specifically, the architecture of the network model integrated in the network representation learning apparatus involves applying a second neural network to the layer plane. By setting up a one-to-one correspondence between the first and second neural networks, the intermediate node representation output by the t-th level first neural network is used as the input to its corresponding t-th level second neural network. The target hidden state information output by the (t-1)-th level second neural network is the input to both the t-th level first neural network and the t-th level second neural network, respectively. This achieves the fusion of multi-level graph features (i.e., the intermediate node representations output by each level of the first neural network), and uses the fused multi-level graph features to update the node representation of the entire graph. This complements the advantages of graph features at different levels, overcoming the shortcomings of related technologies that do not consider the fusion of graph features at different levels. Simultaneously, maintaining the graph features obtained by the low-level second neural network in the high-level graph neural network can alleviate the oversmoothing problem of the second neural network to some extent.

[0157] In some embodiments, the data acquisition module 41 is specifically configured to acquire at least one network under a target domain from a pre-set database; the network under the target domain contains multiple different types of nodes; based on the type of each node in the network under each target domain, the acquired networks under at least one target domain are integrated to build a heterogeneous network corresponding to the target domain task, and heterogeneous network data is obtained. The specific implementation process of this data acquisition module 41 can be found in the above description of step S11; repeated parts will not be described again.

[0158] In some embodiments, taking the determination of the intermediate node representation of the output of the first neural network at level t in the target neural network 42 as an example, the first neural network at level t is specifically configured to use a two-layer graph attention mechanism based on heterogeneous network data to determine the type-level information corresponding to the type of each node in the heterogeneous network, and to determine the node-level information based on the type-level information corresponding to the type of each node; to obtain the pre-determined structural information of each node; the structural information includes the relationship information between each node; and to determine the target attention information between each node based on the node-level information and the structural information, so as to determine the intermediate node representation of the output of the first neural network at level t. The specific implementation process of this first neural network at level t can be referred to the above description of steps S12-1 to S12-3, and the repeated parts will not be repeated.

[0159] Optionally, for the first neural network at level t, it is configured to determine type-level information corresponding to a target type. Specifically, based on the target hidden state information output by the second neural network at level (t-1), the first node representation of each neighboring node adjacent to the specific node is determined; based on the first node representation of each neighboring node adjacent to the specific node, the type of each neighboring node, and the adjacency matrix of the heterogeneous network data, the target type representation of the target type is determined; based on the target type representation and the second node representation of the specific node, the type-level attention score corresponding to the target type is determined; based on the type-level attention scores corresponding to each type and the type-level attention score corresponding to the target type, the type-level information corresponding to the target type is determined.

[0160] Optionally, for the first neural network of level t, it is configured to determine the node-level information of a target neighbor node adjacent to a specific node. Specifically, based on the first node representation of each neighbor node adjacent to the specific node, the second node representation of the specific node, and the type-level information corresponding to the type of each neighbor node, the node-level attention score of the target neighbor node is determined; based on the node-level attention scores of each neighbor node adjacent to the specific node and the node-level attention score of the target neighbor node, the node-level information of the target neighbor node is determined.

[0161] Optionally, the network representation learning device further includes a network isomorphism module, which is configured to isomorphize heterogeneous networks to obtain isomorphic networks. Based on the third node representation of each node in the isomorphic network, a preset similarity algorithm is used to determine the structural similarity between each node in the isomorphic network to obtain the structural-level information of each node. For the t-th level first neural network, it is configured to determine the target attention information between each node. Specifically, it obtains the first fusion weight of node-level information and the second fusion weight of structural-level information. Based on the node-level information, the first fusion weight of node-level information, the structural-level information, and the second fusion weight of structural-level information, the target attention information between each node is determined.

[0162] Optionally, for the first neural network of level t, it is configured to output intermediate node representations. Specifically, based on the target hidden state information output by the second neural network of level (t-1), the node representation matrix corresponding to each type is determined; based on the target attention information between each node and the node representation matrix corresponding to each type, the intermediate node representation output by the first neural network of level t is determined; the node representation matrix includes the node representation of each node under the corresponding type.

[0163] Optionally, the second neural network is a gated recurrent unit (GRU). For the t-th level GRU unit, it is configured to output target hidden state information. Specifically, based on the intermediate node information output by the t-th level first neural network and the target hidden state information output by the (t-1)-th level GRU, update gate data and reset gate data are determined. Based on the reset gate data, the intermediate node information output by the t-th level first neural network, and the target hidden state information output by the (t-1)-th level GRU, candidate hidden state information is determined. Based on the candidate hidden state information, update gate data, and the target hidden state information output by the (t-1)-th level GRU, the target hidden state information is determined.

[0164] In some embodiments, the sample determination module 43 is specifically configured to filter out a preset number of node pairs from the heterogeneous network, determine whether there is a correlation between the nodes in the node pairs based on the heterogeneous network data, and assign preset labels to the node pairs with correlation; determine the labeled node pair data based on the first network representation, and use the labeled node pair data as training data. The specific implementation process of this sample determination module 43 can be found in the above description of step S13; repeated parts will not be repeated.

[0165] In some embodiments, the logistic regression model 44 is configured to perform link prediction based on the received training data to obtain link prediction results; based on the link prediction results, a weighted loss value is constructed, and the weighted loss value is used for weighted backpropagation to train itself and the target neural network 42 until the weighted loss value converges; when the logistic regression model 44 and the target neural network are trained, the second network representation output by the target neural network is used as the target network representation for the target domain task. The specific implementation process of this logistic regression model 44 can be found in the above description of step S14; repeated parts will not be repeated.

[0166] This disclosure also provides a computer device. Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present disclosure, such as... Figure 5 As shown, this disclosure provides a computer device including: one or more processors 501, a memory 502, and one or more I / O interfaces 503. The memory 502 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the environmental detection alarm methods described in the above embodiments; the one or more I / O interfaces 503 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0167] Among them, processor 501 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 503 is connected between processor 501 and memory 502, and can realize information interaction between processor 501 and memory 502, including but not limited to data bus (Bus).

[0168] In some embodiments, the processor 501, memory 502, and I / O interface 503 are interconnected via bus 504, and thus connected to other components of the computing device.

[0169] According to embodiments of this disclosure, a non-transient computer-readable medium is also provided. This non-transient computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the environmental detection alarm methods described in the above embodiments.

[0170] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined above in the system of this disclosure.

[0171] It should be noted that the non-transient computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any non-transient computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the non-transient computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0173] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A network representation learning method, characterized in that, This method is applied to the extraction of biological network features in the biological field. It utilizes a pre-built target neural network to learn the network representation of a heterogeneous biological network composed of multiple biological networks, so as to obtain a biological network representation for biological link prediction. The target neural network includes an N-level first neural network and an N-1-level second neural network; wherein, the intermediate node output by the t-th level first neural network is represented as the input of the corresponding t-th level second neural network; the target hidden state information output by the (t-1)-th level second neural network is the input of the t-th level first neural network and the input of the t-th level second neural network, respectively; N≥2, 2≤t≤N-1, and N and t are both rounded down; The network representation learning method includes: Acquire heterogeneous network data in the biological field; the heterogeneous network data includes data of each node in a heterogeneous biological network; wherein, the heterogeneous network data is constructed through the following steps: acquire at least two biological networks containing different types of nodes from a public database of various biological networks, the nodes including diseases, genes and miRNAs; based on the type of each node in each of the biological networks, integrate the at least two biological networks to build a heterogeneous biological network for biological link prediction, and extract the data of each node in the heterogeneous biological network as the heterogeneous network data, the heterogeneous network data including the feature vectors of each node in the heterogeneous biological network and the association relationships between nodes; The heterogeneous network data is input into the target neural network. After processing by the first neural network at each level and the second neural network at each level, a first network representation output by the target neural network is obtained. The first neural network is a heterogeneous graph attention network that incorporates node structure information and is used to aggregate neighbor node information and update node representations. The second neural network is a gated recurrent unit (GRU) used to fuse graph features extracted by the first neural network at different levels. Based on the heterogeneous network data and the first network representation, training data is determined; Based on the training data, the logistic regression model and the target neural network are trained. When the training of the logistic regression model and the target neural network is completed, the second network representation output by the target neural network is used as the biological network representation for biological link prediction.

2. The network representation learning method according to claim 1, characterized in that, Determining the representation of intermediate nodes in the output of the first neural network at level t includes: Based on the heterogeneous network data, a two-layer graph attention mechanism is used to determine the type-level information corresponding to the type of each node in the heterogeneous biological network, and based on the type-level information corresponding to the type of each node, the node-level information is determined. Obtain predetermined structural information for each node; the structural information includes the relationship information between each node. Based on the node-level information and the structure-level information, the target attention information between each node is determined to determine the intermediate node representation of the output of the first neural network at level t.

3. The network representation learning method according to claim 2, characterized in that, To determine the type-level information corresponding to a target type, the following information is included: Based on the target hidden state information output by the second neural network at level (t-1), the first node representation of each neighboring node adjacent to the specific node is determined; Based on the first node representation of each neighboring node adjacent to a specific node, the type of each neighboring node, and the adjacency matrix of the heterogeneous network data, the target type representation of the target type is determined. Based on the target type representation and the second node representation of the specific node, determine the type-level attention score corresponding to the target type; Based on the type-level attention scores corresponding to each type and the type-level attention score corresponding to the target type, the type-level information corresponding to the target type is determined.

4. The network representation learning method according to claim 3, characterized in that, The node-level information for determining a target neighbor node adjacent to the specific node includes: Based on the first node representation of each neighboring node adjacent to the specific node, the second node representation of the specific node, and the type-level information corresponding to the type of each neighboring node, the node-level attention score of the target neighboring node is determined; The node-level information of the target neighbor node is determined based on the node-level attention scores of each neighbor node adjacent to the specific node and the node-level attention score of the target neighbor node.

5. The network representation learning method according to claim 2, characterized in that, The steps for determining the structural information of each node include: The heterogeneous biological network is homogenized to obtain a homogeneous network; Based on the third node representation of each node in the homogeneous network, the structural similarity between each node in the homogeneous network is determined by a preset similarity algorithm to obtain the structural-level information of each node.

6. The network representation learning method according to claim 2, characterized in that, The determination of target attention information between nodes based on the node-level information and the structure-level information includes: Obtain the first fusion weight of the node-level information and the second fusion weight of the structure-level information; Based on the node-level information, the first fusion weight of the node-level information, the structure-level information, and the second fusion weight of the structure-level information, the target attention information between each node is determined.

7. The network representation learning method according to claim 2, characterized in that, The determination of the intermediate node representation of the output of the first neural network at level t includes: Based on the target hidden state information output by the second neural network at level (t-1), determine the node representation matrix corresponding to each type; Based on the target attention information between nodes and the node representation matrix corresponding to each type, the intermediate node representation of the output of the first neural network at level t is determined; the node representation matrix includes the node representation of each node under the corresponding type.

8. The network representation learning method according to claim 1, characterized in that, The second neural network is a gated recurrent unit (GRU). Determine the target hidden state information output by the second neural network at level t, including: Based on the intermediate node information received by the GRU at level t from the output of the first neural network at level t and the target hidden state information output by the GRU at level (t-1), the update gate data and reset gate data of the GRU at level t are determined. Based on the reset gate data, the intermediate node information output by the first neural network at level t, and the target hidden state information output by the GRU at level (t-1), the candidate hidden state information of the GRU at level t is determined. Based on the candidate hidden state information, the update gate data, and the target hidden state information output by the GRU at level (t-1), the target hidden state information output by the GRU at level t is determined.

9. The network representation learning method according to claim 1, characterized in that, The step of determining training data based on the heterogeneous network data and the first network representation includes: A predetermined number of node pairs are selected from the heterogeneous biological network, and based on the heterogeneous network data, it is determined whether there is a correlation between the nodes in the node pairs, and a predetermined label is set for the node pairs with the correlation. Based on the first network representation, labeled node pairs are determined and used as training data.

10. The network representation learning method according to claim 1, characterized in that, The process of training the logistic regression model and the target neural network based on the training data, and then using the second network representation output by the target neural network as the biological network representation for biological link prediction upon completion of training of the logistic regression model and the target neural network, includes: The training data is input into the logistic regression model to perform link prediction, and the link prediction results are obtained. Based on the link prediction results, a weighted loss value is constructed, and the logistic regression model and the target neural network are trained by weighted backpropagation of the weighted loss value until the weighted loss value converges. When the logistic regression model and the target neural network are trained, the second network representation output by the target neural network is used as the biological network representation for biological link prediction.

11. A network representation learning device, characterized in that, This invention relates to biological network feature extraction in the biological field. The device comprises: a data acquisition module, a target neural network, a sample determination module, and a logistic regression model. The target neural network includes an N-level first neural network and an N-1-level second neural network. The intermediate nodes output by the t-th level first neural network represent the inputs of the corresponding t-th level second neural network. The target hidden state information output by the (t-1)-th level second neural network consists of the inputs of the t-th level first neural network and the t-th level second neural network, respectively. N ≥ 2, 2 ≤ t ≤ N-1, and both N and t are rounded down. The data acquisition module is configured to acquire heterogeneous network data in the biological field; the heterogeneous network data includes data of each node in a heterogeneous biological network; wherein, the heterogeneous network data is constructed through the following steps: acquiring at least two biological networks containing different types of nodes from a public database of various biological networks, the nodes including diseases, genes, and miRNAs; integrating the at least two biological networks based on the types of each node in each of the biological networks to build a heterogeneous biological network for biological link prediction, and extracting data of each node in the heterogeneous biological network as the heterogeneous network data, the heterogeneous network data including feature vectors of each node in the heterogeneous biological network and the association relationships between nodes; The target neural network is configured to receive the heterogeneous network data, process it through the first neural network at each level and the second neural network at each level, and output a first network representation; wherein, the first neural network is a heterogeneous graph attention network that incorporates node structure information, used to aggregate neighbor node information and update node representation; the second neural network is a gated recurrent unit (GRU), used to fuse graph features extracted by the first neural network at different levels to alleviate the oversmoothing problem; The sample determination module is configured to determine training data based on the heterogeneous network data and the first network representation; The logistic regression model is configured to train itself and the target neural network based on the training data. When the training of itself and the target neural network is completed, the second network representation output by the target neural network is used as the biological network representation for biological link prediction.

12. A computer device, wherein, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the network representation learning method as described in any one of claims 1 to 10 are performed.

13. A computer-defined non-transient readable storage medium, wherein, The computer non-transient readable storage medium stores a computer program that, when executed by a processor, performs the steps of the network representation learning method as described in any one of claims 1 to 10.

Images