A method for constructing a directed graph embedding representation model, and an embedding representation learning method

Abstract

The application provides a construction method of a directed graph embedding representation model, which comprises the following steps: T1, constructing an initial model, wherein the initial model comprises a feature acquisition module and a topology information aggregation module; T2, obtaining a directed graph, and performing multi-round iterative training on the initial model to obtain a directed graph embedding representation model, wherein each round of iterative training comprises the following steps: T21, acquiring a current round feature representation corresponding to each entity; T22, acquiring a current round topology representation corresponding to each entity; T23, randomly selecting a plurality of entities from all entities of the directed graph as a plurality of positive samples, and constructing a negative sample set corresponding to each positive sample; T24, acquiring the current round feature representation and the current round topology representation corresponding to each positive sample, and the current round topology representation corresponding to each negative sample; and T25, calculating a current round iterative loss according to a preset loss function, and updating parameters of the initial model according to the current round iterative loss.

Description

Technical Field

[0001] This invention relates to the field of graph representation learning, specifically to graph embedding techniques in the field of graph representation learning, and more specifically, to a method for constructing a directed graph embedding representation model and an embedding representation learning method. Background Technology

[0002] A graph is a network-like data structure consisting of a non-empty set of nodes and a set of edges describing the relationships between the nodes. Nodes represent entities, and edges represent the connections between two nodes, i.e., the relationships between two entities. Graphs can be broadly classified into directed graphs and undirected graphs. In undirected graphs, edges have no direction; in directed graphs, edges have direction. Graph data can be used to describe the connections between objects in the real world, such as social networks, communication networks, and transportation networks. Furthermore, the analysis and processing of graph data can provide insights into social structures, languages, and different communication patterns. The results of graph data analysis can also be applied to various application scenarios such as recommender systems and intelligent question answering. Therefore, graph data has always been a hot topic in academic research.

[0003] Real-world graph data is often high-dimensional and difficult to process. In order to process graph data, researchers have proposed graph embedding methods. Graph embedding is a type of graph representation learning. Simply put, it maps graph data to a low-dimensional vector space to obtain an embedded representation (vector representation) of the graph data. When converting graph data into an embedded representation, the structural information and potential characteristics of the graph data should be preserved as much as possible.

[0004] With the continuous development of deep learning, researchers have proposed using deep learning models to process graph data and obtain its embedded representation. However, graph data differs from structured data such as images and text; it exhibits non-Euclidean characteristics, making it difficult to extract features using traditional convolutional neural networks, thus hindering the acquisition of its embedded representation. Therefore, existing techniques generally employ graph neural networks to extract features from graph data to obtain structural knowledge and subsequently, the embedded representation. Existing graph embedding methods based on graph neural networks mainly fall into two categories: spectral domain-based methods and spatial domain-based methods.

[0005] Among them, the graph embedding method based on the spectral domain is represented by the scheme proposed in reference [1]. It uses graph convolutional neural network (GCN) to model the neighborhood structure of nodes in graph data to obtain the embedding representation of graph data. This type of method is based on the theory of spectral analysis and requires a symmetric adjacency matrix as input. Therefore, this type of method will add reverse links when processing directed graphs to satisfy the symmetry of the adjacency matrix. However, the direction of the links (edges) in graph data has important reference value in some real systems. If the directed graph is simply converted into a bidirectional graph, it will cause the loss of graph data structure information. For example, in the citation network, the research published earlier cannot cite the research published later. For example, if literature A cites literature B in a citation network, this fact can be represented as (literature A, citation, literature B). If the graph data constructed by the citation network is processed by the graph embedding method based on the spectral domain, in order to satisfy the symmetry of the adjacency matrix, the reverse link (literature B, citation, literature A) will be added to the original constructed graph data. After this processing, it will cause counterfactual information error and will not be able to correctly process the directional information in the directed graph.

[0006] The spatial domain-based graph embedding method, represented by the scheme proposed in reference [2], uses the GraphSAGE graph neural network to pass messages through the direction of the links, gradually updating the features of each node in the graph data to obtain the embedded representation of the graph data. Although this type of method does not require the symmetry operation of the adjacency matrix, when processing directed graphs, this type of method only uses the direction of the links as the guide for message passing, ignoring the complex relationships between the directions of the directed links, resulting in the loss of topological information in the local graph structure. For example, when using this type of method to process graph data constructed by citation networks, when there are two facts in the citation network, namely, document A citing document B and document C citing document B, this type of method can only use the citation direction from document A to document B and the citation direction from document C to document B as the path for message passing, and cannot discover the correlation between document A and document C (because the message cannot be propagated through non-existent edges or edges with incorrect directions), and therefore cannot effectively extract the complex topological information contained in the link connection direction of the directed graph.

[0007] In summary, although existing graph embedding methods based on graph neural networks can process graph data and obtain embedded representations of graph data, existing graph embedding methods have at least one of the following limitations when processing directed graphs: (1) they cannot correctly process the directional information in directed graphs; (2) they ignore the topological structure between the links in the directed graph, and cannot effectively extract the complex topological information contained in the link connection method in the directed graph, resulting in the inability to obtain a good embedded representation of the directed graph, which is not conducive to the execution of downstream tasks related to graph data.

[0008] References:

[0009] [1]Kipf, Thomas and Max Welling. "Semi-Supervised Classification with Graph Convolutional Networks." ArXiv abs / 1609.02907(2016):n.pag.

[0010] [2]Hamilton, William L., Zhitao Ying and Jure Leskovec. "InductiveRepresentation Learning on Large Graphs." NIPS (2017). Summary of the Invention

[0011] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method for constructing a directed graph embedding representation model, a method for learning a directed graph embedding representation, and a protein classification system.

[0012] The objective of this invention is achieved through the following technical solutions.

[0013] According to a first aspect of the present invention, a method for constructing a directed graph embedding representation model is provided. The method includes: T1, constructing an initial model, the initial model including a feature acquisition module and a topology information aggregation module; T2, acquiring a directed graph and performing multiple rounds of iterative training on the initial model to obtain a directed graph embedding representation model, wherein the directed graph includes multiple entities and multiple directed edges connecting any two entities, and each round of iterative training includes: T21, acquiring the current round feature representation corresponding to each entity in the directed graph through the feature acquisition module; T22, extracting the topology information of each entity in the directed graph through the topology information aggregation module to obtain the current round topology representation corresponding to each entity; T23, randomly selecting multiple entities from all entities in the directed graph as multiple positive samples, and Construct a set of negative samples corresponding to each positive sample, wherein the set of negative samples includes multiple negative samples, each of which is an entity randomly selected from entities other than its corresponding positive sample entity; T24, based on the current round feature representation corresponding to each entity obtained in step T21 and the current round topology representation corresponding to each entity obtained in step T22, obtain the current round feature representation and current round topology representation corresponding to each positive sample constructed in step T23, and the current round topology representation corresponding to each negative sample; T25, based on the current round feature representation and current round topology representation corresponding to each positive sample obtained in step T24, and the current round topology representation corresponding to each negative sample, calculate the current round iteration loss according to a preset loss function, and update the parameters of the initial model according to the current round iteration loss.

[0014] In some embodiments of the present invention, the feature acquisition module is configured to perform feature encoding on each entity in the directed graph to obtain a representation of the feature perspective corresponding to each entity.

[0015] In some embodiments of the present invention, the feature acquisition module includes a multi-layered cascaded graph neural network.

[0016] In some embodiments of the present invention, the feature acquisition module includes a two-layer graph attention network.

[0017] In some embodiments of the present invention, in step T21, the feature acquisition module is configured to perform feature encoding on each entity in the following manner: T211, using a preset first function to aggregate the representation of the previous round feature perspective of all first-order adjacent entities corresponding to the entity, so as to obtain the current round hidden layer state corresponding to the entity; T212, using a preset second function to aggregate the current round hidden layer state of all first-order adjacent entities corresponding to the entity, so as to obtain the representation of the current round feature perspective corresponding to the entity.

[0018] In some embodiments of the present invention, the first function is:

[0019]

[0020] Among them, s i ′ represents entity e i The corresponding current hidden layer state, ReLU(*) represents the activation function, N i Represents entity e i The set of all first-order adjacent entities, α ij Represents entity e j For entity e i Importance parameter, s j Represents entity e j The corresponding representation of the previous round of features;

[0021] The second function is:

[0022]

[0023] Among them, z i Represents entity e i The corresponding representation of the current round's feature view, LeakReLU(*) represents the activation function, s j ′ represents entity e j The corresponding current hidden layer state;

[0024] In each iteration, the importance parameter is calculated as follows:

[0025]

[0026] Among them, w T This represents the transpose of the learnable parameters w, where W represents the weight matrix, and s i Represents entity e i The corresponding representation of the previous round of feature perspectives, s k Represents entity e k The corresponding representation of the previous round of features. This indicates a serial operation.

[0027] In some embodiments of the present invention, in step T22, the current round topology information corresponding to each entity is extracted in the following manner: T221, based on the directed graph, determine all first-order adjacent entities and all second-order adjacent entities corresponding to the entity, and construct a subgraph corresponding to the entity based on it, wherein the subgraph includes one or more topological paths, and each topological path is composed of an entity, the entity's first-order adjacent entities, the entity's second-order adjacent entities, the directed edges between the entity and the entity's first-order adjacent entities, and the directed edges between the entity's first-order adjacent entities and the entity's second-order adjacent entities; T222, perform multiple random samplings on the subgraph obtained in step T221 to obtain multiple topological paths, and classify each sampled topological path according to a preset classification rule. The process involves classifying entities and modeling their first-order and second-order adjacent entities as virtual nodes on each topological path; T223, obtaining the feature representation of each virtual node on each topological path obtained in step T222, wherein the current round feature view representation of the first-order adjacent entity and the current round feature view representation of the second-order adjacent entity on each topological path are concatenated to obtain the feature representation of each virtual node on each topological path; T224, processing the feature representation of each virtual node on each topological path obtained in step T223 using the topological information aggregation module to obtain the influence factor of each virtual node on each topological path, and performing average pooling on all influence factors according to a preset third function to obtain the current round topological representation of the entity.

[0028] In some embodiments of the present invention, the preset classification rules are as follows: if a topological path contains a directed edge from an entity to its first-order neighbor and a directed edge from its second-order neighbor to its first-order neighbor, the topological path is classified as a head-to-head topological pattern; if a topological path contains a directed edge from an entity to its first-order neighbor and a directed edge from its first-order neighbor to its second-order neighbor, the topological path is classified as a head-to-tail topological pattern; if a topological path contains a directed edge from an entity to its first-order neighbor and a directed edge from its second-order neighbor to its first-order neighbor, the topological path is classified as a tail-to-head topological pattern; and if a topological path contains a directed edge from an entity to its first-order neighbor and a directed edge from its first-order neighbor to its second-order neighbor, the topological path is classified as a tail-to-tail topological pattern.

[0029] In some embodiments of the present invention, the topology information aggregation module includes multiple sub-aggregation networks, and each sub-aggregation network is configured to process virtual nodes on a topology path of a certain category to obtain the influence factor of the virtual nodes on that topology path, wherein each sub-aggregation network is a multilayer perceptron.

[0030] In some embodiments of the present invention, each multilayer sensor includes three fully connected layers.

[0031] In some embodiments of the present invention, the third function is:

[0032]

[0033] Where, z′ i Represents entity e i The corresponding current round topology representation, where D represents the hyperparameter, PointPlus represents vector addition bitwise, and c j Let represent the influence factor of the virtual node on the j-th topological path, and C represent the set of influence factors of all virtual nodes on all topological paths.

[0034] In some embodiments of the present invention, the preset loss function is:

[0035]

[0036] Where d(*) represents the similarity metric function, z i Represents entity e i The corresponding representation of the current round's feature perspective, z′ i Represents entity e i The corresponding current round topology representation, ε N Represents entity e i The corresponding set of negative samples, z j ′ represents entity e j The corresponding current round topology representation.

[0037] According to a second aspect of the present invention, a directed graph embedding representation learning method is provided, the method comprising: S1, obtaining a directed graph, the directed graph including multiple entities and multiple directed edges connecting any two entities; S2, processing the directed graph obtained in step S1 using a directed graph embedding representation model obtained by the method described in the first aspect of the present invention to obtain a feature view representation and a topological representation corresponding to each entity, and concatenating the obtained feature view representation and topological representation corresponding to each entity to obtain an embedding representation corresponding to each entity.

[0038] According to a third aspect of the present invention, a protein classification system is provided, the system comprising: a data processing module for acquiring the molecular structure of a protein synthesis to be detected and constructing a directed graph based on the molecular structure of the protein synthesis to be detected, wherein the protein synthesis to be detected includes multiple protein molecules, the directed graph includes multiple entities and multiple directed edges connecting any two entities, and the entities are any protein molecules in the protein synthesis to be detected; a directed graph embedding representation model obtained by the method of the first aspect of the present invention, for processing the directed graph obtained by the data processing module to obtain an embedding representation of each protein molecule; and a pre-trained protein classification model for classifying protein molecules based on the embedding representation of each protein molecule obtained by the directed graph embedding representation model to obtain the category of each protein molecule in the protein synthesis to be detected.

[0039] Compared with the prior art, the advantages of the present invention are: (1) a topology information aggregation module is introduced to extract the topology information of entities to capture the potential structural information between different link connections in the directed graph; (2) training samples are provided for iterative training based on the feature representation and topology representation of entities in the directed graph, and the iterative loss is calculated based on them to optimize the feature acquisition module and the topology information aggregation model, thereby improving the effect of the embedded representation of the directed graph. Attached Figure Description

[0040] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0041] Figure 1 This is a schematic diagram of the construction method of a directed graph embedding representation model according to an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating an example of a topology path type according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the directed graph embedding representation learning method according to an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of a protein classification system according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] As described in the background section, although existing graph embedding methods based on graph neural networks can process graph data and obtain embedded representations of graph data, existing graph embedding methods have at least one of the following limitations when processing directed graphs: (1) they cannot correctly process the directional information in directed graphs; (2) they ignore the topological structure between the links in the directed graph, and cannot effectively extract the complex topological information contained in the link connection mode in the directed graph, resulting in the inability to obtain a good embedded representation of the directed graph, which is not conducive to performing downstream tasks related to graph data.

[0047] To address the aforementioned problems, this invention proposes a novel graph embedding scheme for directed graphs. In this scheme, a subgraph is constructed for each entity in the directed graph to capture potential structural information between different link relationships, resulting in a topological representation of each entity. Furthermore, information about each entity from a feature perspective is learned from the directed graph to capture directional information, resulting in a feature perspective representation of each entity. Finally, based on the obtained topological and feature perspective representations of each entity, an embedded representation of the directed graph is obtained, ensuring that the embedded representation of the directed graph effectively represents the directional information and topological information in the link connection methods contained within the directed graph.

[0048] To better understand the present invention, the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0049] According to one embodiment of the present invention, such as Figure 1The present invention provides a method for constructing a directed graph embedding representation model, the method comprising: T1, constructing an initial model, the initial model including a feature acquisition module and a topology information aggregation module; T2, obtaining a directed graph and performing multiple rounds of iterative training on the initial model to obtain a directed graph embedding representation model, wherein the directed graph includes multiple entities and multiple directed edges connecting any two entities, and each round of iterative training includes: T21, obtaining the current round feature representation corresponding to each entity in the directed graph through the feature acquisition module; T22, extracting the topology information of each entity in the directed graph through the topology information aggregation module to obtain the current round topology representation corresponding to each entity; T23, randomly selecting multiple entities from all entities in the directed graph as multiple positive samples, and constructing a topology representation model corresponding to each entity. A set of negative samples corresponding to a positive sample, wherein the set of negative samples includes multiple negative samples, each of which is an entity randomly selected from entities other than its corresponding positive sample entity; T24, based on the current round feature representation corresponding to each entity obtained in step T21 and the current round topology representation corresponding to each entity obtained in step T22, obtain the current round feature representation and current round topology representation corresponding to each positive sample constructed in step T23, and the current round topology representation corresponding to each negative sample; T25, based on the current round feature representation and current round topology representation corresponding to each positive sample obtained in step T24, and the current round topology representation corresponding to each negative sample, calculate the current round iteration loss according to a preset loss function, and update the parameters of the initial model according to the current round iteration loss.

[0050] To further understand this invention, the following detailed explanation of the proposed solution will focus on the initial model construction in step T1 and the initial model iterative training in step T2.

[0051] I. Initial Model Construction

[0052] To enable the model to extract directional information and topological information between link directions in a directed graph, a feature acquisition module and a topology information aggregation module are included in the initial model construction. The feature acquisition module and the topology information aggregation module are described in detail below.

[0053] 1.1 Feature Acquisition Module

[0054] The feature acquisition module is used to learn information about all entities in the directed graph from a feature perspective, in order to extract the directional information contained in the directed graph. According to one embodiment of the present invention, the feature acquisition module is configured to perform feature encoding on each entity in the directed graph to obtain a representation of the feature perspective corresponding to each entity. According to one embodiment of the present invention, the feature acquisition module includes a multi-layer cascaded graph neural network. According to one embodiment of the present invention, the feature acquisition module includes a two-layer graph attention network. It should be noted that the feature acquisition module can use graph neural network models such as Graph Attention Networks (GAT) and DiffWise networks to extract the representation of entities in the directed graph from a feature perspective; the present invention does not impose specific limitations on the feature acquisition module.

[0055] 1.2 Topology Information Aggregation Module

[0056] The topology information aggregation module is used to aggregate the topology information surrounding all entities in a directed graph to extract the topology information contained in the link connection methods of the directed graph. According to one embodiment of the present invention, the topology information aggregation module includes multiple sub-aggregation networks, each of which is a multilayer perceptron. According to one embodiment of the present invention, each multilayer perceptron includes three fully connected layers. It should be noted that the sub-aggregation networks can use neural network models such as multilayer perceptrons and recurrent neural networks to aggregate the topology information surrounding entities in the directed graph; the present invention does not impose specific limitations on the topology information aggregation module.

[0057] II. Initial Model Iterative Training

[0058] In step T2, the initial model is trained iteratively through multiple rounds using a directed graph to obtain a directed graph embedding representation model. The directed graph includes multiple entities and multiple directed edges connecting any two entities. Each round of iterative training includes steps T21-T25. Step T21 is used to obtain the current-round feature representation for each entity; step T22 is used to obtain the current-round topological feature representation for each entity; step T23 is used to construct multiple positive samples and a set of negative samples corresponding to the positive samples; step T24 is used to obtain the feature representations and topological representations corresponding to the positive and negative samples; and step T25 is used to calculate the current-round iteration loss and update the parameters of the initial model based on it. The following provides a detailed explanation of each step.

[0059] Step T21: Obtain the current round feature representation for each entity.

[0060] In step T21, the feature acquisition module obtains the current round feature representation corresponding to each entity in the directed graph. According to an embodiment of the present invention, the feature acquisition module is configured to perform feature encoding on each entity using steps T211-T212 to obtain the current round feature representation corresponding to each entity in the directed graph.

[0061] In step T211, a preset first function is used to aggregate the previous round feature view representations of all first-order adjacent entities corresponding to the entity, in order to obtain the current round hidden layer state corresponding to the entity, and the first function is:

[0062]

[0063] Among them, s i ′ represents entity e i The corresponding current hidden layer state, ReLU(*) represents the activation function, N i Represents entity e i The set of all first-order adjacent entities, α ij Represents entity e j For entity e i Importance parameter, s j Represents entity e j The corresponding representation of the previous round of features.

[0064] In step T212, a preset second function is used to aggregate the current wheel hidden state of all first-order adjacent entities corresponding to the entity, so as to obtain the representation of the current wheel feature viewpoint corresponding to the entity, and the second function is:

[0065]

[0066] Among them, z i Represents entity e i The corresponding representation of the current round's feature view, LeakReLU(*) represents the activation function, s j ′ represents entity e j The corresponding current hidden layer state.

[0067] It should be noted that the importance parameter is calculated in each iteration as follows:

[0068]

[0069] Among them, w T This represents the transpose of the learnable parameters w, where W represents the weight matrix, and s i Represents entity e i The corresponding representation of the previous round of feature perspectives, s k Represents entity e kThe corresponding representation of the previous round of features. This indicates a concatenation operation. It should be noted that during the first round of iterative training, the initial features of entities in the directed graph are used to calculate the first-round hidden state for each entity and the importance parameters between entities. It should also be noted that when acquiring directed graphs, the required directed graphs are generally selected from publicly available directed graph datasets, and these datasets provide the initial features of the entities in the directed graphs. If the acquired directed graph does not provide initial features of the entities, the initial features of the entities can be obtained by coarsely processing the collected data using methods such as word bags. Since how to obtain the initial features of entities in a directed graph is a technique well-known to those skilled in the art, this invention will not describe it.

[0070] Step T22: Obtain the current round topology feature representation for each entity.

[0071] In step T22, the topology information of each entity in the directed graph is extracted by the topology information set module to obtain the current round topology representation corresponding to each entity. According to an embodiment of the present invention, in step T22, steps T221-224 are used to extract the current round topology information corresponding to each entity.

[0072] In step T221, based on the directed graph, all first-order adjacent entities and all second-order adjacent entities corresponding to the entity are determined, and a subgraph corresponding to the entity is constructed based on them. The subgraph includes one or more topological paths, and each topological path is composed of an entity, the entity's first-order adjacent entities, the entity's second-order adjacent entities, the directed edges between the entity and the entity's first-order adjacent entities, and the directed edges between the entity's first-order adjacent entities and the entity's second-order adjacent entities.

[0073] In step T222, multiple random samplings are performed on the subgraph obtained in step T221 to obtain multiple topological paths. Each topological path is classified according to a preset classification rule, and the first-order adjacent entities and the second-order adjacent entities of the entities on each topological path are modeled as a virtual node. According to an embodiment of the present invention, the preset classification rule is as follows: if a topological path contains directed edges from an entity to its first-order neighbor and directed edges from its second-order neighbor to its first-order neighbor, the topological path is classified as a head-to-head topological pattern; if a topological path contains directed edges from an entity to its first-order neighbor and directed edges from its first-order neighbor to its second-order neighbor, the topological path is classified as a head-to-tail topological pattern; if a topological path contains directed edges from an entity to its first-order neighbor and directed edges from its second-order neighbor to its first-order neighbor, the topological path is classified as a tail-to-head topological pattern; if a topological path contains directed edges from an entity to its first-order neighbor and directed edges from its second-order neighbor to its second-order neighbor, the topological path is classified as a tail-to-tail topological pattern. To better understand the topological path types, such as... Figure 2 As shown, it illustrates four types of topology paths. Among them, the head-to-head topology path type can be represented as (e i ,r1,e j ) and (e j r2,e k The head-to-tail topology path type can be represented as (e i r ，1 e j ) and (e j r3,e k Tail-to-head topology path type can be represented as (e i r4,e j ) and (e j r2,e k Tail-to-tail topology path type can be represented as (e i r4,e j ) and (e j r3,e k ), where r1 represents a directed edge from e i Pointing to e j r2 indicates that there is a directed edge from e k Pointing to e j r3 indicates that there is a directed edge from e j Pointing to e k r4 indicates that there is a directed edge from e j Pointing to e iIt is important to note that directed graph data contains two special data forms: bidirectional edges and self-loops. A bidirectional edge implies a symmetric relationship between two entities, such as two users following each other or frequent bidirectional interactions in a social network. A self-loop indicates that an entity has a link pointing to itself, such as some proteins in a protein network exhibiting self-interaction. Considering that not all real-world systems exhibit bidirectional edges and self-loops, for bidirectional edges, the entity's two-hop neighbor is allowed to overlap with the entity itself, i.e., it jumps back to the entity; for self-loops, any of the three nodes can overlap, i.e., it repeatedly cycles within the self-loop. However, for both bidirectional edges and self-loops, only one link direction is considered in each topology path sampling process. It should be noted that the number of random samplings is determined according to actual needs, and this invention does not impose any special restrictions.

[0074] In step T223, the feature representation of the virtual node on each topological path sampled in step T222 is obtained. Specifically, the representations of the current round feature views of the first-order neighboring entities and the second-order neighboring entities of each entity on each topological path are concatenated to obtain the feature representation of the virtual node on each topological path. Taking node e as an example... i Node e i First-order adjacent node e j and node e i The second-order adjacent node e k Taking the constructed topological path as an example, the characteristics of the virtual nodes on this topological path are represented as follows: Among them, z j Represents the first-order adjacent node e j The corresponding representation of the current round's feature perspective, z k Represents the second-order adjacent node e k The corresponding representation of the current round's feature perspective, This indicates a serial operation.

[0075] In step T224, the topology information aggregation module processes the feature representations of virtual nodes on each topology path obtained in step T223 to obtain the influence factors of virtual nodes on each topology path, and performs average pooling on all influence factors according to a preset third function to obtain the current round topology representation corresponding to the entity, wherein the third function is:

[0076]

[0077] Where, z′ i Represents entity e i The corresponding current round topology representation, where D represents the hyperparameter, PointPlus represents vector addition bitwise, and c jLet represent the influence factor of the virtual node on the j-th topological path, and C represent the set of influence factors of virtual nodes on all topological paths. It should be noted that the topology information aggregation module includes multiple sub-aggregation networks, and each sub-aggregation network is configured to process virtual nodes on a category of topological paths to obtain the influence factors of virtual nodes on that category of topological paths. For example, if a sampled topological path belongs to a head-to-head topological pattern, the feature representation of the virtual node on that topological path is used as the network input of the sub-aggregation network corresponding to the head-to-head topological pattern, and the output of the sub-aggregation network is used as the influence factor of the virtual node on that topological path. The processing method for virtual nodes on topological paths of other patterns is similar, and will not be elaborated further here. It should be noted that each type of topological path physically represents a different data structure, requiring different sub-aggregation networks to process different types of topological paths. In this case, to enable the sub-aggregation networks to process different types of topological paths separately, the backpropagation gradients can be distributed to different sub-aggregation networks during training according to a preset weight ratio. This allows the four sub-aggregation networks to receive different weights, enabling them to process their corresponding types of topological paths. The preset weight ratio can be determined based on the proportion of each type of topological path to all sampled topological paths. Specifically, the weight ratio of the sub-aggregation network corresponding to the head-to-head topological pattern is the ratio of all head-to-head topological paths to all sampled topological paths; the weight ratio of the sub-aggregation network corresponding to the head-to-tail topological pattern is the ratio of all head-to-tail topological paths to all sampled topological paths; the weight ratio of the sub-aggregation network corresponding to the tail-to-head topological pattern is the ratio of all tail-to-head topological paths to all sampled tail-to-head paths; and the weight ratio of the sub-aggregation network corresponding to the tail-to-tail topological pattern is the ratio of all tail-to-tail topological paths to all sampled tail-to-head paths.

[0078] Step T23: Construct a set of multiple positive samples and a set of negative samples corresponding to the positive samples.

[0079] In step T23, multiple entities are randomly selected from all entities in the directed graph as multiple positive samples, and a set of negative samples corresponding to each positive sample is constructed. The set of negative samples includes multiple negative samples, each of which is an entity randomly selected from entities other than its corresponding positive sample entity. It should be noted that the reason for constructing a set of positive samples and their corresponding negative samples is that this basic structure of contrastive learning can capture the fundamental structure and relationship between positive and negative samples. In each iteration of training, it can narrow the gap with positive samples and widen the gap with negative samples, prompting the model to optimize in a positive direction.

[0080] Step T24: Obtain the feature representations and topological representations corresponding to positive and negative samples.

[0081] In step T24, based on the current round feature representation of each entity obtained in step T21 and the current round topology representation of each entity obtained in step T22, the current round feature representation and current round topology representation of each positive sample constructed in step T23, as well as the current round topology representation of each negative sample, are obtained.

[0082] Step T25: Calculate the current iteration loss and update the parameters of the initial model based on it.

[0083] In step T25, based on the current round feature representation and current round topological representation corresponding to each positive sample obtained in step T24, and the current round topological representation corresponding to each negative sample, the current round iteration loss is calculated according to a preset loss function, and the parameters of the initial model are updated according to the current round iteration loss. According to an embodiment of the present invention, the preset loss function is:

[0084]

[0085] Where d(*) represents the similarity metric function, z i Represents entity e i The corresponding representation of the current round's feature perspective, z′ i Represents entity e i The corresponding current round topology representation, ε N Represents entity e i The corresponding set of negative samples, z j ′ represents entity e j The corresponding current round topology representation. The similarity metric function is:

[0086] d(u, v) = exp[(u T v) / τ+(v T u) / τ]

[0087] Where u represents any vector, u T Let v be the transpose of vector u, and let v denote any vector. T Let v denote the transpose of vector v, and τ denote the temperature parameter. It should be noted that in each iteration of training, the loss corresponding to each positive sample is calculated based on a preset loss function and a similarity metric function. The average of the losses corresponding to all positive samples is then summed to obtain the current iteration loss. This current iteration loss is used to update the parameters of the initial model, completing one iteration update.

[0088] The directed graph embedding representation model obtained based on the foregoing embodiments can be used to obtain directed graph embedding representations and to perform downstream tasks related to graph data based on the obtained directed graph embedding representations.

[0089] According to one embodiment of the present invention, such as Figure 3 As shown, a directed graph embedding representation learning method is provided. The method includes: S1, obtaining a directed graph, which includes multiple entities and multiple directed edges connecting any two entities; S2, processing the directed graph obtained in step S1 using the directed graph embedding representation model obtained by the method described in the previous embodiment to obtain the feature view representation and topological representation corresponding to each entity, and concatenating the obtained feature view representation and topological representation corresponding to each entity to obtain the embedding representation corresponding to each entity.

[0090] There are many downstream tasks related to graph data. A common downstream task is node classification, which can also be called entity classification. For example, modeling protein-protein interactions as graph data and generating the embedding representation of the protein molecule corresponding to each entity, and then classifying the protein molecule based on the protein molecule embedding.

[0091] According to one embodiment of the present invention, such as Figure 4 As shown, a protein classification system is provided. The system includes: a data processing module for acquiring the molecular structure of a protein synthesis to be detected and constructing a directed graph based on the molecular structure of the protein synthesis to be detected. The protein synthesis to be detected includes multiple protein molecules, and the directed graph includes multiple entities and multiple directed edges connecting any two entities, where each entity is any protein molecule in the protein synthesis to be detected. A directed graph embedding representation model obtained by the method described in the foregoing embodiments is used to process the directed graph obtained by the data processing module to obtain the embedding representation of each protein molecule. A pre-trained protein classification model is used to classify protein molecules based on the embedding representation of each protein molecule obtained by the directed graph embedding representation model to obtain the category of each protein molecule in the protein synthesis to be detected. It should be noted that the processing methods of other downstream tasks related to graph data are consistent with the processing principle of the protein classification system described in this embodiment, and will not be repeated here.

[0092] The beneficial effects of the present invention are: (1) introducing a topology information aggregation module to extract the topology information of entities in order to capture the potential structural information between different link connections in a directed graph; (2) providing training samples for iterative training based on the feature representation and topology representation of entities in a directed graph, and optimizing the feature acquisition module and the topology information aggregation model based on iterative loss calculation, thereby improving the effect of the embedded representation of a directed graph.

[0093] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0094] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0095] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0096] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a directed graph embedding representation model, characterized in that, The method includes: T1. Construct an initial model, which includes a feature acquisition module and a topology information aggregation module; T2. Obtain a directed graph of the molecular structure corresponding to the protein synthesizer, and perform multiple rounds of iterative training on the initial model based on the obtained directed graph to obtain a directed graph embedding representation model. The protein synthesizer includes various protein molecules, and the directed graph includes multiple entities and multiple directed edges connecting any two entities. Each round of iterative training includes: T21. Obtain the current round feature representation corresponding to each entity in the directed graph through the feature acquisition module; T22. Extract the topology information of each entity in the directed graph through the topology information set module to obtain the current round topology representation of each entity; T23. Randomly select multiple entities from all entities in the directed graph as multiple positive samples, and construct a set of negative samples corresponding to each positive sample, wherein the set of negative samples includes multiple negative samples, and each negative sample is an entity randomly selected from entities other than its corresponding positive sample entity; T24. Based on the current round feature representation of each entity obtained in step T21 and the current round topology representation of each entity obtained in step T22, obtain the current round feature representation and current round topology representation of each positive sample constructed in step T23, and the current round topology representation of each negative sample. T25. Based on the current round feature representation and current round topology representation corresponding to each positive sample obtained in step T24, and the current round topology representation corresponding to each negative sample, calculate the current round iteration loss according to the preset loss function, and update the parameters of the initial model according to the current round iteration loss.

2. The method according to claim 1, characterized in that, The feature acquisition module is configured to perform feature encoding on each entity in the directed graph to obtain a representation of the feature perspective corresponding to each entity.

3. The method according to claim 2, characterized in that, The feature acquisition module includes a multi-layered cascaded graph neural network.

4. The method according to claim 3, characterized in that, The feature acquisition module includes a two-layer graph attention network.

5. The method according to claim 4, characterized in that, In step T21, the feature acquisition module is configured to encode the features of each entity in the following manner: T211. Use a preset first function to aggregate the previous round feature view representation of all first-order adjacent entities corresponding to the entity, so as to obtain the current round hidden layer state corresponding to the entity. T212. Use a preset second function to aggregate the current wheel hidden state of all first-order adjacent entities corresponding to the entity, so as to obtain the representation of the current wheel feature view of the entity.

6. The method according to claim 5, characterized in that, The first function is: in, Representing entities The corresponding current hidden layer state, This represents the activation function. Representing entities The set of all corresponding first-order adjacent entities, Representing entities For entities Importance parameters Representing entities The corresponding representation of the previous round of features; The second function is: in, Representing entities The corresponding representation of the current round's feature perspective, This represents the activation function. Representing entities The corresponding current hidden layer state; In each iteration, the importance parameter is calculated as follows: in, Represents learnable parameters transpose, Represents the weight matrix. Representing entities The corresponding representation of the previous round of features. Representing entities The corresponding representation of the previous round of features. This indicates a serial operation.

7. The method according to claim 6, characterized in that, In step T22, the current round topology information corresponding to each entity is extracted in the following manner: T221. Based on the directed graph, determine all first-order adjacent entities and all second-order adjacent entities corresponding to the entity, and construct a subgraph corresponding to the entity based on it. The subgraph includes one or more topological paths, and each topological path is composed of an entity, the entity's first-order adjacent entities, the entity's second-order adjacent entities, the directed edge between the entity and the entity's first-order adjacent entities, and the directed edge between the entity's first-order adjacent entities and the entity's second-order adjacent entities. T222. Multiple random samplings are performed on the subgraph obtained in step T221 to obtain multiple topological paths. Each topological path is classified according to a preset classification rule. The first-order adjacent entities and the second-order adjacent entities of the entities on each topological path are modeled as a virtual node. T223. Obtain the feature representation of the virtual node on each topological path sampled in step T222. The feature representation of the first-order neighbor of the entity on each topological path and the feature representation of the second-order neighbor of the entity on each topological path are concatenated to obtain the feature representation of the virtual node on each topological path. T224. The topology information aggregation module is used to process the feature representation of the virtual node on each topology path obtained in step T223 to obtain the influence factor of the virtual node on each topology path, and the average pooling of all influence factors is performed according to the preset third function to obtain the current round topology representation of the entity.

8. The method according to claim 7, characterized in that, The preset classification rules are as follows: If a topological path contains directed edges from an entity to its first-order neighbor, and directed edges from its second-order neighbor to its first-order neighbor, then the topological path is classified as a head-to-head topological pattern. If a topological path contains directed edges from an entity to its first-order neighbor and from its first-order neighbor to its second-order neighbor, then the topological path is classified as a head-to-tail topological pattern. If there are directed edges in the topological path from the first-order adjacent entity of the entity to the entity, and directed edges from the second-order adjacent entity of the entity to the first-order adjacent entity of the entity, then the topological path is classified as a tail-to-head topological pattern. If a topological path contains directed edges from a first-order neighboring entity to the entity, and directed edges from a first-order neighboring entity to a second-order neighboring entity, then the topological path is classified as a tail-to-tail topological pattern.

9. The method according to claim 8, characterized in that, The topology information aggregation module includes multiple sub-aggregation networks, and each sub-aggregation network is configured to process virtual nodes on a topology path of a certain category to obtain the influence factor of virtual nodes on that topology path. Each sub-aggregation network is a multilayer perceptron.

10. The method according to claim 9, characterized in that, Each of the multilayer perceptrons includes three fully connected layers.

11. The method according to claim 9, characterized in that, The third function is: in, Representing entities The corresponding current round topology representation, Indicates hyperparameters, This indicates that the vectors are added bitwise. Indicates the first Influence factors of virtual nodes on a topological path This represents the set of influence factors of virtual nodes on all topological paths.

12. The method according to claim 11, characterized in that, The preset loss function is: in, This represents the similarity measurement function. Representing entities The corresponding representation of the current round's feature perspective, Representing entities The corresponding current round topology representation, Representing entities The corresponding set of negative samples, Representing entities The corresponding current round topology representation.

13. A directed graph embedding representation learning method, characterized in that, The method includes: S1. Obtain a directed graph, which includes multiple entities and multiple directed edges connecting any two entities; S2. The directed graph obtained in step S1 is processed using the directed graph embedding representation model obtained by any one of the methods described in claims 1-12 to obtain the feature view representation and topological representation corresponding to each entity, and the obtained feature view representation and topological representation corresponding to each entity are concatenated to obtain the embedding representation corresponding to each entity.

14. A protein classification system, characterized in that, The system includes: The data processing module is used to obtain the molecular structure of the protein synthesis to be detected and to construct a directed graph based on the molecular structure of the protein synthesis to be detected. The protein synthesis to be detected includes a variety of protein molecules, and the directed graph includes multiple entities and multiple directed edges connecting any two entities. The entities are any protein molecules in the protein synthesis to be detected. The directed graph embedding representation model obtained by any one of claims 1-12 is used to process the directed graph obtained by the data processing module to obtain the embedding representation of each protein molecule. The pre-trained protein classification model is used to classify protein molecules based on the embedding representation of each protein molecule obtained from the directed graph embedding representation model, so as to obtain the category of each protein molecule in the protein synthesis to be detected.

15. A computer-readable storage medium, characterized in that, It contains a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 13.

16. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the electronic device to perform the steps of the method as described in any one of claims 1 to 13.

Images