A role-oriented heterogeneous network representation learning method

Abstract

This paper provides a role-oriented heterogeneous network representation learning method, which belongs to the field of heterogeneous network representation learning and alleviates the problem of the lack of research on roles in heterogeneous network structures in the existing technology. The method includes: crawling raw data, performing data cleaning and preprocessing, and constructing a heterogeneous network; using a heterogeneous anonymous walk module to capture node neighborhood structure information; using a similar structure merging module to merge similar anonymous walk paths; and using a topic model to obtain role-oriented node representations.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and complex network technology, and in particular to a role-oriented heterogeneous network representation learning method. Background Art

[0002] As complex networks garner increasing attention, the problem of role discovery in these networks has also gradually attracted the attention of researchers. In complex networks, communities and roles are two distinct concepts. Nodes belonging to the same community are closer and more densely connected, while nodes belonging to the same role share similar neighborhood structures regardless of their distance. Therefore, two nodes belonging to the same role may belong to different communities. Another fundamental difference is that communities are defined differently across networks, while roles capture more general concepts, namely, structural patterns. Therefore, roles are universal across networks; they can be learned in one network and applied to another, whereas communities are not. Existing role discovery methods have performed well on various downstream tasks. They capture role features at different scales to learn more accurate role representations for nodes, but they all ignore the heterogeneity of nodes. For example, different types of neighboring nodes have different influences on the central node, and the neighborhood structures they form also vary greatly. Furthermore, the different distributions of structural patterns can profoundly influence the role of a node. Heterogeneous networks have attracted increasing research attention in recent years. Due to the greater variety of node and edge types, the structures in heterogeneous networks are more diverse. For shallow models, decomposition-based and relation-based methods focus on proximity. While meta-path-based random walk methods consider structural information in heterogeneous networks, they focus on only a few manually selected meta-paths and still only learn node proximity. Deeper models almost always learn node representations by maximizing similarity with neighboring nodes, making it difficult to capture similarities beyond the neighborhood, let alone structural similarity. Therefore, further research in the field of role discovery requires consideration of node heterogeneity. Summary of the Invention

[0003] The purpose of the present invention is to provide a role-oriented heterogeneous network representation learning method, which realizes the combination of project paper data to mine the role of authors in the network.

[0004] The present invention provides a role-oriented heterogeneous network representation learning method, comprising:

[0005] Obtain relevant data for a specific task, perform data cleaning and preprocessing, and construct a heterogeneous network. The dataset contains information on papers, scholars, and institutions, and each field in the dataset represents a different meaning.

[0006] Utilize heterogeneous anonymous walking modules to capture node neighborhood structure information;

[0007] Use the same structure merging module to merge similar anonymous walk paths, including:

[0008] The isomorphism test method is used to treat HAW samples with different structural patterns but isomorphic as the same type, so as to alleviate the problem of the number of HAW sequence types increasing with the increase of walk length l;

[0009] The specific steps of heterogeneous homogeneous testing are as follows:

[0010] (1) Definition of label composite set: Given two labeled heterogeneous subgraphs H and H', for each node, its label is set to the combination of the HAW position id and the node type. If it is the first iteration, the label composite set of node v only contains its own label. If it is not the first iteration, the elements of the label composite set of node v are the labels generated by node v's neighbor nodes in the previous iteration.

[0011] (2) Sorting the composite set: Aggregate the labels of the neighboring nodes of node v to obtain a new label, sort the elements in the composite set in ascending order according to the position id, and concatenate the sorted elements into a string s, with the label generated by node v in the previous iteration as the prefix;

[0012] (3) Label compression: Each label string s is mapped to a new label through the mapping function f if and only if When , the compression tags are the same;

[0013] (4) Relabeling: Use the compressed label as the i-th round label of node v in the subgraph;

[0014] (5) Generate subgraph representation: After the iteration, the new label is mapped to a one-hot vector, and the representation of each subgraph is obtained through pooling operation, which is used to calculate the similarity between subgraphs;

[0015] The topic model is used to obtain role-oriented node representation.

[0016] Furthermore, relevant data for a specific task is obtained and data cleaning and preprocessing are performed, including:

[0017] Relevant fields in the data are filtered to construct a heterogeneous information network. At the same time, garbled characters and empty field data are deleted to reduce noise. The heterogeneous information network includes three types of nodes: authors, papers, and projects. The relationships between authors’ published papers and papers’ affiliated projects are used as edges between nodes to construct an author relationship network.

[0018] Furthermore, a heterogeneous anonymous walk module is used to capture the node neighborhood structure information, including:

[0019] Different node sequences are obtained by random walk sampling, and the structural characteristics of the node neighborhood are obtained by anonymizing the walk sequences.

[0020] Furthermore, the topic model is used to obtain role-oriented node representations, including:

[0021] Using the topic model, a probabilistic description of the local structural pattern of each node is obtained, and the final structural topic will contain the structural pattern distribution indicating the node attributes.

[0022] The role-oriented heterogeneous network representation learning method provided by the present invention cleans and preprocesses the data, constructs a heterogeneous network, uses a heterogeneous anonymous walk module to capture node neighborhood structure information, then uses a similar structure merging module to merge similar anonymous walk paths, and finally uses a topic model to obtain role-oriented node representation, realizing the combination of project paper data to mine the author's role in the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A flowchart of a role-oriented heterogeneous network representation learning method provided by an embodiment of the present invention;

[0025] Figure 2 This is an application flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0027] The terms "including," "having," and any variations thereof, as used in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0028] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a role-oriented heterogeneous network representation learning method, including:

[0029] S1: Obtain relevant data for a specific task, perform data cleaning and preprocessing, and build a heterogeneous network.

[0030] Specifically, relevant fields in the data are filtered to build a heterogeneous information network, while garbled characters are removed and data with empty fields are deleted to reduce noise.

[0031] S2: Use heterogeneous anonymous walker modules to capture node neighborhood structure information.

[0032] Specifically, random walk sampling is used to obtain different node sequences, and the structural characteristics of the node neighborhood are obtained by anonymizing the walk sequences.

[0033] S3: Use the similar structure merging module to merge similar anonymous walk paths.

[0034] Specifically, the isomorphism test method is used to treat HAW samples with different structural patterns but isomorphic as the same category, so as to alleviate the problem of the number of HAW sequence types increasing with the increase of walk length l.

[0035] S4: Use topic model to obtain role-oriented node representation.

[0036] Specifically, using the topic model, a probabilistic description of the local structural pattern of each node is obtained, and the final structural topic will contain the structural pattern distribution indicating the node attributes.

[0037] First, the embodiment of the present invention proposes a heterogeneous anonymous walk (HAW) strategy to capture heterogeneous structural patterns. Given a heterogeneous graph H = {V, E, φ}, there is a random walk sequence w = (v0, v1, ... v l ), the corresponding HAW is a two-tuple sequence c = (g(v0), g(v1), ..., g(v l )), where g(v i )=(f(v i ),φ(v i)). Anonymous walks in heterogeneous networks can reconstruct the complete neighborhood centered on v based on the anonymous walk distribution starting from node v. To reconstruct the complete neighborhood of a node, it is also necessary to know the exact distribution of the heterogeneous structure. In order to solve the problem that the number of HAW sequence types increases exponentially with the increase of the walk length l, the present invention proposes a homogeneous structure merging mechanism. The isomorphic testing method is used to regard HAW samples with different structural patterns but isomorphic as the same category, so as to alleviate the problem of the number of HAW sequence types increasing with the increase of the walk length l. The WL algorithm can obtain a feature set for each node, which represents the role positioning of each node. Therefore, for some irregular graph structures, the WL algorithm can be used to determine whether the graph is isomorphic. The key idea is to expand the original label of the node by sorting and splicing the labels of neighboring nodes, and map these expanded labels to new labels. Specifically, each HAW is first regarded as a subgraph. For any node v i ∈H, s is the combination of the node's position id and node type in HAW, that is:

[0038]

[0039] Then get the node v i The feature labels of all neighbor nodes Update node v i Features:

[0040]

[0041] in For node v i After the previous iteration, the label Hash is an injective function, and the above steps are repeated. Since the ultimate goal is to alleviate the problem of the rapidly increasing number of HAW types, it is necessary to cluster all isomorphic HAW subgraphs to achieve the goal of reducing the number of HAW types. First, the label is mapped to a one-hot vector, that is, s→t, and then a pooling operation is performed to obtain the representation of each HAW subgraph:

[0042]

[0043] Here, R can be an operation such as mean, sum, or max, and the obtained value is used as the representation of the HAW subgraph. Finally, a type label is assigned to each HAW subgraph through a clustering algorithm. The specific steps of heterogeneous isomorphism testing are as follows:

[0044] (1) Label composite set definition: Given two labeled heterogeneous subgraphs H and H', for each node, its label is set to the combination of the HAW location id and the node type. If this is the first iteration, the label composite set of node v only contains its own label. If this is not the first iteration, the elements of the label composite set of node v are the labels generated by node v's neighbors in the previous iteration.

[0045] (2) Sorting the composite set: Aggregate the labels of the neighboring nodes of node v to obtain a new label, sort the elements in the composite set in ascending order according to the position id, and concatenate the sorted elements into a string s, with the label generated by node v in the previous iteration as the prefix (for example, s = <1A,0B2B3C4C>);

[0046] (3) Label compression: Each label string s is mapped to a new label through the mapping function f if and only if When , the compression tags are the same;

[0047] (4) Relabeling: Use the compressed label as the i-th round label of node v in the subgraph;

[0048] (5) Generate subgraph representation: After the iteration, the new label is mapped to a one-hot vector, and the representation of each subgraph is obtained through pooling operation, which is used to calculate the similarity between subgraphs.

[0049] Through the above heterogeneous isomorphism test steps, the number of HAW types has been effectively controlled. Next, the present invention uses the graph structure topic model to capture the local structural pattern (role characteristics) of the nodes in the heterogeneous graph. The graph structure topic model is a probabilistic model, so it can more accurately capture the distribution differences of the local structural patterns of the nodes. First, the graph structure topic model is defined as follows: Given a graph H = (V, E, φ), W l It is a set of heterogeneous anonymous walks of length l, with the number of structural topics being K. The topic model on the heterogeneous graph aims to learn a node-topic matrix Among them, row R i Corresponding to a distribution, R ik Represents node v i The probability of belonging to the k-th structural topic. In addition, the definition from v i The starting set of heterogeneous anonymous walks is W (i) , N=|W (i) | is the number of walks starting from each node.

[0050] Graph-structured topics are similar to topic modeling in natural language processing (NLP), where heterogeneous anonymous walks correspond to words, and the set of traversals starting from each node corresponds to documents. By analogy, nodes can be probabilistically described by their local structural patterns, so structural topics will contain the distribution of structural patterns that indicate node attributes. Since random walks in heterogeneous networks generate a large number of diverse walk sequences, some of which are meaningless, the model may overfit these meaningless sequences and ignore more important ones, thus affecting model efficiency and accuracy. Therefore, the model must first select highly indicative anchor structures from the numerous structural patterns before further topic modeling. To illustrate the graph-structured topic module's ability to select anchor structures, the selected anchor structures must not only reflect walk co-occurrence but also reflect heterogeneous structural topics. Specifically, if the walk-topic matrix U is separable, then walk sequences with non-zero weights are called anchor structures.

[0051] Next, the model first defines the walk co-occurrence matrix It represents the number of co-occurrences of two walk sequences starting from different nodes. Then, the co-occurrence matrix M is subjected to non-negative matrix decomposition technology to extract the anchor structure:

[0052]

[0053] Iteratively update H, Z until convergence, then pass A k =argmax(Z k ) Select the walk with the largest weight as the indicative walk, because the walk with the largest weight can better explain the occurrence of other walks, where k = 1,…,α, A is the index set of the anchor structure, and Z is the k-th row of Z. Therefore, the walk selected not only shows the co-occurrence of walks, but also indicates the underlying topic structure.

[0054] Based on the selected anchor structure, we continue to learn the walk-topic distribution U. We use the LDA fast optimization method with the anchor structure as the main indicator and the non-anchor structure as auxiliary information to obtain

[0055]

[0056] Among them, Q is the rearranged walk co-occurrence matrix, and the anchor structure A is located in the first α rows. is the corresponding row of the k-th anchor structure in Q. In addition, this model defines the node-walk matrix as Y i,w Indicates that w is in D i Finally, through:

[0057]

[0058] Obtain the node-topic distribution where denotes the pseudo-inverse. Thus, the topic structure modeling of the nodes is completed, and the node representation containing heterogeneous structures is obtained.

[0059] In a specific application example, the present invention proposes a role-oriented heterogeneous network representation learning algorithm, which combines project paper data to mine the roles of authors in the network. The main steps include the following:

[0060] Step 1, data preprocessing. The data set contains information in multiple aspects such as papers, scholars, institutions, etc. The meanings represented by each field in the data set are different. After being collected from the basic research knowledge base of the National Natural Science Foundation of China, the original data is directly stored in csv format. Since the data types of the original data are not completely unified, there are also many dirty data and inconsistent data fields. First, the data is cleaned to remove the data with missing fields, and then the cleaned data is preprocessed. In addition, due to the large number of duplicate names in the original data, and the names of the authors exist in both Chinese and pinyin forms, such as "Dong Guohua" and "DongGuohua", this will undoubtedly have a great impact on network construction. In order to reduce the impact brought by the duplicate name problem, this paper first sets a unique ID for all institutions, then constructs an author dictionary according to the relationship of the author's affiliated institutions, sets a unique ID index for the authors, and finally uses the pypinyin tool in python to convert the names of the authors into pinyin. For example, "10026-dongguohua".

[0061] Step 2, heterogeneous network construction. Since the number of nodes and edges in the original network is too large, some of the data is extracted for case analysis. This chapter selects projects and papers in related fields such as artificial intelligence, computer vision, and cloud computing under the discipline of information science to construct a heterogeneous network. Specifically, the heterogeneous information network constructed in this paper includes 3 node types: authors, papers, and projects. The relationships of authors publishing papers and papers being affiliated with projects are used as the edges between nodes to construct an author relationship network. The project cooperation network constructed by the present invention contains 206,095 author nodes and 368,628 cooperation edges, including 123,104 author nodes, 73,549 paper nodes, and 8,442 project nodes; 294,919 paper-author edges and 73,709 paper-project edges. The number of author nodes is much larger than the number of project nodes, and the number of paper-author connections is 3 to 4 times that of paper-project connections, which indicates that there are significant differences in the number of different types of nodes in the complex networks in the real world, and this characteristic will have a certain impact on the accuracy of the algorithm.

[0062] Step 3: Role Discovery. First, embedding representations are generated for all nodes in the heterogeneous project collaboration network. The similarity between nodes is then calculated using the Euclidean distance formula. For any given scholar ID, the top-k similar author list can be obtained by calculating the similarity between the embedding vectors of that scholar and other scholars.

[0063] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0064] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. However, these modifications, changes, or replacements do not deviate from the essence of the corresponding technical solutions within the scope of the technical solutions of the embodiments of the present invention. They should all be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A role-oriented heterogeneous network representation learning method, comprising: Obtain relevant data for a specific task, perform data cleaning and preprocessing, and construct a heterogeneous network. The dataset contains information on papers, scholars, and institutions, and each field in the dataset represents a different meaning. Utilize heterogeneous anonymous walking modules to capture node neighborhood structure information; Use the same structure merging module to merge similar anonymous walk paths, including: The isomorphism test method is used to treat HAW samples with different structural patterns but isomorphic as the same type, so as to alleviate the problem of the number of HAW sequence types increasing with the increase of walk length l; The specific steps of heterogeneous homogeneous testing are as follows: (1) Definition of label composite set: Given two labeled heterogeneous subgraphs H and H', for each node, its label is set to the combination of HAW's location id and node type; if it is the first iteration, the label composite set of node v only contains its own label; if it is not the first iteration, the label composite set elements of node v are the labels generated by node v's neighbor nodes in the previous iteration; (2) Sorting the composite set: Aggregate the labels of the neighboring nodes of node v to obtain a new label, sort the elements in the composite set in ascending order according to the position id, and concatenate the sorted elements into a string s, with the label generated by node v in the previous iteration as the prefix; (3) Label compression: Each label string s is mapped to a new label through the mapping function f if and only if When , the compression tags are the same; (4) Relabeling: Use the compressed label as the i-th round label of node v in the subgraph; (5) Generate subgraph representation: After the iteration, the new label is mapped to a one-hot vector, and the representation of each subgraph is obtained through pooling operation, which is used to calculate the similarity between subgraphs; The topic model is used to obtain role-oriented node representation; among them, the heterogeneous information network includes three node types: author, paper, and project. The relationship between authors publishing papers and papers affiliated with projects is used as the edge between nodes to construct the author relationship network.

2. The role-oriented heterogeneous network representation learning method according to claim 1, characterized in that: Obtain relevant data for a specific task and perform data cleaning and preprocessing, including: Filter relevant fields in the data to build a heterogeneous information network, remove garbled characters and delete data with empty fields to reduce noise.

3. The role-oriented heterogeneous network representation learning method according to claim 2, characterized in that: A heterogeneous anonymous walk module is used to capture node neighborhood structure information, including: Different node sequences are obtained by random walk sampling, and the structural characteristics of the node neighborhood are obtained by anonymizing the walk sequences.

4. The role-oriented heterogeneous network representation learning method according to claim 1, characterized in that The topic model is used to obtain role-oriented node representations, including: Using the topic model, a probabilistic description of the local structural pattern of each node is obtained, and the final structural topic will contain the structural pattern distribution indicating the node attributes.

Images