Community discovery method and device based on heterogeneous graph

Abstract

The embodiment of the invention provides a community discovery method and device based on a heterogeneous graph. The method comprises a plurality of rounds of graph iteration, and at least one round of graph iteration comprises the following steps: carrying out local movement processing on each entity vertex representing a current community; the local movement processing comprises the following steps: traversing each neighbor entity vertex meeting a preset relationship with the neighbor entity vertex, and determining the gain of a deterministic index after the entity vertex is moved to a community represented by the neighbor entity vertex; according to the gain, determining whether the movement is carried out or not; the deterministic index is determined according to distribution of feature values of various auxiliary vertexes associated with each entity vertex in a community; and for each community obtained after local movement processing, aggregating a plurality of entity vertexes in any community into a new entity vertex to represent the community. And a stable and effective community discovery result can be obtained.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the computer field, and more particularly, to a community discovery method and apparatus based on a heterogeneous graph. Background Art

[0002] In mathematics, a graph is a structure composed of a set of vertices (also called nodes or points) and a set of edges (also called arcs or lines) connecting these vertices, which is used to simulate the relationship between entities. There are two basic elements in the graph structure, namely vertices and edges. Among them, vertices are the basic units in the graph, representing entities or objects. Edges, also called connecting edges, are line segments connecting two vertices in the graph, representing a certain relationship or connection between vertices. Graph data may contain users' private data.

[0003] Graph structures include homogeneous graphs and heterogeneous graphs. Homogeneous graphs: All vertices in the graph belong to the same category and have the same type of edges. Heterogeneous graphs: The graph contains different types of vertices and edges, and vertices and edges are divided into different categories according to their attributes and meanings.

[0004] Currently, community discovery and other problems are often solved based on graph algorithms. It can be understood that graph algorithms are a series of algorithms executed on graph data. Community discovery is the process of identifying groups of vertices with similarities or dense connections in graph data, where the connection density inside these groups is higher than the connection density outside. In the prior art, community discovery is often performed based on homogeneous graphs, and no stable and effective community discovery solution based on heterogeneous graphs is provided. Summary of the invention

[0005] One or more embodiments of this specification describe a community discovery method and device based on a heterogeneous graph, which can obtain stable and effective community discovery results.

[0006] In a first aspect, a community discovery method based on a heterogeneous graph is provided, wherein the heterogeneous graph includes M entity vertices representing M users, and a plurality of auxiliary vertices representing respective optional feature values ​​of N types of features of the users, wherein any entity vertex has a connection edge with the auxiliary vertex corresponding to the actual feature value of the user, and the method includes a plurality of rounds of graph iterations, wherein at least one round of graph iteration includes:

[0007] For each entity vertex representing the current community, a local movement process is performed; the local movement process includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community;

[0008] For each community obtained after local movement processing, several entity vertices in any community are aggregated into a new entity vertex to represent the community.

[0009] In a possible implementation, the N types of features include operation features when a user performs a preset network operation.

[0010] Furthermore, the preset network operation includes a login operation, a coupon collection operation or a click operation, and the operation characteristics include an application based on the operation, a mobile phone model based on the operation or a network protocol address based on the operation.

[0011] In a possible implementation, before the several rounds of graph iterations, the method further includes:

[0012] Initialize M communities, each of which contains one entity vertex.

[0013] In a possible implementation manner, the preset relationship includes: a second-degree neighbor entity vertex having at least one commonly connected auxiliary vertex with the second-degree neighbor entity vertex with the second-degree neighbor entity vertex having multiple commonly connected auxiliary vertices with the second-degree neighbor entity vertex with the second-degree neighbor entity vertex.

[0014] In a possible implementation, before the several rounds of graph iterations, the method further includes:

[0015] If the first entity vertex and the second entity vertex in the heterogeneous graph satisfy the preset relationship, adding the first entity vertex and the second entity vertex to the auxiliary graph, and adding a connecting edge between the two in the auxiliary graph;

[0016] The traversing of each neighbor entity vertex that satisfies a preset relationship includes:

[0017] In the auxiliary graph, entity vertices having connecting edges therewith are searched as neighbor entity vertices that satisfy a preset relationship therewith.

[0018] In a possible implementation, the certainty index of any community is determined in the following manner:

[0019] According to the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community, the information entropy of the auxiliary vertex of this type in the community is determined; the information entropy reflects the distribution of feature values;

[0020] According to the information entropy of the auxiliary vertex, the certainty index of the auxiliary vertex is determined; wherein, the smaller the information entropy, the larger the certainty index;

[0021] Based on the weighted sum of the certainty indicators of various auxiliary vertices, the certainty indicator of the community is obtained.

[0022] Furthermore, determining the information entropy of the auxiliary vertex in the community includes:

[0023] Determine the occurrence probability of each feature value according to the ratio of the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community to the number of entity vertices M;

[0024] According to the occurrence probability of each feature value, the uncertainty of the feature value is calculated to obtain the initial information entropy of this type of auxiliary vertex in the community;

[0025] Determine whether the initial information entropy is less than or equal to a preset value;

[0026] If it is determined that the initial information entropy is less than or equal to the preset value, the initial information entropy is retained unchanged;

[0027] If it is determined that the initial information entropy is greater than a preset value, the initial information entropy is updated to the preset value.

[0028] Furthermore, the weighted number of occurrences of the feature value is determined in the following manner:

[0029] The weighted number of occurrences of the feature value is determined according to the number of auxiliary vertices corresponding to the feature value connected to each entity vertex in the community and the edge weight of the connecting edge.

[0030] In a possible implementation, the method further includes:

[0031] A plurality of connection edges between a plurality of entity vertices in any community and the same auxiliary vertex are aggregated into a new connection edge, wherein the edge weight of the new connection edge is the sum of the edge weights of the plurality of connection edges before the aggregation.

[0032] In a possible implementation manner, determining whether to perform the movement according to the gain includes:

[0033] Determine whether there is a neighbor entity vertex with a positive gain;

[0034] If there is a neighbor entity vertex with a positive gain, move the entity vertex to the community where the neighbor entity vertex with the largest gain is located;

[0035] If there is no neighbor entity vertex with a positive gain, the movement is not performed.

[0036] In a possible implementation, the at least one round of graph iteration further includes:

[0037] Determine whether the number of communities obtained in the current round is reduced compared to the number of communities obtained in the previous round;

[0038] If the number of communities decreases, the next round of graph iteration is triggered, otherwise the graph iteration is terminated.

[0039] In a possible implementation, the several rounds of graph iteration form multi-level communities, and a hierarchical relationship is formed between the community obtained by the N+1th round of graph iteration and the community obtained by the Nth round of graph iteration.

[0040] In a second aspect, a community discovery device based on a heterogeneous graph is provided, wherein the heterogeneous graph includes M entity vertices representing M users, and a plurality of auxiliary vertices representing respective optional feature values ​​of N types of features of the users, and a connecting edge is provided between any entity vertex and the auxiliary vertex corresponding to the actual feature value of the user, and the device is used to perform at least one round of graph iteration among a plurality of rounds of graph iteration, including:

[0041] A moving unit is used to perform local movement processing for each entity vertex representing the current community; the local movement processing includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of feature values ​​of various auxiliary vertices associated with each entity vertex in the community;

[0042] The aggregation unit is used for aggregating several entity vertices in any community into a new entity vertex to represent the community after the mobile unit performs local movement processing on each community.

[0043] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0044] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0045] Through the method and device provided by the embodiment of this specification, in at least one round of graph iteration, firstly, for each entity vertex representing the current community, a local movement process is performed; the local movement process includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community; then, for each community obtained after the local movement process, several entity vertices in any community are aggregated into a new entity vertex to represent the community. As can be seen from the above, the embodiment of this specification can make full use of the auxiliary vertex information for community discovery, while avoiding adding auxiliary vertices to the community, and effectively adapting to the heterogeneous graph structure. In addition, a new indicator calculation method for deterministic indicators for community discovery is proposed, which evaluates the closeness of the community according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community, and determines the community to which each entity vertex belongs through several rounds of graph iteration based on the deterministic index, without randomness problems, and multiple calculation results are consistent, so that stable and effective community discovery results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0047] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

[0048] Figure 2 A flow chart of a community discovery method based on a heterogeneous graph according to an embodiment is shown;

[0049] Figure 3 A schematic diagram of an auxiliary diagram according to an embodiment is shown;

[0050] Figure 4 A schematic block diagram of a community discovery apparatus based on a heterogeneous graph according to an embodiment is shown. DETAILED DESCRIPTION

[0051] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0052] Figure 1This is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. This implementation scenario involves community discovery based on heterogeneous graphs. It can be understood that a community usually includes multiple subjects, which can also be called entities. Community discovery is often used to identify group control subjects. In other words, the identification of group control subjects is a typical application scenario of community discovery.

[0053] The following is a brief description of the group control scenario. Machine traffic is the use of machine means to automatically implement certain operations and requests, such as logging in, collecting coupons, and clicking operations. The purposes of machine traffic include brushing, crawling, etc. Group control is the most serious type of machine traffic, usually referring to the centralized or distributed control of a group of intelligent agents or devices, guiding the overall behavior to achieve a predetermined goal under specific environments or conditions. For example, multiple mobile phones are controlled by one or more computers to automatically perform a series of tasks or operations. The group control system is completed through the collaboration of software and hardware. The hardware devices may include group control hosts, hubs, computer hosts, and terminal mobile phones. The software provides a sub-control function, allowing multiple mobile phones to be controlled in a local LAN environment. Among them, the aforementioned intelligent agents or devices each correspond to a corresponding user, and the identification of the group control subject is the identification of the user participating in the group control. The identification of the group control subject is a long-term capability in the field of traffic security, and has played a huge role in advertising, marketing, search and push, etc. In the embodiment of this specification, through community discovery based on heterogeneous graphs, support is provided for the identification of group control groups composed of group control subjects, which is conducive to maintaining traffic security.

[0054] Reference Figure 1 , shows a typical example of a heterogeneous graph, which includes M entity vertices representing M users and several auxiliary vertices representing the optional feature values ​​of N types of features of the users, and there is a connecting edge between any entity vertex and the auxiliary vertex corresponding to the actual feature value of the user. For example, Figure 1The entity vertices U1, U2, U3, and U4 represent users U1, U2, U3, and U4 respectively, and the auxiliary vertices A1, A2, B1, B2, and B3, wherein the auxiliary vertices A1 and A2 correspond to two optional feature values ​​of feature A respectively, and the auxiliary vertices B1, B2, and B3 correspond to three optional feature values ​​of feature B respectively. The actual feature value of feature A of user U1 corresponds to auxiliary vertex A1, and there is a connection edge between entity vertex U1 and auxiliary vertex A1; the actual feature value of feature B of user U1 corresponds to auxiliary vertex B1, and there is a connection edge between entity vertex U1 and auxiliary vertex B1; the actual feature value of feature A of user U2 corresponds to auxiliary vertex A2, and there is a connection edge between entity vertex U2 and auxiliary vertex A2; the actual feature value of feature B of user U2 corresponds to auxiliary vertex B2, and there is a connection edge between entity vertex U2 and auxiliary vertex B2; the actual feature value of feature A of user U3 corresponds to auxiliary vertex A1, and there is a connection edge between entity vertex U3 and auxiliary vertex A1; the actual feature value of feature B of user U3 corresponds to auxiliary vertex B3, and there is a connection edge between entity vertex U3 and auxiliary vertex B3; the actual feature value of feature A of user U4 corresponds to auxiliary vertex A2, and there is a connection edge between entity vertex U4 and auxiliary vertex A2; the actual feature value of feature B of user U4 corresponds to auxiliary vertex B2, and there is a connection edge between entity vertex U4 and auxiliary vertex B2.

[0055] In the embodiments of this specification, the heterogeneous graph may be a directed graph or an undirected graph, and the connecting edges may have edge weights or may not have edge weights.

[0056] The following are some basic concepts about graphs.

[0057] Directed graph: A graph with directed edges, indicating that the relationship between vertices is directed, from one vertex to another.

[0058] Undirected graph: A graph with no direction on the edges, indicating that the relationship between vertices is bidirectional, or has no specific direction.

[0059] Edge weight: A numerical value assigned to an edge in a graph that represents some property of the edge, such as the cost or strength of traveling from one vertex to another.

[0060] Weighted graph: The edges in the graph are assigned edge weights, such as edge weights representing length, time, cost or capacity.

[0061] Unweighted graph: The edges in the graph have no edge weights, or all edges have the same edge weight, which only indicates the existence of a connection without conveying other information.

[0062] It should be noted that Figure 1As just an example of a heterogeneous graph, in practice, the scale of a heterogeneous graph will be much larger, usually containing more entity vertices and auxiliary vertices, for example, the number of vertices reaches hundreds of thousands or millions.

[0063] In the embodiment of this specification, the vertices in the heterogeneous graph are divided into entity vertices and auxiliary vertices, and the entity vertices are divided into non-overlapping communities. The relationship between the entity vertices is not considered, that is, there is no association relationship within the community. In order to obtain stable and effective community discovery results, a corresponding solution is provided.

[0064] Figure 2 A flowchart of a community discovery method based on a heterogeneous graph according to an embodiment is shown. The heterogeneous graph includes M entity vertices representing M users and several auxiliary vertices representing the optional feature values ​​of N types of features of the users. There is a connecting edge between any entity vertex and the auxiliary vertex corresponding to the actual feature value of the user. The method can be based on Figure 1 The implementation scenario shown in Figure 1 is as follows. Figure 2 As shown, the community discovery method based on heterogeneous graphs in this embodiment includes several rounds of graph iterations, wherein at least one round of graph iterations includes the following steps: Step 21, for each entity vertex representing the current community, local movement processing is performed; the local movement processing includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community; Step 22, for each community obtained after the local movement processing, aggregating several entity vertices in any community into a new entity vertex to represent the community. The specific execution method of each of the above steps is described below.

[0065] First, in step 21, for each entity vertex representing the current community, a local movement process is performed; the local movement process includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of the feature values ​​of various auxiliary vertices associated with each entity vertex in the community. It can be understood that the distribution of the feature values ​​of the auxiliary vertices in the community should be more compact, and the compactness is measured by the deterministic index to determine whether the entity vertex is moved to join the community.

[0066] In one example, the N types of features include operation features when a user performs a preset network operation.

[0067] In this example, the above-mentioned preset network operation is an operation that may be executed automatically. By analyzing and processing the heterogeneous graph including its operation characteristics, it is helpful to discover the group control subject. Each group control subject constitutes a group control group, and the group control group is a community.

[0068] Furthermore, the preset network operation includes a login operation, a coupon collection operation or a click operation, and the operation characteristics include an application based on the operation, a mobile phone model based on the operation or a network protocol address based on the operation.

[0069] In this example, some typical operations in group control scenarios and the operation features extracted for community discovery are given. The above operation features are helpful for identifying group control subjects. For example, each group control subject in a group control group may use the same application, mobile phone model or network protocol address. Figure 1 In the figure, auxiliary vertices A1 and A2 correspond to two optional feature values ​​of feature A respectively, where feature A can represent the network protocol address on which the operation is based, the optional feature value corresponding to auxiliary vertex A1 is ip1, and the optional feature value corresponding to auxiliary vertex A2 is ip2.

[0070] In one example, before the several rounds of graph iterations, the method further includes:

[0071] Initialize M communities, each of which contains one entity vertex.

[0072] In this example, an entity vertex is initialized as a community. Each community contains only the entity vertex, and the number of communities is the same as the number of entity vertices. For example, Figure 1 In , four communities are initialized, and entity vertices U1, U2, U3, and U4 each correspond to a community.

[0073] In one example, the preset relationship includes: a second-degree neighbor entity vertex with which it has at least one auxiliary vertex connected in common; or a second-degree neighbor entity vertex with which it has multiple auxiliary vertices connected in common.

[0074] In this example, for the case where there is no direct association relationship between entity vertices, an indirect association relationship between the two is established through auxiliary vertices. When two entity vertices have at least one commonly connected auxiliary vertex, or when two entity vertices have multiple commonly connected auxiliary vertices, it is considered that the two satisfy the preset relationship, that is, the two may be in the same community. For example, Figure 1In the figure, entity vertices U1 and U3 are connected to auxiliary vertex A1, and there are two connecting edges between them, so they are each other's second-degree neighbor entity vertices. Entity vertices U2 and U4 are connected to auxiliary vertex A2, and they are connected to auxiliary vertex B2, and there are two connecting edges between them, so they are each other's second-degree neighbor entity vertices.

[0075] In one example, before the several rounds of graph iterations, the method further includes:

[0076] If the first entity vertex and the second entity vertex in the heterogeneous graph satisfy the preset relationship, adding the first entity vertex and the second entity vertex to the auxiliary graph, and adding a connecting edge between the two in the auxiliary graph;

[0077] The traversing of each neighbor entity vertex that satisfies a preset relationship includes:

[0078] In the auxiliary graph, entity vertices having connecting edges therewith are searched as neighbor entity vertices that satisfy a preset relationship therewith.

[0079] In this example, an auxiliary graph is pre-constructed, and all entity vertices that satisfy the preset relationship are added to the auxiliary graph, and there is an edge between two vertices that satisfy the preset relationship. The auxiliary graph can accelerate the local movement process, and by adding restrictions to the auxiliary graph, that is, setting the above-mentioned preset relationship, the community size can be further controlled and the calculation can be accelerated, so that small-scale communities can be detected more accurately. For example, two entity vertices are required to be connected to more than three identical auxiliary vertices before these two entity vertices are added to the auxiliary graph. It can be understood that the auxiliary graph only contains entity vertices, not auxiliary vertices.

[0080] Figure 3 A schematic diagram of an auxiliary diagram according to an embodiment is shown. Figure 3 , assuming that the aforementioned preset relationship is specifically the second-degree neighbor entity vertex with which it has at least one commonly connected auxiliary vertex. Corresponding to Figure 1 In a heterogeneous graph, entity vertices U1 and U3 have a common connected auxiliary vertex A1. Entity vertices U1 and U3 are added to the auxiliary graph, and a connecting edge between them is added to the auxiliary graph; entity vertices U2 and U4 have a common connected auxiliary vertex A2. Entity vertices U2 and U4 are added to the auxiliary graph, and a connecting edge between them is added to the auxiliary graph. It should be noted that Figure 3 This is just an example of an auxiliary graph. In practice, the auxiliary graph will be much larger and usually contain more entity vertices, for example, the number of vertices reaches hundreds of thousands or millions.

[0081] In one example, the certainty index for any community is determined as follows:

[0082] According to the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community, the information entropy of the auxiliary vertex of this type in the community is determined; the information entropy reflects the distribution of feature values;

[0083] According to the information entropy of the auxiliary vertex, the certainty index of the auxiliary vertex is determined; wherein, the smaller the information entropy, the larger the certainty index;

[0084] Based on the weighted sum of the certainty indicators of various auxiliary vertices, the certainty indicator of the community is obtained.

[0085] In this example, the deterministic index is determined based on the information entropy of the auxiliary vertices, where the information entropy can evaluate the closeness of the community. The larger the information entropy, the greater the uncertainty of the information, the more dispersed the auxiliary nodes in the community are, and the less they conform to the form of the community. Therefore, the deterministic index can be used to determine whether the entity vertex moves into a community. It can be understood that each type of auxiliary vertex has a corresponding category weight, and the above weighted summation is performed based on the category weight.

[0086] Furthermore, determining the information entropy of the auxiliary vertex in the community includes:

[0087] Determine the occurrence probability of each feature value according to the ratio of the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community to the number of entity vertices M;

[0088] According to the occurrence probability of each feature value, the uncertainty of the feature value is calculated to obtain the initial information entropy of this type of auxiliary vertex in the community;

[0089] Determine whether the initial information entropy is less than or equal to a preset value;

[0090] If it is determined that the initial information entropy is less than or equal to the preset value, the initial information entropy is retained unchanged;

[0091] If it is determined that the initial information entropy is greater than a preset value, the initial information entropy is updated to the preset value.

[0092] In this example, the information entropy is controlled within a preset value, so as to avoid the information entropy of one type of auxiliary vertices among multiple types of auxiliary vertices being too large, so that the information entropy of other types of auxiliary vertices has too little effect on the certainty index of the community.

[0093] Furthermore, the weighted number of occurrences of the feature value is determined in the following manner:

[0094] The weighted number of occurrences of the feature value is determined according to the number of auxiliary vertices corresponding to the feature value connected to each entity vertex in the community and the edge weight of the connecting edge.

[0095] In this example, when the heterogeneous graph is an unweighted graph, the edge weights of each connection edge can be considered to be the same. When the heterogeneous graph is a weighted graph, the edge weights of each connection edge can also be initially set to be the same. In the community, there can be multiple entity vertices connected to the same auxiliary vertex. The number of auxiliary vertices can be understood as the number of connection edges between multiple entity vertices in the community and the same auxiliary vertex. For example, Figure 1 In , if the community includes entity vertices U1 and U3, then the number of auxiliary vertex A1 in the community is 2.

[0096] In one example, determining whether to perform the movement according to the gain includes:

[0097] Determine whether there is a neighbor entity vertex with a positive gain;

[0098] If there is a neighbor entity vertex with a positive gain, move the entity vertex to the community where the neighbor entity vertex with the largest gain is located;

[0099] If there is no neighbor entity vertex with a positive gain, the movement is not performed.

[0100] In this example, for each entity vertex representing the current community, there may be multiple neighbor entity vertices that satisfy a preset relationship with it. Whether the entity vertex moves and to which community the neighbor entity vertex moves can be determined based on the gain of the deterministic indicator.

[0101] The following example illustrates the calculation process of the certainty index.

[0102] Assume that there are N types of auxiliary vertices, and the sample space of each type is S i , the auxiliary vertices directly connected to the entity vertex t constitute a set S t , the set size is N. For example, there are two types of auxiliary vertices, including the application apk based on the operation and the network protocol address ip based on the operation, then the auxiliary vertex set connected to the entity vertex may be: {'apk':{'apk1','apk2'}, "ip":{"ip1","ip2"}}. It can be understood that apk1 and apk2 are two optional feature values ​​of the feature apk, and ip1 and ip2 are two optional feature values ​​of the feature ip.

[0103] The set of auxiliary vertices directly connected to community C is S C, which is a set of sets, with a size of N. For example, there are two types of auxiliary vertices, including the application apk based on the operation and the network protocol address ip based on the operation, then the set of auxiliary vertices of the community may be:

[0104] {'apk':{"apk1":{"cnt":10}, "apk2":{"cnt":9}},

[0105] ip':{"ip1":{"cnt":8},"ip2":{"cnt":9}}}. Among them, cnt represents the weighted occurrence number of feature values. For example, the weighted occurrence number of feature value apk1 in this set is 10, the weighted occurrence number of feature value apk2 is 9, the weighted occurrence number of feature value ip1 is 8, and the weighted occurrence number of feature value ip2 is 9.

[0106] The node size of community C is recorded as com_size. For example, com_size = 20, which represents the number of entity vertices in the community. Then, on the auxiliary vertex of apk class ["cnt"] / com_s i ze = {'item1': 0.5, 'item2': 0.45}, it can be understood that item1 is the probability of occurrence of the feature value apk1, and item2 is the probability of occurrence of the feature value apk2. Then the information entropy of the community distributed on the auxiliary vertices of the apk class is calculated as -0.5*log(0.5)-0.45*log(0.45). The greater the information entropy of the distribution, the greater the uncertainty of the information, the more dispersed the auxiliary vertices in the community are, and the less it conforms to the form of the community. In addition, the information entropy can be controlled within a preset value. For example, the preset value is 10. When the information entropy is greater than 10, the information entropy is reset to 10.

[0107] Define the certainty index of a single type of auxiliary vertex certainty = 1-min(entropy,10) / 10. The larger this value is, the closer the information entropy is to 0, which means the distribution is more concentrated and more in line with the community form. Among them, entropy represents the initial information entropy.

[0108] Defining Deterministic Indicators for Communities Among them, com_size represents the number of entity vertices in the community, i represents the category of auxiliary vertices, and entropy i represents the initial information entropy of the i-th auxiliary vertex, weight i Represents the category weight of the i-th auxiliary vertex.

[0109] It should be noted that there is no unique way to calculate the certainty index. The above is only an example, and various changes can be made on this basis, such as multiplying by a preset constant, dividing by a preset constant, etc., which are not listed here one by one.

[0110] Then, in step 22, for each community obtained after the local movement processing, several entity vertices in any community are aggregated into a new entity vertex to represent the community. It can be understood that the purpose of aggregation is to facilitate the next round of graph iteration, and step 21 is continued.

[0111] In one example, the method further includes:

[0112] A plurality of connection edges between a plurality of entity vertices in any community and the same auxiliary vertex are aggregated into a new connection edge, wherein the edge weight of the new connection edge is the sum of the edge weights of the plurality of connection edges before the aggregation.

[0113] In this example, for the aggregation of entity vertices and connecting edges in any community, the edge weights of the connecting edges are updated, so as to facilitate the determination of the weighted occurrence number of feature values ​​in subsequent graph iterations and further calculation of deterministic indicators.

[0114] In one example, the at least one round of graph iteration further includes:

[0115] Determine whether the number of communities obtained in the current round is reduced compared to the number of communities obtained in the previous round;

[0116] If the number of communities decreases, the next round of graph iteration is triggered, otherwise the graph iteration is terminated.

[0117] In this example, a termination condition for graph iteration is given, that is, the number of communities remains unchanged. After the graph iteration is terminated, the community results obtained by the last round of graph iteration can be output. For example, a total of 2 communities are obtained, community 1 includes entity vertices U1 and U2, and community 2 includes entity vertices U3 and U4. There is no overlap between the two communities, that is, any entity vertex has a unique community to which it belongs.

[0118] In one example, the several rounds of graph iteration form multi-level communities, and a hierarchical relationship is formed between the community obtained by the N+1th round of graph iteration and the community obtained by the Nth round of graph iteration.

[0119] For example, a community obtained in the Nth round of graph iteration includes entity vertices U1 and U2, and a community obtained in the N+1th round of graph iteration includes entity vertices U1, U2, and U4, and a hierarchical relationship is formed between the two communities.

[0120] Through the method provided by the embodiment of this specification, in at least one round of graph iteration, firstly, for each entity vertex representing the current community, a local movement process is performed; the local movement process includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community; then for each community obtained after the local movement process, several entity vertices in any community are aggregated into a new entity vertex to represent the community. As can be seen from the above, the embodiment of this specification can make full use of the auxiliary vertex information for community discovery, while avoiding adding auxiliary vertices to the community, and effectively adapting to the heterogeneous graph structure. In addition, a new indicator calculation method for deterministic indicators for community discovery is proposed, which evaluates the closeness of the community according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community, and determines the community to which each entity vertex belongs through several rounds of graph iteration based on the deterministic index, without randomness problems, and multiple calculation results are consistent, so that stable and effective community discovery results can be obtained.

[0121] According to an embodiment of another aspect, a community discovery device based on a heterogeneous graph is also provided, wherein the heterogeneous graph includes M entity vertices representing M users, and a number of auxiliary vertices representing each optional feature value of N types of features of the user, and there is a connecting edge between any entity vertex and the auxiliary vertex corresponding to the actual feature value of the user, and the device is used to execute the method provided in the embodiment of this specification and execute at least one round of graph iteration among several rounds of graph iteration. Figure 4 FIG. 1 is a schematic block diagram of a community discovery apparatus based on a heterogeneous graph according to an embodiment. Figure 4 As shown, the device 400 includes:

[0122] The moving unit 41 is used to perform local movement processing for each entity vertex representing the current community; the local movement processing includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of feature values ​​of various auxiliary vertices associated with each entity vertex in the community;

[0123] The aggregation unit 42 is used for aggregating several entity vertices in any community obtained after the local movement processing by the movement unit 41 into a new entity vertex to represent the community.

[0124] Optionally, as an embodiment, the N types of features include operation features when the user performs preset network operations.

[0125] Furthermore, the preset network operation includes a login operation, a coupon collection operation or a click operation, and the operation characteristics include an application based on the operation, a mobile phone model based on the operation or a network protocol address based on the operation.

[0126] Optionally, as an embodiment, the device further includes:

[0127] The initialization unit is used to initialize M communities before the several rounds of graph iterations, each community including an entity vertex.

[0128] Optionally, as an embodiment, the preset relationship includes: a second-degree neighbor entity vertex with which it has at least one auxiliary vertex connected in common; or a second-degree neighbor entity vertex with which it has multiple auxiliary vertices connected in common.

[0129] Optionally, as an embodiment, the device further includes:

[0130] A construction unit, configured to, before the plurality of rounds of graph iterations, if the first entity vertex and the second entity vertex in the heterogeneous graph satisfy the preset relationship, add the first entity vertex and the second entity vertex to the auxiliary graph, and add a connecting edge between the first entity vertex and the second entity vertex in the auxiliary graph;

[0131] The moving unit 41 is specifically used to search for entity vertices having connecting edges with the auxiliary graph obtained by the building unit as neighbor entity vertices that satisfy a preset relationship with the auxiliary graph.

[0132] Optionally, as an embodiment, the certainty index of any community is determined in the following manner:

[0133] According to the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community, the information entropy of the auxiliary vertex of this type in the community is determined; the information entropy reflects the distribution of feature values;

[0134] According to the information entropy of the auxiliary vertex, the certainty index of the auxiliary vertex is determined; wherein, the smaller the information entropy, the larger the certainty index;

[0135] Based on the weighted sum of the certainty indicators of various auxiliary vertices, the certainty indicator of the community is obtained.

[0136] Furthermore, determining the information entropy of the auxiliary vertex in the community includes:

[0137] Determine the occurrence probability of each feature value according to the ratio of the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community to the number of entity vertices M;

[0138] According to the occurrence probability of each feature value, the uncertainty of the feature value is calculated to obtain the initial information entropy of this type of auxiliary vertex in the community;

[0139] Determine whether the initial information entropy is less than or equal to a preset value;

[0140] If it is determined that the initial information entropy is less than or equal to the preset value, the initial information entropy is retained unchanged;

[0141] If it is determined that the initial information entropy is greater than a preset value, the initial information entropy is updated to the preset value.

[0142] Furthermore, the weighted number of occurrences of the feature value is determined in the following manner:

[0143] The weighted number of occurrences of the feature value is determined according to the number of auxiliary vertices corresponding to the feature value connected to each entity vertex in the community and the edge weight of the connecting edge.

[0144] Optionally, as an embodiment, the aggregation unit 42 is also used to aggregate several connection edges between several entity vertices in any community and the same auxiliary vertex into a new connection edge, and the edge weight of the new connection edge is the sum of the edge weights of the several connection edges before the aggregation.

[0145] Optionally, as an embodiment, the mobile unit 41 includes:

[0146] A judging subunit, used for judging whether there is a neighbor entity vertex with a positive gain;

[0147] A moving subunit is used to move the entity vertex to the community where the neighbor entity vertex with the largest gain is located if the judging subunit judges that there is a neighbor entity vertex with the positive gain;

[0148] The non-movement subunit is used for not performing the movement if the determination subunit determines that there is no neighbor entity vertex with a positive gain.

[0149] Optionally, as an embodiment, the device further includes:

[0150] A judging unit, configured to judge, in at least one round of graph iteration, whether the number of communities obtained in the current round is reduced compared with the number of communities obtained in the previous round;

[0151] The trigger unit is used to trigger the next round of graph iteration if the judgment unit determines that the number of communities is reduced, otherwise the graph iteration is terminated.

[0152] Optionally, as an embodiment, the several rounds of graph iteration form a multi-level community, and a hierarchical relationship is formed between the community obtained by the N+1th round of graph iteration and the community obtained by the Nth round of graph iteration.

[0153] Through the device provided by the embodiment of this specification, in at least one round of graph iteration, the mobile unit 41 first performs local movement processing for each entity vertex representing the current community; the local movement processing includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, and determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; according to the gain, determining whether to perform the movement; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community; then the aggregation unit 42 aggregates several entity vertices in any community into a new entity vertex to represent the community for each community obtained after the local movement processing. As can be seen from the above, the embodiment of this specification can make full use of the auxiliary vertex information for community discovery, while avoiding adding auxiliary vertices to the community, and effectively adapting to the heterogeneous graph structure. In addition, a new method for calculating deterministic indicators for community discovery is proposed. The closeness of the community is evaluated according to the distribution of feature values ​​of various auxiliary vertices associated with each entity vertex in the community. The community to which each entity vertex belongs is determined through several rounds of graph iterations based on the deterministic indicator. There is no randomness problem, and the results of multiple calculations are consistent, so stable and effective community discovery results can be obtained.

[0154] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described.

[0155] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 2 The method described.

[0156] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0157] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A community discovery method based on a heterogeneous graph, wherein the heterogeneous graph includes M entity vertices representing M users and a plurality of auxiliary vertices representing respective optional feature values ​​of N types of features of the users, wherein there is a connecting edge between any entity vertex and the auxiliary vertex corresponding to the actual feature value of the user, and the method includes a plurality of graph iterations, wherein at least one graph iteration includes: For each entity vertex representing the current community, local movement processing is performed; The local movement processing includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; determining whether to perform the movement according to the gain; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community; For each community obtained after local movement processing, several entity vertices in any community are aggregated into a new entity vertex to represent the community.

2. The method of claim 1, wherein: The N types of features include operation features when a user performs a preset network operation.

3. The method of claim 2, wherein: The preset network operation includes a login operation, a coupon collection operation or a click operation, and the operation characteristics include an application based on the operation, a mobile phone model based on the operation or a network protocol address based on the operation.

4. The method of claim 1, wherein: Before the several rounds of graph iterations, the method further includes: Initialize M communities, each of which contains one entity vertex.

5. The method of claim 1, wherein: The preset relationship includes: a second-degree neighbor entity vertex with which it has at least one auxiliary vertex connected in common; or a second-degree neighbor entity vertex with which it has multiple auxiliary vertices connected in common.

6. The method of claim 1, wherein: Before the several rounds of graph iterations, the method further includes: If the first entity vertex and the second entity vertex in the heterogeneous graph satisfy the preset relationship, adding the first entity vertex and the second entity vertex to the auxiliary graph, and adding a connecting edge between the two in the auxiliary graph; The traversal of each neighbor entity vertex that satisfies a preset relationship includes: In the auxiliary graph, entity vertices having connecting edges therewith are searched as neighbor entity vertices that satisfy a preset relationship therewith.

7. The method of claim 1, wherein: The deterministic index for any community is determined as follows: According to the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community, the information entropy of the auxiliary vertex of this type in the community is determined; the information entropy reflects the distribution of feature values; According to the information entropy of the auxiliary vertex, the certainty index of the auxiliary vertex is determined; wherein, the smaller the information entropy, the larger the certainty index; Based on the weighted sum of the certainty indicators of various auxiliary vertices, the certainty indicator of the community is obtained.

8. The method of claim 7, wherein: The determining of the information entropy of the auxiliary vertex in the community includes: Determine the occurrence probability of each feature value according to the ratio of the weighted occurrence number of each feature value of any type of auxiliary vertex associated with each entity vertex in the community to the number of entity vertices M; According to the occurrence probability of each feature value, the uncertainty of the feature value is calculated to obtain the initial information entropy of this type of auxiliary vertex in the community; Determine whether the initial information entropy is less than or equal to a preset value; If it is determined that the initial information entropy is less than or equal to the preset value, the initial information entropy is retained unchanged; If it is determined that the initial information entropy is greater than a preset value, the initial information entropy is updated to the preset value.

9. The method of claim 7, wherein: The weighted number of occurrences of the feature value is determined in the following manner: The weighted number of occurrences of the feature value is determined according to the number of auxiliary vertices corresponding to the feature value connected to each entity vertex in the community and the edge weight of the connecting edge.

10. The method of claim 1, wherein: The method further comprises: A plurality of connection edges between a plurality of entity vertices in any community and the same auxiliary vertex are aggregated into a new connection edge, wherein the edge weight of the new connection edge is the sum of the edge weights of the plurality of connection edges before the aggregation.

11. The method of claim 1, wherein: The step of determining whether to perform the movement according to the gain includes: Determine whether there is a neighbor entity vertex with a positive gain; If there is a neighbor entity vertex with a positive gain, move the entity vertex to the community where the neighbor entity vertex with the largest gain is located; If there is no neighbor entity vertex with a positive gain, the movement is not performed.

12. The method of claim 1, wherein: The at least one round of graph iteration further includes: Determine whether the number of communities obtained in the current round is reduced compared to the number of communities obtained in the previous round; If the number of communities decreases, the next round of graph iteration is triggered, otherwise the graph iteration is terminated.

13. The method of claim 1, wherein: The several rounds of graph iteration form multi-level communities, and a hierarchical relationship is formed between the community obtained by the N+1th round of graph iteration and the community obtained by the Nth round of graph iteration.

14. A community discovery device based on a heterogeneous graph, the heterogeneous graph comprising M entity vertices representing M users, and a plurality of auxiliary vertices representing respective optional feature values ​​of N types of features of the users, wherein there is a connecting edge between any entity vertex and the auxiliary vertex corresponding to the actual feature value of the user, the device being used to perform at least one round of graph iteration among a plurality of rounds of graph iteration, comprising: A moving unit, used for performing local moving processing on each entity vertex representing the current community; The local movement processing includes: traversing each neighbor entity vertex that satisfies a preset relationship with it, determining the gain of the deterministic index after moving the entity vertex to the community represented by the neighbor entity vertex; determining whether to perform the movement according to the gain; the deterministic index is determined according to the distribution of the characteristic values ​​of various auxiliary vertices associated with each entity vertex in the community; The aggregation unit is used for aggregating several entity vertices in any community obtained after the mobile unit performs local movement processing into a new entity vertex to represent the community.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 13.

16. A computing device comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 13 is implemented.

Images