A method and system for predicting overlapping communities based on node information and label propagation

Abstract

This invention relates to a method and system for predicting overlapping communities based on node information and label propagation, belonging to the field of computer science. The method preprocesses user attribute information from acquired complex network datasets and processes the relational datasets within the dataset to construct the user network topology and generate a feature network. It transforms the correlation between key user attributes into an attribute modularity matrix, merges the topology matrix and the attribute modularity matrix into a weighted modularity matrix, and then decomposes the weighted modularity matrix to obtain a weighted vectorized representation of the user network. It multiplies the influence calculated from each attribute factor according to Bayes' probability formula to obtain the final user influence. Based on the user influence calculation method, it obtains the label importance required for community partitioning. It uses user label influence and user importance to discover communities. This invention provides more reliable prediction results for complex network community partitioning.

Description

Technical Field

[0001] This invention belongs to the field of computer science and relates to a method and system for predicting overlapping communities based on node information and label propagation. Background Technology

[0002] Complex networks are ubiquitous in all aspects of human life, including social networks, biological networks, engineering networks, and information networks. In the 1990s, Watts and Strogatz published their small-world network model in the top journal *Nature*, followed by Albert and Barbasi's scale-free network model in the top journal *Science*. This led to the emergence of complex networks as a new branch of study. Furthermore, complex social networks exhibit strong social effects. These effects manifest as diverse yet closely connected groups, with relatively frequent contact between individuals within each group. If an individual is divided into multiple groups, this is known as overlapping community detection.

[0003] Traditional community detection methods focus solely on network topology information. These methods are designed to divide communities by analyzing relationships between individuals, revealing close relationships within individual communities but sparser relationships between different communities. However, they fail to consider the attribute information carried by users within the network. Many existing classic overlapping community detection methods do not utilize user attribute information. Therefore, they have not yet been used to detect overlapping communities in networks. How to utilize user attribute information for prediction is a pressing problem that needs to be solved. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method and system for predicting overlapping communities based on node information and label propagation.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for predicting overlapping communities based on node information and label propagation, comprising the following steps:

[0007] S1: Preprocess the user attribute information in the acquired complex network dataset, including preprocessing such as removing non-text parts from short text documents of user attributes, word segmentation, and removal of stop words; process the relation dataset in the acquired dataset, including user relationship processing and removal of inactive users; and complete the construction of the user network topology and generate the feature network.

[0008] S2: Based on the feature network obtained in S1, feature processing is performed to capture key attribute information about the user from the overall distribution around the user attributes. Then, the correlation between the key user attributes is transformed into an attribute modularity matrix. Next, the topology matrix and the attribute modularity matrix are adaptively fused into a weighted modularity matrix. Then, the weighted modularity matrix is ​​decomposed into eigenvalues ​​and eigenvectors to obtain a weighted vectorized representation of the user network.

[0009] S3: Based on the key user attribute information obtained in S2, use a Bayesian network model to learn the prior probability of user node attributes. By identifying important user nodes, use relevant domain expert knowledge to obtain the prior probability of each attribute, learn the attribute values ​​with significant influence, establish a Bayesian network model of user attribute-influence, multiply the influence calculated by each attribute factor according to the Bayesian probability formula to obtain the final user influence, obtain the influence of all users in the network and sort them in ascending order;

[0010] S4: Based on the user influence information obtained in S3, calculate the network user importance using user prior attributes and total user influence, and then obtain the tag importance required in the community segmentation process based on the user influence calculation method.

[0011] S5: Based on the user tag importance and user importance calculation method obtained in S4, perform community discovery and obtain the community.

[0012] Optionally, the preprocessing operation on the dataset in S1 includes the following steps:

[0013] S11: Obtain the short text attribute information published by each user from the dataset. For this short text attribute information, remove non-text parts including HTML tags, non-English characters, punctuation marks, modal particles and metonyms. Use proprietary word segmentation technology on the LDA model corpus.

[0014] S12: Process the user relationships in the dataset, and then determine the activity of the users in the dataset. If a user has no connection with any other user, he / she is determined to be an inactive user and is removed.

[0015] Optionally, the specific steps of S2 include:

[0016] S21: Random walk is used to obtain empirical statistics of user feature pairs for user attribute information. Then, an unbiased estimate of the joint probability of the observed features is given, from which key user attribute information is extracted and normalized.

[0017] S22: Calculate the similarity of the Pearson correlation coefficient between key user attribute information. The closer the absolute value of the Pearson correlation coefficient is to 1, the stronger the linear relationship. When it is close to +1, the stronger the positive correlation is, and when it is close to -1, the stronger the negative correlation is. Then, construct the obtained similarity into an attribute modularity matrix, and then fuse the topology matrix and the attribute modularity matrix into a weighted modularity matrix through an adaptive fusion coefficient matrix.

[0018] S23: Obtain the weighted modularity matrix above, decompose the weighted modularity matrix into eigenvalues ​​and eigenvectors, and then map them into user vectors. Combine the user vectorization representation to obtain the weighted network matrix between users.

[0019] Optionally, in step S3, based on the Bayesian probability formula, the influence of each attribute is multiplied using the multiplication principle to obtain the final user influence. The user influence calculation formula is as follows:

[0020]

[0021] in, Indicates user influence. This indicates the influence of each user attribute.

[0022] Optionally, S4 specifically includes:

[0023] S41: Node normalization based on prior node attributes is insufficient; more importantly, there are close connections between users. Therefore, a user importance calculation method based on prior user attributes and user influence algorithms is proposed:

[0024]

[0025] in, On behalf of users The prior importance of Used to measure the relationship between adjacent users. The degree of impact; Indicates user neighborhood set, For users The degree;

[0026] S42. Based on user influence and user importance, the formula for calculating the importance of user tags in the network is as follows:

[0027]

[0028] Indicates to the user In the label Importance Indicates user Surrounding tags A set of.

[0029] Optionally, in step S5, community discovery is performed based on the obtained user tag importance and user importance calculation method. Specific steps include:

[0030] S51: In complex networks, each user is initialized using a unique community tag;

[0031] S52: Calculate the user importance and label importance of each user in the network, and initialize the iteration count to 1;

[0032] S53: Labels are propagated through the network in order of user importance. In each iteration of the propagation, each user updates their label to the label of their maximum number of neighbors. When the number of labels on a user's neighboring nodes reaches the maximum value, the label with the highest user label importance is selected to update the user.

[0033] S54: If the number of iterations reaches the maximum number of iterations or each user's label is the same as the labels of most of its neighboring users, then the users with the same label are placed in the same community and the operation ends; otherwise, the number of iterations is incremented by 1 and the process goes to S53.

[0034] An overlapping community partitioning prediction system based on node information and label propagation, the system comprising:

[0035] The data preprocessing module is used to acquire and preprocess user attribute information from complex network datasets, process relational datasets in the acquired datasets, and complete the construction of user network topology and generation of feature networks.

[0036] The network weighted vectorization processing module is used to transform the correlation between key user attributes into an attribute modularity matrix. Then, the topology matrix and the attribute modularity matrix are adaptively fused into a weighted modularity matrix. The weighted modularity matrix is ​​then decomposed into eigenvalues ​​and eigenvectors to obtain the weighted vectorized representation of the user network.

[0037] The Bayesian network model building module learns the prior probabilities of user node attributes based on the Bayesian network model. By identifying important user nodes and using relevant domain expert knowledge to obtain the prior probabilities of each attribute, it learns the attribute values ​​with significant influence and builds a Bayesian network model of user attributes and influence.

[0038] The community segmentation module calculates user influence based on a Bayesian network model of user attributes and influence. Then, based on the user influence calculation method, it obtains the tag importance and user importance required in the community segmentation process. Finally, it uses the parameters obtained above to perform community segmentation.

[0039] The beneficial effects of this invention are as follows: This invention provides an overlapping community detection method based on user attribute information in complex networks. It introduces user attribute information into traditional community detection methods, fusing network topology information and node attribute information into a weighted modularity matrix, and decomposing the matrix to obtain a weighted vectorized representation of the user network. User influence is obtained based on user attribute information. User importance and label importance are calculated based on prior user attributes and total user influence. Community detection is then performed on the user-weighted network through label propagation based on user importance and label importance. This paper incorporates user importance and label importance into the label propagation process, avoiding randomness in label propagation and obtaining stable and effective results. Experiments on real datasets show that the predicted output of this invention has a better fit with the actual community partitioning, and the predicted output has good stability without significant fluctuations. Therefore, this invention provides more reliable and advantageous prediction results for community detection in complex networks.

[0040] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0042] Figure 1 This is a flowchart illustrating the process of detecting and predicting overlapping communities in complex networks as implemented in this invention. Detailed Implementation

[0043] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0044] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0045] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0046] This invention provides a method for detecting overlapping communities based on user attribute information in complex networks, employing... Figure 1 The flowchart for community detection is shown below, and Table 1 shows the prediction results of the community detection model of this invention.

[0047] In one embodiment, such as Figure 1 As shown, this invention proposes an overlapping community detection method based on user attribute information in complex networks. Its key feature is that the community detection process utilizes the relationships between users in a social network and the short text attribute information carried by the users themselves, including the following steps:

[0048] S1. Preprocess the user attribute information in the acquired complex network dataset, including preprocessing such as removing non-text parts from short text documents of user attributes, word segmentation, and removal of stop words; process the relation dataset in the acquired dataset, including user relationship processing and removal of inactive users; and complete the construction of the user network topology and generate the feature network.

[0049] S2. Based on the feature network obtained in S1, feature processing is performed to capture key attribute information about the user from the local distribution around the user attributes. Then, the correlation between the key user attributes is transformed into the attribute modularity matrix. Next, the topology matrix and the attribute modularity matrix are adaptively fused into a weighted modularity matrix. Then, the weighted modularity matrix is ​​decomposed into the form of eigenvalues ​​and eigenvectors to obtain the weighted vectorized representation of the user network.

[0050] S3. Based on the key user attribute information obtained in S2, use a Bayesian network model to learn the prior probability of user node attributes. By identifying important user nodes, use relevant domain expert knowledge to obtain the prior probability of each attribute, learn the attribute values ​​with significant influence, establish a Bayesian network model of user attributes and influence, multiply the influence calculated by each attribute factor according to the Bayesian probability formula to obtain the final user influence, obtain the influence of all users in the network and sort them in ascending order.

[0051] S4. Based on the user influence information obtained in S3, calculate the network user importance using user prior attributes and total user influence, and then obtain the tag importance required in the community segmentation process based on the user influence calculation method.

[0052] S5. Based on the user tag importance and user importance calculation method obtained in S4, perform community discovery and obtain the community;

[0053] The preprocessing operations on the dataset in S1 include the following steps:

[0054] S11. Obtain the short text attribute information published by each user from the dataset. For these short text attribute information, delete non-text parts including HTML tags, non-English characters and punctuation marks, modal particles, metonyms, etc., and then use proprietary word segmentation technology on the LDA model corpus.

[0055] S12. Process the user relationships in the dataset, and then determine the activity of the users in the dataset. If a user has no connection with any other user, he / she is determined to be an inactive user and is removed.

[0056] The specific steps of S2 include:

[0057] S21. Random walks are used to obtain empirical statistics of user feature pairs for user attribute information. Then, an unbiased estimate of the joint probability of the observed features is given. From this, key user attribute information is extracted and normalized.

[0058] S22. Calculate the similarity of Pearson correlation coefficients between key user attribute information. The closer the absolute value of the Pearson correlation coefficient is to 1, the stronger the linear relationship. A value closer to +1 indicates a stronger positive correlation, and a value closer to -1 indicates a stronger negative correlation. Then, construct the obtained similarity into an attribute modularity matrix. Finally, use an adaptive fusion coefficient matrix to fuse the topology matrix and the attribute modularity matrix into a weighted modularity matrix.

[0059] S23. Obtain the weighted modularity matrix above, decompose the weighted modularity matrix into eigenvalues ​​and eigenvectors, and then map them into user vectors. Combine the user vectorization representation to obtain the weighted network matrix between users.

[0060] In S3, based on Bayes' probability formula, the multiplication principle is used to multiply the influence of each attribute to obtain the final user influence. The user influence calculation formula is:

[0061]

[0062] in, Indicates user influence. Indicates the influence of various user attributes

[0063] The specific steps in S4 include:

[0064] S41. Node normalization based on prior node attributes is insufficient; more importantly, there are close connections between users. Therefore, a user importance calculation method based on prior user attributes and user influence algorithms is proposed:

[0065]

[0066] in, On behalf of users The prior importance of Used to measure the relationship between adjacent users. The degree of impact; Indicates user neighborhood set, For users The degree;

[0067] S42. Based on user influence and user importance, the formula for calculating the importance of user tags in the network is as follows:

[0068]

[0069] Indicates to the user In the label Importance Indicates user Surrounding tags A set of.

[0070] S5 performs community discovery based on the user tag importance and user importance calculation method obtained in S4. Specific steps include:

[0071] S51. In complex networks, each user is initialized using a unique community label (identifier).

[0072] S52. Calculate the user importance and label importance of each user in the network, and initialize the iteration count to 1.

[0073] S53. Labels are propagated through the network in order of user importance. In each iteration of propagation, each user updates their label to the label of their maximum number of neighbors. When the number of labels on a user's neighboring nodes reaches the maximum value, the label with the highest user label importance is selected to update the user.

[0074] S54. If the maximum number of iterations is reached or each user's label is the same as the labels of most of their neighbors, then the users with the same label are placed in the same community, and the process ends. Otherwise, increment the iteration count by 1 and go to S53.

[0075] In one embodiment, the prediction performance of the current prediction method proposed in this invention is shown in Table 1:

[0076] Table 1. Prediction results of the method on various datasets.

[0077]

[0078] In one embodiment, the present invention also provides an overlapping community detection and prediction system based on user attribute information in complex networks, comprising:

[0079] The data preprocessing module is used to acquire and preprocess user attribute information from complex network datasets, process relational datasets in the acquired datasets, and complete the construction of user network topology and generation of feature networks.

[0080] The network weighted vectorization processing module is used to transform the correlation between key user attributes into an attribute modularity matrix. Then, the topology matrix and the attribute modularity matrix are adaptively fused into a weighted modularity matrix. The weighted modularity matrix is ​​then decomposed into eigenvalues ​​and eigenvectors to obtain the weighted vectorized representation of the user network.

[0081] The Bayesian network model building module learns the prior probabilities of user node attributes based on the Bayesian network model. By identifying important user nodes and using relevant domain expert knowledge to obtain the prior probabilities of each attribute, it learns the attribute values ​​with significant influence and builds a Bayesian network model of user attributes and influence.

[0082] The community segmentation module calculates user influence based on a Bayesian network model of user attributes and influence. Then, based on the user influence calculation method, it obtains the tag importance and user importance required in the community segmentation process. Finally, it uses the parameters obtained above to perform community segmentation.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for predicting overlapping communities based on node information and label propagation, characterized in that: The method includes the following steps: S1: Preprocess the user attribute information in the acquired complex network dataset, including preprocessing such as removing non-text parts from short text documents of user attributes, word segmentation, and removal of stop words; process the relation dataset in the acquired dataset, including user relationship processing and removal of inactive users; and complete the construction of the user network topology and generate the feature network. S2: Based on the feature network obtained in S1, feature processing is performed to capture key attribute information about the user from the overall distribution around the user attributes. Then, the correlation between the key user attributes is transformed into an attribute modularity matrix. Next, the topology matrix and the attribute modularity matrix are adaptively fused into a weighted modularity matrix. Then, the weighted modularity matrix is ​​decomposed into eigenvalues ​​and eigenvectors to obtain a weighted vectorized representation of the user network. S3: Based on the key user attribute information obtained in S2, use a Bayesian network model to learn the prior probability of user node attributes. By identifying important user nodes, use relevant domain expert knowledge to obtain the prior probability of each attribute, learn the attribute values ​​with significant influence, establish a Bayesian network model of user attribute-influence, multiply the influence calculated by each attribute factor according to the Bayesian probability formula to obtain the final user influence, obtain the influence of all users in the network and sort them in ascending order; S4: Based on the user influence information obtained in S3, calculate the importance of network users using user prior attributes and user influence, and obtain the importance of user tags in the network based on user influence and user importance. S5: Based on the user tag importance and user importance calculation method obtained in S4, perform community discovery and acquire the community; specific steps include: S51: In complex networks, each user is initialized using a unique community tag; S52: Calculate the user importance and user tag importance of each user in the network, and initialize the iteration count to 1; S53: Labels are propagated through the network in descending order of user importance. In each iteration of the propagation, each user updates their label to the label of their maximum number of neighbors. When the number of labels on a user's neighboring nodes reaches the maximum value, the label with the highest user label importance is selected to update the user's label. S54: If the number of iterations reaches the maximum number of iterations or each user's label is the same as the labels of most of its neighboring users, then the users with the same label are placed in the same community and the operation ends; otherwise, the number of iterations is incremented by 1 and the process goes to S53.

2. The overlapping community partitioning prediction method based on node information and label propagation according to claim 1, characterized in that: The preprocessing of the dataset in S1 includes the following steps: S11: Obtain the short text attribute information published by each user from the dataset. For this short text attribute information, remove non-text parts including HTML tags, non-English characters, punctuation marks, modal particles and metonyms. Use proprietary word segmentation technology on the LDA model corpus. S12: Process the user relationships in the dataset, and then determine the activity of the users in the dataset. If a user has no connection with any other user, he / she is determined to be an inactive user and is removed.

3. The overlapping community partitioning prediction method based on node information and label propagation according to claim 2, characterized in that: The specific steps of S2 include: S21: Random walk is used to obtain empirical statistics of user feature pairs for user attribute information. Then, an unbiased estimate of the joint probability of the observed features is given, from which key user attribute information is extracted and normalized. S22: Calculate the similarity of the Pearson correlation coefficient between key user attribute information. The closer the absolute value of the Pearson correlation coefficient is to 1, the stronger the linear relationship. When it is close to +1, the stronger the positive correlation is, and when it is close to -1, the stronger the negative correlation is. Then, construct the obtained similarity into an attribute modularity matrix, and then fuse the topology matrix and the attribute modularity matrix into a weighted modularity matrix through an adaptive fusion coefficient matrix. S23: Obtain the weighted modularity matrix above, decompose the weighted modularity matrix into eigenvalues ​​and eigenvectors, and then map them into user vectors. Combine the user vectorization representation to obtain the weighted network matrix between users.

4. The overlapping community partitioning prediction method based on node information and label propagation according to claim 3, characterized in that: In S3, based on the Bayesian probability formula, the influence of each attribute is multiplied using the multiplication principle to obtain the final user influence. The user influence calculation formula is as follows: in, Indicates user influence. This indicates the influence of each user attribute.

5. A system for predicting overlapping communities based on node information and label propagation, used to implement the prediction method according to any one of claims 1 to 4, characterized in that: The system includes: The data preprocessing module is used to acquire and preprocess user attribute information from complex network datasets, process relational datasets in the acquired datasets, and complete the construction of user network topology and generation of feature networks. The network weighted vectorization processing module is used to transform the correlation between key user attributes into an attribute modularity matrix, adaptively fuse the topology matrix and the attribute modularity matrix into a weighted modularity matrix, and then decompose the weighted modularity matrix into eigenvalues ​​and eigenvectors to obtain a weighted vectorized representation of the user network. The Bayesian network model building module learns the prior probabilities of user node attributes based on the Bayesian network model. By identifying important user nodes and using relevant domain expert knowledge to obtain the prior probabilities of each attribute, it learns the attribute values ​​with significant influence and builds a Bayesian network model of user attributes and influence. The community segmentation module calculates user influence based on a Bayesian network model of user attributes and influence. Then, based on the user influence calculation method, it obtains the user tag importance and user importance required in the community segmentation process. Finally, it uses the parameters obtained above to perform community segmentation.

Images