Community division method based on information granularity
Through the community division algorithm of information granularity principle, combined with user social relations and label themes, the shortcomings of K-Means clustering in high-dimensional sparse data are solved, efficient user clustering and clustering merging are achieved, and clustering is improved. The accuracy and effectiveness of the algorithm.
Patent Information
- Application Number
- CN202411007981.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-15
AI Technical Summary
The K-Means clustering algorithm has problems such as insufficient accuracy and sensitivity to initial values when processing high-dimensional sparse social network data, and its performance is poor on high-dimensional data, resulting in local minimum values and 'dimensionality disasters of clustering results. 'Phenomenon cannot fully reflect the user's initiative and personalization.
Introducing the information granularity principle, by generating initial equivalence relationship sets, dividing initial communities and merged categories, using information granularity community division algorithm, combining the comprehensive similarity of user social relations and tag topics, user clustering and aggregation are achieved Class merger reduces the impact of high-dimensionality and sparseness of data.
Improve the similarity of user interests and the tightness of social relations within the community, realize efficient user feature vector storage and calculation, reduce spatial complexity, solve the problems of high-dimensionality and sparseness, and improve the effectiveness of clustering algorithms. Sex and accuracy.
Smart Images

Figure CN118965133A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of social network data mining, and in particular to a community segmentation method based on information granularity. Background Art
[0002] Community division usually adopts cluster analysis. Clustering is one of the main techniques of data mining. It divides data objects into multiple clusters so that objects in the same cluster are more similar, while objects in different clusters are more different. Among clustering algorithms, K-Means clustering is a widely used algorithm with the advantages of simplicity, speed, scalability and high efficiency. However, K-Means clustering also has some defects. One of them is that the number of clusters needs to be set in advance, but it is often not known in advance how many clusters should be divided for a given data set, resulting in the lack of accuracy of the algorithm. Another defect of K-Means clustering is that it is relatively sensitive to the initial value. Once the initial value is not selected well, the clustering result is easy to fall into the local minimum.
[0003] In addition, as the size of social network data sets continues to expand, the increase in data dimensions makes the data more sparse, and the distance gap between data gradually narrows. Many algorithms can achieve good clustering results on low-dimensional data, but their performance in processing high-dimensional data is unsatisfactory, which will cause the "dimensionality disaster" phenomenon. At the same time, due to the increase in noise features and redundant features, the effectiveness of clustering algorithms is greatly reduced.
[0004] Information granularity computing is a new concept and new method in the field of artificial intelligence research. It covers all theories, methods and technologies related to granularity and is an effective tool for solving complex problems, mining massive data and processing uncertain information. With the development of social networks, a large amount of complex and massive high-dimensional data has been generated, which requires people to improve their ability to obtain effective knowledge from high-dimensional data. The current clustering algorithm for high-dimensional sparse data has a certain one-sided description of things, which cannot fully reflect the development dynamics of things, and cannot better reflect the initiative and personalization of users. Therefore, it is a practical method to introduce the principle of information granularity into the clustering research of high-dimensional data. Summary of the invention
[0005] Aiming at the shortcomings of K-Means clustering algorithm and the defects of high dimensionality and sparsity of data, the principle of information granularity is applied to user clustering analysis. The membership degree of user equivalence relation and generalized equivalence relation are given. On this basis, a community partitioning algorithm based on information granularity is proposed.
[0006] A community division method based on information granularity includes the following steps:
[0007] Step S1: Generate an initial equivalence relation set;
[0008] Step S2: Divide the initial community;
[0009] Step S3: Merge categories.
[0010] Preferably, in step S1 of the present invention, an initial equivalence relation set is generated, and the specific process is as follows: let D be the complete set of users, denoted as D = {u i}, where i = 1, 2...n; user u i The initial equivalence relation is R i ={{[u i ]},{D-[u i ]}}, where [u i ]={u j Sim(u i ,u j )≥β1},i,j=1,2...,n,Sim(u1,u j ) is user u i and u j Similarity model, β1 is the similarity threshold.
[0011] Preferably, the initial community is divided in step S2 of the present invention, and the specific process is: further processing the initial equivalence set to generate a generalized equivalence relationship set to achieve community division; assuming R i and R j are two initial equivalence relations, user u i The generalized equivalence relation Ri′={{[ui]},{D-[ui]}}, where [u i ]=∪{u j μ(R i ,R j )≥β2},i,j=1,2...,n, β2 is the membership threshold of the equivalence relation.
[0012] Preferably, the merging of classes in step S3 of the present invention is specifically carried out as follows: each initial class is regarded as a large granularity, and each granularity is merged according to the information granularity division method. First, the similarity between each class and other classes is calculated, and then the two classes with the largest similarity are merged; this process is repeated until the similarity is less than a certain threshold; let C i and C j For two initial classes, Class similarity Sim(C i ,C j ) is calculated as shown in formula (1).
[0013]
[0014] in, After calculating the similarity between classes, we can start merging the two classes with the greatest similarity until the similarity is less than a certain threshold.
[0015] The advantages of the proposed community division method based on information granularity are reflected in two aspects: on the one hand, the comprehensive similarity of integrating user social relations and tag topics realizes a comprehensive evaluation mechanism of user individual and social attributes, so that users in the community not only have high interest similarity, but also have high social relationship closeness; on the other hand, by using the information granularity method for user clustering and cluster merging, the interval can be set so that it can achieve self-optimization with a certain step size.
[0016] Community division is based on the principle of information granularity. Since the feature vector has a high dimension and the data is highly sparse, an efficient method for storing and calculating user feature vectors is designed, which reduces the spatial complexity of data storage. It is superior to the traditional K-Means clustering algorithm and effectively solves the problems caused by the high dimensionality and sparsity of data. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flowchart of community division of information granularity of the present invention.
[0018] Figure 2 This is a schematic diagram of the clustering results of community division. DETAILED DESCRIPTION
[0019] like Figure 1 As shown, a community division method based on information granularity includes the following steps:
[0020] Step S1: Generate an initial equivalence relation set;
[0021] The specific process is: Let D be the complete set of users, denoted as D = {u i}, where i = 1, 2...n; user u i The initial equivalence relation is R i ={{[u i ]},{D-[u i ]}}, where [u i ]={u j Sim(u i ,u j )≥β1},i,j=1,2...,n,Sim(u1,u j ) is user u i and u j Similarity model, β1 is the similarity threshold.
[0022] Step S2: Divide the initial community. The specific process is: further process the initial equivalence set to generate a generalized equivalence relationship set to achieve community division;
[0023] Assume R i and R j are two initial equivalence relations, user u i The generalized equivalence relation R i ′={{[u i ]},{D-[u i ]}}, where [u i ]=∪{u j μ(R i ,R j )≥β2},i,j=1,2...,n, β2 is the membership threshold of the equivalence relation.
[0024] Step S3: Merge classes. The specific process is: take each initial class as a large granularity and merge each granularity according to the information granularity division method. First, calculate the similarity between each class and other classes, and then merge the two classes with the largest similarity; repeat this process until the similarity is less than a certain threshold; let C i and C j For two initial classes, Class similarity Sim(C i ,C j ) is calculated as shown in formula (1).
[0025]
[0026] in, After calculating the similarity between classes, we can start merging the two classes with the greatest similarity until the similarity is less than a certain threshold.
[0027] Example 1
[0028] Step S1 generates an initial set of equivalence relations
[0029] Assume that there are 5 users u1, u2, ...u5, and their similarity values are shown in Table 1. j The comprehensive similarity of (j=1,2…5) is Sim(u1,u j )={1,0.4,0.5,07,0.3}. Assume β1=0.5, then the initial equivalence relation of user u1 is R1={{u1,u3,u4},{u2,u5}}, and the initial equivalence relations of u2,u3,u4,u5 are R2,R3,R4,R5 respectively, that is, the initial equivalence relation set R *Includes the following subsets: R1 = {{u1,u3,u4},{u2,u5}}, R2 = {{u2,u3,u5},{u1,u4}}, R3 = {{u1,u2,u3,u4},{u5}}, R4 = {{u1,u3,u4},{u2,u5}}, R5 = {{u2,u5},{u1,u3,u4}}.
[0030] Table 1 Examples of comprehensive user similarity
[0031] <![CDATA[Sim(u i ,in j )]]> <![CDATA[u1]]> <![CDATA[u2]]> <![CDATA[u3]]> <![CDATA[u4]]> <![CDATA[u5]]> <![CDATA[u1]]> 1 0.4 0.5 0.7 0.3 <![CDATA[u2]]> 0.4 1 0.6 0.4 0.5 <![CDATA[u3]]> 0.5 0.6 1 0.6 0.3 <![CDATA[u4]]> 0.7 0.4 0.6 1 0.2 <![CDATA[u5]]> 0.3 0.5 0.3 0.2 1
[0032] Step S2: Divide the initial community
[0033] The initial equivalence set is further processed to generate a generalized equivalence relation set R' to achieve community division.
[0034] Assume R i and R j are two initial equivalence relations, user u i The generalized equivalence relation R i ′={{[u i ]},{D-[u i ]}}, where [u i ]=∪{u j |μ(R i ,R j )≥β2},i,j=1,2...,n, β2 is the membership threshold of the equivalence relation.
[0035] Assuming that the initial equivalence relation set of 5 users has been obtained, the corresponding generalized equivalence relation set R' is calculated according to Definition 2. First, calculate μ(R i ,R j ), and the calculation results are shown in Table 2.
[0036] Table 2μ(R i ,R j )
[0037] μ <![CDATA[R1]]> <![CDATA[R2]]> <![CDATA[R3]]> <![CDATA[R4]]> <![CDATA[R5]]> <![CDATA[R1]]> 1 1 / 5 3 / 4 1 0 <![CDATA[R2]]> 1 / 5 1 2 / 5 1 / 5 2 / 3 <![CDATA[R3]]> 3 / 4 2 / 5 1 3 / 4 1 / 5 <![CDATA[R4]]> 1 1 / 5 3 / 4 1 0 <![CDATA[R5]]> 0 2 / 3 1 / 5 0 1
[0038] Assuming β2 = 0.6, the obtained generalized equivalent relation set R' includes the following subsets: R1' = {{u1,u3,u4},{u2,u5}}, R2' = {{u2,u5},{u1,u3,u4}}, R3' = {{u1,u3,u4},{u2,u5}}, R4' = {{u1,u3,u4},{u2,u5}}, R5' = {{u2,u5},{u1,u3,u4}}. Based on the above results, the initial classification {{u1,u3,u4},{u2,u5}} can be obtained.
[0039] Step S3: Merge categories
[0040] After the initial community classification is generated, similar users may be classified into different classes. To avoid this incorrect classification result, it is necessary to further merge the classes to obtain the optimized community classification result. Take each initial class as a large granularity and merge each granularity according to the information granularity classification method. First, calculate the similarity between each class and other classes, and then merge the two classes with the largest similarity; repeat this process until the similarity is less than a certain threshold. Let C i and C j For two initial classes, Class similarity Sim(C i ,C j ) is calculated as shown in formula (1).
[0041]
[0042] in, After calculating the similarity between classes, we can start merging the two classes with the greatest similarity until the similarity is less than a certain threshold. In the above example, the initial classification is {{u1,u3,u4},{u2,u5}}, that is, C1 = {u1,u3,u4}, C2 = {u2,u5}, then Sim(C1,C2) = 0.
[0043] The selection of similarity and similarity threshold of information granularity equivalence relation is as follows: Let R be the equivalence relation, X be the domain of the problem, and for If satisfied Then R1 is said to be finer than R2, and is recorded as R1≤R2. The above explanation is proposed from the perspective of the coarseness of the attribute granularity. Generally speaking, the smaller the attribute granularity, the fewer attributes it contains, and the coarser the equivalence relationship it corresponds to. The division of the attribute set based on this equivalence relationship is also coarser, so the attribute granularity is also coarser, and vice versa. Therefore, for the definition of R1 being finer than R2, it can also be said that R2 is coarser than R1. The clustering result analysis adopts the principle of information granularity. If the granularity value is larger, the number of clusters is smaller, and the clustering result is coarser; conversely, the smaller the granularity value, the more clusters are more, and the clustering result is more detailed. Therefore, clustering using information granularity can divide objects of different granularities according to the specific requirements of the problem.
[0044] Experiment on community division based on information granularity
[0045] In order to evaluate the effect of the algorithm on dynamic community mining, the experiment needs to be compared under the same data set. The collected and preprocessed Weibo data is used as the experimental data set, and the following four algorithms are compared: i ,u j) is denoted as Tt-K-means; K-means based on comprehensive user similarity is denoted as CP-K-means; K-means based on Tt_Sim(u i ,u j ) is divided into information granularity communities, denoted as TtIG; the information granularity community division based on comprehensive user similarity is denoted as CPIG.
[0046] 1) Evaluation indicators
[0047] This paper uses the clustering evaluation criterion as the evaluation index. The calculation methods of I index and Dunn index are as follows:
[0048] (1) I index
[0049] The I index measures the separation between classes by the maximum value of the distance between the current class and the class center among all classes, and measures the compactness within the class by the sum of the distances between each point in the class and the class center. The effect of the number of clusters on the index can be eliminated by introducing the 1 / m coefficient. The I index is used to balance the separation between classes and the compactness within classes. The core idea is to find a maximum value to optimize the clustering result. Assuming the number of clusters is m, the index is defined as formula (2).
[0050]
[0051] (2) Dunn indicator
[0052] The Dunn index is an internal evaluation mechanism. The evaluation result depends only on the clustering data itself. Its purpose is to identify whether the clusters are compact (i.e., whether the variance in the same cluster is small enough) and whether they are properly separated (whether the means of different clusters are relatively far apart). For a given assigned cluster, the higher the Dunn index, the better the clustering result. Assume that the number of clusters is m, and the index is defined as formula (3).
[0053]
[0054] Where δ(c i ,c j ) represents the distance measure between clusters, and Δk represents the distance within a cluster. The specific definitions are as shown in equations (4) and (5).
[0055] δ(c i ,c j )=min(dist(x i ,x j )),x i ∈c i ∩x j ∈c j (4)
[0056] Δk=max(dist(xi ,x j )),x i ∈c τ ∩x j ∈c τ (5)
[0057] 2) Experimental steps and results analysis
[0058] (1) TtIG algorithm: Perform community division based on information granularity based on label topic similarity. First, determine the parameters: ① Adjust the parameter μ. Since the average user label in the data set is 2.3, μ = 0.1 is obtained according to formula (5). ② Adjust the parameters β1 and β2 to adjust the granularity of the class division. Due to the characteristics of clustering algorithms based on information granularity, it can be seen that a major difficulty in adjustment is the setting of β1 and β2. β1 that is too small or too large is likely to cause clustering deviation. Set β1 = 0.5, β2∈[0.2,0.8], and the step size is 0.1 to divide the community under different parameter combinations. In order to better describe the theme of each user community, the hot user labels in the class are extracted based on the clustering results, and the top 10 labels are used as representatives of the theme to describe the user community. Table 3 shows the results of the top five categories from the most to the least number of users in the divided community.
[0059] Table 3 Popular user tags based on tag topic community division
[0060]
[0061] As shown in Table 3, after testing, it is found that when β1=0.5 and β2=0.6, better results are obtained. This also verifies that in Weibo, due to the lack of user tags, when dividing communities based on content analysis, user tags do not contribute more to community mining than user Weibo content. Therefore, the results of community division are consistent with the results of the LDA topic model, but the probability distribution of topics in the results is not proportional to the number of community users, which is consistent with the actual situation, that is, communities with high topic probabilities may not necessarily have many people. In addition, from the results, the overlap rate of Top-n popular user tags and keywords in the topic model is not high. The reason for this is that the tags of Weibo users are not annotations of resources on the website (such as Weibo texts), but a description of personal interests and preferences. Therefore, they generally reflect the categories of user interests and the embodiment of personality, which is different from the keywords extracted from Weibo.
[0062] (2) CPIG algorithm: Perform information granularity community division based on comprehensive similarity. According to the experimental data set, the social relationship in this paper is represented by the forwarding interaction relationship. Therefore, the parameter γ = 0, μ is still 0.1, α, β are calculated by formula (5) to obtain α = 0.6, β = 0.4, and continue to adjust the parameters β1, β2 to adjust the granularity of the class division.
[0063] Select K-means, combined with label topic similarity Tt_Sim(u i ,u j ) and comprehensive similarity Sim(u i ,u j ), and compared with the TtIG and CPIG proposed in this paper. Taking different combinations of β1 and β2 with different particle sizes, the I index is calculated, and the results are shown in Table 4. The Dunn index is calculated, and the results are shown in Table 5.
[0064] Table 4 I index of four clustering algorithms
[0065]
[0066] Table 5 DUNN indicators of four clustering algorithms
[0067]
[0068] As shown in Tables 4 and 5, for the I index and Dunn index, under the same perspective, the algorithm in this paper is generally better than the K-means algorithm; and the K-means based on comprehensive similarity is better than the K-means based on label topics. However, the comparison between the information granularity division based on label topics and the K-means based on comprehensive similarity does not show a rule of superiority. Under the parameter settings of β1=0.5 and β2=0.6, the community division method based on comprehensive similarity is better than the community division method based on label topic similarity, and the CPIG algorithm also achieves the best division effect.
[0069] Visualize the community division results. Because the experimental data set involves a large number of character nodes, it is impossible to clearly display all the character nodes and their relationship lines on a graph, so some character nodes are randomly selected as experimental data. The clustering results of community division are shown in Figure 2. Figure 2 shown.
[0070] Through experiments, we found that the best result is obtained by extracting 1 / 10 of the character nodes in the original data set for calculation and display. The nodes with the same light and dark colors in the figure constitute subgroups, which are the initial community divisions before merging. Figure 2 The four larger person nodes represent the nodes with the largest intermediate centrality, that is, the core person of the subgroup, which means that he is the person node with the most forwarding and @ other people in this subgroup, or the node with more common interests with other nodes in the subgroup.
Claims
1. A community division method based on information granularity, characterized by The steps include: Step S1: Generate an initial equivalence relation set; Step S2: Divide the initial community; Step S3: Merge categories.
2. The community division method based on information granularity according to claim 1 is characterized in that In the above step S1, the initial equivalence relation set is generated, and the specific process is as follows: Let D be the complete set of users, denoted as D = {u i }, where i = 1, 2...n; user u i The initial equivalence relation is R i ={{[u i ]},{D-[u i ]}}, where [u i ]={u j Sim(u i ,u j )≥β1},i,j=1,2...,n,Sim(u1,u j ) is user u i and u j Similarity model, β1 is the similarity threshold.
3. The community division method based on information granularity according to claim 2 is characterized in that The specific process of dividing the initial community in the above step S2 is as follows: further processing the initial equivalence set to generate a generalized equivalence relationship set to achieve community division; Assume R i and R j are two initial equivalence relations, user u i The generalized equivalence relation R i ′={{[u i ]},{D-[u i ]}}, where [u i ]=∪{u j μ(R i ,R j )≥β2},i,j=1,2...,n, β2 is the membership threshold of the equivalence relation.
4. The community division method based on information granularity according to claim 3 is characterized in that The specific process of merging classes in step S3 is as follows: take each initial class as a large granularity and merge each granularity according to the information granularity division method; first calculate the similarity between each class and other classes, and then merge the two classes with the largest similarity; repeat this process until the similarity is less than a certain threshold; let C i and C j For two initial classes, Class similarity Sim(C i ,C j ) is calculated as in formula (1); in, After calculating the similarity between classes, we can start merging the two classes with the greatest similarity until the similarity is less than a certain threshold.
Citation Information
Patent Citations
Method and system for dividing user community structure of microblog network
CN112712116A
