Implementation method of multi-group member intimacy-thinness relationship based on set aggregation k-means method

Abstract

The invention discloses a method for realizing a multi-group member closeness-thinness relationship based on a set aggregation k-means method, and the method comprises the steps: carrying out the collection and summarization of a k-means method aggregation result through the distance between group members in a large time scale range, thereby achieving the quantitative description of the group member closeness-thinness and discretization relationship, and carrying out the visual display. The affiliation of the members is further defined and purified, and the members are evolved towards a better direction.

Description

Technical Field

[0001] The invention belongs to the technical field of cluster analysis, and in particular relates to a method for realizing the affinity relationship among members of multiple groups based on a set aggregation k-means method. Background Art

[0002] There is a type of problem that needs to be solved at work: it is necessary to define the closeness and distance between members of a large number of groups. These groups cannot participate in the group and only know which members are in the group.

[0003] Generally speaking, the closeness of group members usually refers to the degree of intimacy and distance between individuals in the group. "Determining the closeness of group members" can be achieved through a variety of methods and tools. The main methods are as follows: one is questionnaire survey, two is social network analysis, three is observation method, four is interview, five is team activities, six is ​​sentiment analysis, seven is communication pattern analysis, and eight is cluster analysis. Due to the sense of distance from the target group, one, three, four, and five cannot be carried out. What can be carried out are social network analysis, sentiment analysis, communication pattern analysis, and cluster analysis, which all belong to the category of data analysis.

[0004] Social Network Analysis (SNA): Use data analysis tools to construct and analyze social network graphs, which can help identify central figures, marginal figures, and subgroups within a group.

[0005] Sentiment analysis: Analyzing the content of communications between group members, such as emails, social media posts, or meeting minutes, to identify emotional tendencies and infer the closeness between members.

[0006] Communication pattern analysis: Analyzing the frequency and pattern of communication between group members, such as who communicates with whom frequently, the duration and content of communication, can reveal the close relationships between members.

[0007] Cluster analysis: Use clustering algorithms to identify natural groupings within a population, which may be based on similarities or interaction patterns among members.

[0008] The main disadvantages of Social Network Analysis (SNA) are: it requires a large amount of data support; it involves sensitive information; social networks change and the analysis model needs to be constantly updated. The reasons for the above disadvantages are as follows: the difficulty of data collection, data privacy and ethical issues, and the dynamic nature of social networks.

[0009] The main disadvantages of sentiment analysis are: difficulty understanding complex contexts, which may lead to misjudgment; privacy leakage and ethical issues; and difficulty quantifying the intensity of sentiment. The reasons for the above shortcomings are as follows: context and cultural differences; privacy and ethical issues; and the identification of sentiment intensity and subtle emotions.

[0010] The main disadvantages of communication pattern analysis are: privacy issues; it is challenging to obtain comprehensive communication data. The reasons for the above disadvantages are as follows: privacy and security issues; collecting comprehensive communication data may need to span multiple platforms and systems, and integrating data from different sources requires format, compatibility and synchronization issues.

[0011] The main disadvantages of cluster analysis are: some clustering algorithms (such as k-means) require the number of clusters to be specified in advance; clustering results are highly dependent on the quality of the data and feature selection; for large-scale data sets, a lot of computing resources and time may be required. The reasons for the above disadvantages are as follows: the optimal value of the cluster is usually not obvious and may require trial and error; Cartesian products may consume a lot of computing resources and time. Summary of the invention

[0012] In order to overcome the shortcomings of the prior art, the present invention provides a method for realizing the closeness and distance relationship of multiple group members based on the set aggregation k-means method. The present invention collects and summarizes the results of k-means aggregation within a large time scale according to the distance between group members, so as to quantitatively characterize the closeness and distance and discrete relationship of group members, and perform visual display, further define and purify the belonging of members, and evolve towards a better direction.

[0013] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0014] Step 1: Starting from the aggregation relationship table A, first associate it with the group B to which the number belongs to obtain table C;

[0015] The fields in the aggregation relationship table A include: the distance information of the set S when the numbers in the same cluster take the same value of feature F; Table C only has the group name in addition to the aggregation relationship table A;

[0016] Step 2: From Table C, label the day, cluster, and group with the number cluster_id2 to obtain Table D;

[0017] Step 3: Find the number of different number pairs of cluster_id2 in Table D, and sum the number of times the same number pair comes together in a certain period of time. The result is n_cnt, and Table E is obtained;

[0018] Step 4: Find the number of cluster_id2 different number pairs in Table D that have at least one occurrence, and sum the number of the same number pair in a certain period of time. The result is u_cnt, and Table F is obtained;

[0019] Step 5: Table E and Table F are associated by number pairs, and the distance is calculated using the formula: distance = 1-n_cnt / u_cnt, to obtain Table G;

[0020] Step 6: Table A1 with the position of the numbers in the group is associated with G according to a1.hm=g.id1, ​​and the information of all A1 is obtained at the same time. If G can be associated, the information of G is obtained to obtain the final result table H; a1.hm represents the number column of table A1, and g.id1 represents the number 1 column of table G.

[0021] A computer program enables a computer to execute the above-mentioned method for realizing the affinity relationship.

[0022] An electronic device comprises: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned method for realizing the affinity relationship.

[0023] A computer-readable storage medium stores a computer program, which implements the above-mentioned method for realizing the affinity relationship when executed by a processor.

[0024] A chip includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the above-mentioned method for realizing the affinity relationship.

[0025] A computer program product, comprising a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and the method for realizing the above-mentioned affinity relationship is implemented when the instructions are executed by the at least one processor.

[0026] The beneficial effects of the present invention are as follows:

[0027] Compared with the above-mentioned social network analysis, sentiment analysis and communication pattern analysis, the present invention effectively overcomes their shortcomings and has some unique advantages:

[0028] First, the data source is relatively simple: only the location information of members is needed, which can improve the efficiency of model creation;

[0029] Second, privacy and security are protected: because the data source is relatively single, privacy and security are objectively protected to the maximum extent;

[0030] Third, the use of unique mathematical methods has achieved the effect of "a little effort to achieve a great result": 1). Because of the preparation of "k-means method for clustering", there are more theoretical and practical preparations and ecosystems, which have overcome many shortcomings and turned disadvantages into advantages, so that we can have the current development; 2). Although the data source is single and has its disadvantages, we can ignore the accidental gathering of group members at a certain time and event, and grasp the inevitability of the closeness and distance of group members through statistics within a large time scale; 3). The combination of the relationship between two members is used to characterize the relationship between the entire group members. If a group has n members, there will be at most n (n-1) closeness and distance measures between two members in order. For example, in the example above, "group 8" has 8 numbers, and there will be at most 8*7 (56) two-to-two relationships, but there are actually only 4 two-to-two relationships and 5 isolated relationships. Finally, such relationships can be visualized. In short, some simple and effective mathematical methods have been used to overcome "many technical difficulties and dilemmas" and achieve better results.

[0031] The beneficial effects brought by the technical solution of the present invention are as follows: within a large time scale, the results of k-means aggregation are aggregated and summarized by the distance between group members, so as to quantitatively characterize the closeness and discreteness of group members, and perform visual display, further define and purify the affiliation of members, and evolve towards a better direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a closeness relationship diagram of the numbers of "group 1" in the embodiment of the present invention;

[0033] Figure 2 This is a closeness relationship diagram of the numbers of "group 8" in the embodiment of the present invention;

[0034] Figure 3 This is a flow chart of the implementation of the k-means method in an embodiment of the present invention;

[0035] Figure 4 This is a closeness relationship diagram of the numbers of "group 8" in the embodiment of the present invention;

[0036] Figure 5 This is a flowchart for implementing the “multi-group member affinity relationship” of an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0038] Each group (Group, represented by G) has several entities (Entity, represented by E), and the entity has a set (Set, represented by S) under certain features (Features, represented by F). The problem to be solved by the present invention is: in a large time scale, for each group G, which entities E often have similar sets under the same value of certain features F, and have close relationships, otherwise, they have distant relationships. This technology belongs to the field of data mining.

[0039] The following two examples can both measure the closeness of group members. The members here are specific individuals, and these individuals all have location information. The data is public data obtained by crawling the Internet, and the mobile phone number information has been encrypted.

[0040] Embodiment 1:

[0041] Do members of different groups gather together frequently?

[0042] The data taken here is between 20240810-20240815. The columns are: start date (ymd_start), end date (ymd_end), number (id1), cluster number (id2), group name (g_name), number of times the number and cluster number appear together (n_cnt), number of times the number or cluster number appears (u_cnt), and distance (distance).

[0043] Where: distance = 1-n_cnt / u_cnt

[0044] Note: "Number of times a number and an aggregate number appear together (n_cnt)" refers to the number of times a number and an aggregate number are aggregated; "Number of times a number or an aggregate number appears (u_cnt)" refers to the total number of times a number and an aggregate number are aggregated, but the aggregation of a number and an aggregate number is only counted once.

[0045] The entity here is "person", the feature is geohash, and the set is the time interval under this geohash. If two members of a group often have similar time intervals under the same geohash within a certain period of time, that is, they often gather together, they have a close relationship, otherwise, they have a distant relationship.

[0046] Table 1 Output results of Example 1

[0047]

[0048]

[0049] Group 1: There are three numbers. The three numbers have no "cluster number", so they are separated from each other; the schematic diagram is as follows Figure 1 ;

[0050] Group 8: Number 85 and number 86 are relatively close; number 86 and number 87 are second closest; number 81, number 82, number 83, number 84, number 88 are isolated. Figure 2 :

[0051] Embodiment 2:

[0052] Do members of different groups often travel together?

[0053] The data taken here is between 20240810-20240815. The columns are: start date (ymd_start), end date (ymd_end), number (id1), peer number (id2), group name (g_name), number of times the number and peer number appear together (n_cnt), number of times the number or peer number appears (u_cnt), and distance (distance).

[0054] Where: distance = 1-n_cnt / u_cnt;

[0055] The entity here is "person", the feature is a certain day, and the set is all the "time periods + geohash" experienced on a certain day. If within a certain period of time, two members of the group have similar trajectories on more days and experience similar time intervals on more of the same geohash, that is, they are traveling together, they have a close relationship, otherwise, they have a distant relationship.

[0056] The above mentioned clustering or peering is realized on the basis of “set clustering k-means”. The implementation flow chart of set clustering k-means is as follows: Figure 3 As shown;

[0057] The "result table" A above stores information such as the distance of the numbers in the same cluster (the cluster here is broad, including the above cluster or the same line) in the set S when the feature F takes the same value.

[0058] Take aggregation as an example, as shown in Table 2-1:

[0059] Table 2-1 Table A

[0060]

[0061] The output results are shown in Table 2-2:

[0062] Table 2-2 Output results of Table A

[0063]

[0064] Numbers 1 to 3 belong to the first group, "number 1" is the centroid (the center of this group), the durations of numbers 1 to 3 are 5.68, 5.85, and 7.73 respectively, and numbers 2 to 3 are very similar to the time intervals of "number 1" under the same geohsh1, with distances of 0, 0.199809, and 0.264906 respectively. The situation of the second group is similar.

[0065] Geohash: On the earth, people usually use a pair of longitude and latitude coordinates to represent a point. It can be understood that geohash is an algorithmic idea, which is to represent the longitude and latitude coordinates with a string of digits and letters (digits 0-9, 26 letters minus a, i, l, o, a total of 32 digits and letters). The more digits the geohash has, the smaller the range it represents, and the fewer digits it has, the larger the range it represents.

[0066] The implementation steps of the "multi-group member affinity relationship" of the present invention are described by taking "group 8" of the above "Example 1" as an example:

[0067] Precondition: Table B: HM (number), group_name (group name), this is the original table to determine the closeness relationship;

[0068] After table B is processed by the "set aggregation k-means method", two tables will be output:

[0069] The first is Table A1, including: ymd_start (start date), ymd_end (end date), HM (number), group_name (group name). This is a table with positional relationship based on B, as shown in "Table 3-1 Table A1", and the data is shown in "Table 3-2 Table A1 output result". The numbers in A1 include two categories: non-clustered numbers, that is, isolated numbers; and clustered numbers, which means that these numbers may be clustered or isolated at times.

[0070] Table 3-1 Table A1

[0071] Fields Field Type meaning ymd_start character start date ymd_end character End Date HM character Number GROUP_NAME character Group Name

[0072] The data is as follows:

[0073] Table 3-2 Output results of Table A1

[0074] ymd_start ymd_end hm group_name Start Date Termination Date Number Group Name 20240810 20240815 Number 81 Group 8 20240810 20240815 Number 82 Group 8 20240810 20240815 Number 83 Group 8 20240810 20240815 Number 84 Group 8 20240810 20240815 Number 85 Group 8 20240810 20240815 Number 86 Group 8 20240810 20240815 Number 87 Group 8 20240810 20240815 Number 88 Group 8

[0075] All the numbers that have positions in “Group 8” are these 8 numbers.

[0076] The second is Table A, which contains numbers with clustering relationships, as shown in "Table 2-1 Table A". A is a clustered number and does not show isolated numbers. Therefore, the distance of number pairs does not count the number of isolated numbers in the number pair.

[0077] Here we need to pay attention to the following: because there are many numbers in the group, some of which have no location information, so it is impossible to determine the closeness of these numbers to other numbers. On the contrary, as long as there is location information, it can be accurately determined. Therefore, the following "Step 6" uses A1 as the basic table. Figure 5 shown.

[0078] Step 1: Take table A as the starting point. First associate it with "number group B" to get table C. C just has one more "group name" than A.

[0079] Table 4-1 Table C

[0080]

[0081] The data is as follows:

[0082] Table 4-2 Output results of Table C

[0083]

[0084] "Group 8" had three gatherings between 20240810 and 20240815: the first was the gathering of "Number 86" and "Number 87" with "Gathering Number" of 339 in 20240814, which lasted for about 1.8 hours at "Position 1"; the second was the gathering of "Number 85" and "Number 86" with "Gathering Number" of 42 in 20240815, which lasted for about 0.38 hours at "Position 2"; the third was the gathering of "Number 85" and "Number 86" with "Gathering Number" of 43 in 20240815, which lasted for about 0.02 hours at "Position 3".

[0085] Step 2: From Table C, label CLUSTER_ID2 for a certain day (ymd), a certain cluster (cluster_id), and a certain group (group_name), and obtain Table D:

[0086] Table 5-1 Table D

[0087]

[0088] CLUSTER_ID2 (cluster ID) is unique within a certain period of time. It uniquely identifies a group of a certain period of time or a certain cluster. The "cluster ID" corresponds to multiple numbers of the cluster.

[0089] The data is as follows:

[0090] Table 5-2 Output results of Table D

[0091]

[0092] CLUSTER_ID2 is 573, which corresponds to numbers 86 and 87 in "position one"; CLUSTER_ID2 is 690, which corresponds to numbers 85 and 86 in "position two"; CLUSTER_ID2 is 690, which corresponds to numbers 85 and 86 in "position three".

[0093] Table D has only one more column than table C: CLUSTER_ID2 (cluster ID within the group). The purpose of this column is to renumber the data between 20240810 and 20240815 according to ymd+cluster_id+group_name to ensure that they are unique within this period. Here they are 573, 690, and 691 respectively.

[0094] Step 3: Find the number of different number pairs in cluster_id2 in D, and sum the number of times the same number pair comes together in a certain period of time. Get Table E:

[0095] Table 6-1 Table E

[0096] Fields Field Type meaning ID1 character Number 1 ID2 character Number 2 N_CNT number Number of gatherings

[0097] id1 and id2 are a pair of numbers belonging to group_name.

[0098] The data is as follows:

[0099] Table 6-2 Output results of Table E

[0100]

[0101] Table E is obtained by counting the number of times based on Table D.

[0102] It should be noted here that the number 85 and number 86 "appear together" a certain number of times is symmetrical, so they appear in two rows, and the same is true for number 86 and number 87.

[0103] Step 4: Find the number of different number pairs in cluster_id2 in which at least one number pair appears in D (number pairs appear in a cluster at the same time or one of them appears in a cluster as one of another number pair), and sum the number of times the same number pair appears in a certain period of time. Get Table F:

[0104] Table 7-1 Table F

[0105] Fields Field Type meaning ID1 character Number 1 ID2 character Number 2 U_CNT number Occurrence

[0106] The data is as follows:

[0107] Table 7-2 Table F output results

[0108] ID1 ID2 U_CNT Number 1 Number 2 Number of times number 1 or number 2 appears Number 85 Number 86 3 Number 86 Number 85 3 Number 86 Number 87 3 Number 87 Number 86 3

[0109] The "output results of Table F in Table 7-2" are also obtained by counting the number of times based on the "output results of Table E in Table 6-2". Similarly, numbers 85 and 86 are symmetrical with numbers 86 and 87.

[0110] The reason why the two groups of number pairs appear 3 times here is that number 86 appears with number 85 twice and with number 87 once, for a total of 3 times, and these 3 times cover numbers 85 and 87, that is, numbers 85 and 87 only appear in the cluster with number 86, and there is no other cluster.

[0111] Step 5: Join Table E and Table F by id1 and id2 to get Table G:

[0112] Table 8-1 Table G

[0113] Fields Field Type meaning ID1 character Number 1 ID2 character Number 2 N_CNT number Number of gatherings U_CNT number Occurrence Distance number distance

[0114] The distance is calculated like this:

[0115] distance=1-n_cnt / u_cnt

[0116] This is used to measure the closeness between number pairs. distance<1.

[0117] The data is as follows:

[0118] Table 8-2 Table G output results

[0119] ID1 ID2 N_CNT U_CNT DISTANCE Number 1 Number 2 The number of times number 1 and number 2 appear together Number of times number 1 or number 2 appears distance Number 85 Number 86 2 3 0.33 Number 86 Number 85 2 3 0.33 Number 86 Number 87 1 3 0.67 Number 87 Number 86 1 3 0.67

[0120] Step 6: The table A1 with the position of the numbers in the group is associated with G to obtain the final result table H:

[0121] Because there must be at least two members in each group to discuss closeness, there must be at least two hms in each group_name. At the same time, for the sake of simplicity and generality, each hm can only belong to one group_name.

[0122] A1 and G are associated by a1.hm=g.id1, ​​and all the information of A1 is obtained. If G can be associated, the information of G is obtained (if it cannot be associated, it is left blank), and the final result table H is obtained (it can be sorted by group_name and id1, and the result is more readable):

[0123] Table 9-1 Table H output results

[0124]

[0125] The final result data is as follows:

[0126] Table 9-2 Table H output results

[0127]

[0128] The key points and protection points of the present invention are:

[0129] 1) Only large time scales can precipitate regularities;

[0130] 2) The relationship between group members is characterized by the relationship between two members;

[0131] 3) The measurement of “number pair” distance only considers clustering but not isolation.

[0132] The closeness measurement of the present invention is carried out on a "large time scale". The gathering of group members at a certain time and event does not explain anything. Within a large time range, it can be seen which members often gather together and summarize the rules. The distance formula is: distance = 1-n_cnt / u_cnt, and the value is relative. In this way, the statistical method is used to overcome the weakness of a single data source. The existing technology solves this problem by collecting multi-dimensional data.

[0133] The relationship between group members of the present invention is described by the relationship between two members, which also simplifies the complexity. The prior art is solved by advanced technology.

[0134] The measure of "closeness" of "number pairs" is the ratio of simultaneous clustering to separate clustering, and the distance is 1 minus "closeness". The number of "isolated" in a "number pair" is not counted, that is, the distance of "number pairs" is considered in the context of "clustering".

Claims

1. A method for realizing the closeness relationship among members of multiple groups based on the set aggregation k-means method, characterized in that: The steps include: Step 1: Starting from the aggregation relationship table A, first associate it with the group B to which the number belongs to obtain table C; The fields in the aggregation relationship table A include: the distance information of the set S when the numbers in the same cluster take the same value of feature F; Table C only has the group name in addition to the aggregation relationship table A; Step 2: From Table C, label the day, cluster, and group with the number cluster_id2 to obtain Table D; Step 3: Find the number of different number pairs of cluster_id2 in Table D, and sum the number of times the same number pair comes together in a certain period of time. The result is n_cnt, and Table E is obtained; Step 4: Find the number of cluster_id2 different number pairs in Table D that have at least one occurrence, and sum the number of the same number pair in a certain period of time. The result is u_cnt, and Table F is obtained; Step 5: Table E and Table F are associated by number pairs, and the distance is calculated using the formula: distance = 1-n_cnt / u_cnt, to obtain Table G; Step 6: Table A1 with the position of the numbers in the group is associated with G according to a1.hm=g.id1, ​​and the information of all A1 is obtained at the same time. If G can be associated, the information of G is obtained to obtain the final result table H; a1.hm represents the number column of table A1, and g.id1 represents the number 1 column of table G.

2. A computer program, characterized in that The computer program enables a computer to execute the method as claimed in claim 1.

3. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as claimed in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method as claimed in claim 1 is implemented.

5. A chip, characterized in that: include: A processor, used to call and run a computer program from a memory, so that a device equipped with the chip executes the method as claimed in claim 1.

6. A computer program product, characterized in that The computer program product comprises a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the method according to claim 1 is implemented.

Images