An unsupervised clustering method for social text data based on event graph model

Abstract

The present invention discloses an unsupervised clustering method for social text data based on an event graph model. The method comprises: obtaining a first node and a second node in a social text data stream, mining the node relationship between the first node and the second node, and generating an initial social event graph between the social text data; obtaining the keyword of the second node, and calculating the local importance of the keyword; obtaining the common keyword of the first node and the second node, and calculating the node relationship weight corresponding to each node relationship; mapping the initial social event graph to generate a target social event graph according to the node relationship weight; performing a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result; and returning the target clustering result according to the node relationship to generate a clustering result of the social text data. By fully mining the potential relationships between social data, efficient clustering of social text is achieved, providing guarantees for the data integrity and timeliness of subsequent public opinion analysis.

Description

Technical Field

[0001] The present invention belongs to the field of computer data mining, and more specifically, relates to an unsupervised clustering method for social text data based on an event graph model. Background Art

[0002] Due to the openness of various social platforms, once an online event is released, it will trigger extensive discussion among netizens on social platforms and generate a large amount of text data. Social text data plays an important role in tracking current events and analyzing public opinion.

[0003] Clustering is a commonly used method in public opinion analysis. Clustering can extract information about related events from social text data and provide it for further public opinion analysis (sentiment analysis, fake news detection, etc.). Traditional text clustering is achieved by calculating the similarity between texts. In the existing technology, texts are represented as vectors through the vector space model (VSM), and then they are grouped using common clustering algorithms such as HAC, K-means, PLPA, etc. In order to overcome the sparsity and high-dimensional characteristics of this representation, some studies focus on effective feature extraction to reduce the dimensionality of feature vectors, and some studies focus on model-based clustering such as DMM, GSDMM, etc. In recent years, neural networks have been widely used in text classification, such as Text-GCN, text-level GNN, etc.

[0004] However, because valuable terms in short texts appear infrequently, processing time is wasted and results are poor. Model-based approaches cluster text by building probabilistic models. These approaches ignore the underlying relationships between text data in social media, resulting in poor clustering quality and interpretability. The application of neural networks to text classification relies on manually labeled training sets and requires a long learning process. With the explosive growth of social media text, these approaches, due to long clustering times, high memory usage, and poor clustering results, cannot simultaneously meet the data integrity and timeliness requirements of subsequent public opinion analysis applications. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides an unsupervised clustering method for social text data based on an event graph model, which aims to solve the problems of slow clustering speed, high clustering overhead, and poor interpretability caused by the explosive growth of social data.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for unsupervised clustering of social text data based on an event graph model is provided, the method comprising the following steps:

[0007] Obtaining a first node and a second node in a social text data stream, and mining a node relationship between the first node and the second node to generate an initial social event graph between the social text data;

[0008] Obtaining a keyword of the second node and calculating the local importance of the keyword; obtaining a common keyword of the first node and the second node, and calculating a node relationship weight corresponding to each of the node relationships according to the local importance of the keyword of the second node and the local importance of the common keyword;

[0009] Mapping the initial social event graph to generate a target social event graph according to the node relationship weights;

[0010] Performing a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result;

[0011] The target clustering result is returned according to the node relationship to generate a clustering result of the social text data.

[0012] Optionally, generating an initial social event graph between each social text data according to the node relationship between the first node and the second node includes:

[0013] Determining a node relationship between the first node and the second node;

[0014] If the first node includes the second node, the node relationship between the first node and the second node is a direct relationship;

[0015] If the first node and the second node have a common keyword, the node relationship between the first node and the second node is an indirect relationship;

[0016] The associated first node and second node are connected according to the node relationship to generate the initial social event graph.

[0017] Optionally, obtaining the keyword of the second node and calculating the local importance of the keyword includes:

[0018] Obtain keywords from each second node based on text segmentation, traverse all second nodes, and calculate the local importance of the keywords of the second nodes:

[0019]

[0020] Where N represents all second nodes {t1, t2, ... t N}, T j Represents the second node t jThe keyword set after word segmentation (1 <j<N)。

[0021] Optionally, calculating the node relationship weight corresponding to each of the node relationships according to the local importance of the keyword of the second node and the common keyword includes:

[0022] If the node relationship between the first node and the second node is a direct relationship, the node relationship weight is a direct relationship weight, and the direct relationship weight is 1;

[0023] If the node relationship between the first node and the second node is an indirect relationship, the node relationship weight is an indirect relationship weight; the indirect relationship weight:

[0024]

[0025] Among them, k c Represents the second node t j The keyword set T after word segmentation j and the first node b i The keyword set B after word segmentation j The common keywords in k a Represents the second node t j Keyword set T j All keywords in .

[0026] Optionally, when the first node and the second node are in a direct relationship, the second node is the source topic of the first node; when the first node and the second node are in an indirect relationship, the second node is the destination topic of the first node;

[0027] The step of mapping the initial social event graph to generate a target social event graph according to the node relationship weights includes:

[0028] Determining a potential relationship between the source topic and the destination topic using the relationship between the first node and the source topic and the indirect relationship between the first node and the destination topic as mapping conditions;

[0029] If there are multiple potential relationships between the source topic and the target topic, the potential relationship with the largest weight is selected as the target potential relationship;

[0030] The initial social event graph is mapped into a target social event graph according to the mapping condition.

[0031] Optionally, performing a depth-first traversal on all nodes in the target social event graph to obtain a target clustering result includes:

[0032] Performing a bidirectional depth-first sequence traversal on the current node in the target social event graph, and if the target potential relationship weight between two nodes is greater than a first threshold, determining that the corresponding two nodes are of the same type;

[0033] The depth-first sequence traversal of the current node is completed, and the bidirectional depth-first sequence traversal is performed again on the new node.

[0034] Optionally, performing a depth-first traversal on all nodes in the target social event graph to obtain a target clustering result further includes:

[0035] Assign the same label to nodes of the same class; if two nodes of the same class do not have a label, a label is automatically assigned; if one of the two nodes of the same class has a label, the label is propagated to the unlabeled node;

[0036] Get the labels of nodes with only one-way relationships and their neighbor nodes, and determine the label with the most occurrences as the label of the node;

[0037] Cluster the nodes according to the labels to obtain the target clustering result.

[0038] Optionally, the generating a clustering result of the social text data by returning the target clustering result according to the node relationship includes:

[0039] According to the node relationship in the initial social event graph, the target clustering result is returned, and the label of the second node is propagated to the first node having a direct relationship therewith, so as to generate a clustering result of the social text data.

[0040] According to another aspect of the present invention, a social text data unsupervised clustering system based on an event graph model is provided, comprising:

[0041] An event graph generation module is configured to obtain a first node and a second node in a social text data stream, and mine a node relationship between the first node and the second node to generate an initial social event graph between the social text data;

[0042] a weight calculation module, configured to obtain a keyword of the second node and calculate the local importance of the keyword; obtain a common keyword of the first node and the second node, and calculate a node relationship weight corresponding to each of the node relationships based on the local importance of the keyword of the second node and the local importance of the common keyword;

[0043] A relationship mapping module, configured to map the initial social event graph to generate a target social event graph according to the node relationship weights;

[0044] A graph clustering module is used to perform a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result; and to return the target clustering result based on the node relationship to generate a clustering result of the social text data.

[0045] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0046] (1) The present invention proposes to construct a social event graph model to transform unstructured social text data into a graph structure, which helps to explore the potential relationships between social data and realize efficient clustering of social text data.

[0047] (2) The keyword-based local weight algorithm proposed in this invention can flexibly calculate the importance of keywords in topics, and thus accurately express the indirect relationship weights in the social event graph.

[0048] (3) The relationship mapping mechanism proposed in this invention maps the social event graph into a smaller domain, and proposes a deep clustering scheme based on bidirectional relationships according to the node relationship characteristics of the social event graph, which can achieve unsupervised and efficient clustering of social text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of an unsupervised clustering method for social text data based on an event graph model disclosed in the first embodiment of the present invention;

[0050] Figure 2 This is an example diagram of the social event graph model disclosed in the first embodiment of the present invention;

[0051] Figure 3 This is an example diagram of the potential relationship mapping diagram disclosed in the first embodiment of the present invention;

[0052] Figure 4 This is an example diagram of an unsupervised clustering graph of the social event model disclosed in Example 1 of the present invention;

[0053] Figure 5 This is a structural diagram of an unsupervised clustering system for social text data based on an event graph model disclosed in Example 2 of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0055] Example 1

[0056] like Figure 1 As shown in FIG, an unsupervised clustering method for social text data based on an event graph model includes:

[0057] S100: Acquire a first node and a second node in a social text data stream, and mine the node relationship between the first node and the second node to generate an initial social event graph between the social text data;

[0058] S200, obtaining a keyword of the second node and calculating the local importance of the keyword; obtaining a common keyword of the first node and the second node, and calculating a node relationship weight corresponding to each node relationship based on the local importance of the keyword of the second node and the local importance of the common keyword;

[0059] S300, mapping the initial social event graph to generate a target social event graph according to the node relationship weights;

[0060] S400, performing a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result;

[0061] S500: Generate a clustering result of social text data based on the target clustering result returned according to the node relationship.

[0062] like Figure 2 As shown, the first node is the blog node b, and the second node is the topic node t. Social text data contains multiple first nodes and second nodes. A blog may be related to multiple topics, and a topic may correspond to multiple blog contents, that is, a first node may be related to multiple second nodes, and a second node may be related to multiple first nodes. Mining the nodes contained in the social text data and judging the relationship between the nodes, if the first node contains the second node, then the node relationship between the first node and the second node is a direct relationship; if the first node and the second node have a common keyword, then the node relationship between the first node and the second node is an indirect relationship. Define that the blog node b1 with the initial topic has a direct relationship with its topic node t1, so there is a direct relationship b1→t1. If the blog node b1 contains the keyword in the topic node t2, then there is a b1→t2 relationship, which is an indirect relationship. By performing node extraction and relationship mining on the social text stream, an event-oriented initial social event graph G(V,E) is obtained, where V={b1,b2,…b N ..,t1,t2,…t M}, V contains two types of nodes, E={E direct ,E indirect}, E contains two types of relationships (direct relationships and indirect relationships).

[0063] The second node may contain multiple keywords, and the degree of association between each keyword and other second nodes may be different, that is, the local importance of each keyword is different. The greater the local importance, the more representative the keyword. Based on the text segmentation, keywords are obtained from each second node, and all second nodes are traversed to calculate the local importance of the keywords of the second nodes:

[0064]

[0065] Where N represents all second nodes {t1, t2, ... t N}, T j Represents the second node t j The keyword set after word segmentation (1 <j<N)。

[0066] After obtaining the local importance of the keyword, the relationship weight between the first node and the second node must be calculated. The larger the relationship weight, the closer the connection between the first node and the second node.

[0067] If the node relationship between the first node and the second node is a direct relationship, the node relationship weight is the direct relationship weight, and the direct relationship weight is 1.

[0068] If the node relationship between the first node and the second node is an indirect relationship, the node relationship weight is the indirect relationship weight; the indirect relationship weight is:

[0069]

[0070] Among them, k c Represents the second node t j The keyword set T after word segmentation j and the first node b i The keyword set B after word segmentation j The common keywords in k a Represents the second node t j Keyword set T j All keywords in .

[0071] Step S300 specifically includes:

[0072] S310: Determine a potential relationship between the source topic and the destination topic using the relationship between the first node and the source topic and the indirect relationship between the first node and the destination topic as mapping conditions.

[0073] When the first node and the second node are in direct relationship, the second node is the source topic of the first node; when the first node and the second node are in indirect relationship, the second node is the destination topic of the first node.

[0074] For example, a first node b1 has a direct relationship with a second node t1 (source topic), and this first node b1 has an indirect relationship with another second node t2 (destination topic). It is defined that there is a potential relationship between the source topic t1 of the first node b1 and its destination topic t2, which is expressed as t1→t2.

[0075] S320: If there are multiple potential relationships between the source topic and the destination topic, select the potential relationship with the largest weight as the target potential relationship.

[0076] Since there are multiple first nodes with the topic label t1 in the social text, and these first nodes may have an indirect relationship with the same topic t2, there will be many potential relationships with different weight values ​​between t1 and t2. The maximum weight is selected as the relationship weight between t1 and t2, and the corresponding potential relationship with the largest weight is the target potential relationship.

[0077] S330: Map the initial social event graph to a target social event graph according to the mapping condition.

[0078] like Figure 3 As shown in (a), in G(V,E), b1→t1, b2→t1 are direct relationships, b1→t3, b2→t2 are indirect relationships; the second nodes t1 and t2 are the source topic and destination topic of the first node b2 respectively. After the mapping of potential relationships, in G'(V',E'), there are potential relationships between topic nodes, such as t1→t2, t3→t2, etc. Figure 3 As shown in (b), two potential relationships are generated between t2 and t3. This is because b4 and b5 have the same source topic and destination topic. At this time, the potential relationship with the largest weight will be retained. Therefore, the weight of t3→t2 is max(w 32 ,w 32 ').

[0079] By mining the potential associations between direct and indirect relationships, the initial social event graph G(V,E) is mapped to the target social time graph G'(V',E'), where V'={t1,t2,…t M},E'={E potential By mapping the node relationship between a first node and a second node into a node relationship between multiple second nodes, it not only helps to mine the potential relationships between social data, but also simplifies massive social text data and maps the social event graph into a smaller domain, thereby improving the efficiency of social text data.

[0080] Step S400 specifically includes:

[0081] S410: Perform a bidirectional depth-first sequence traversal on the current node in the target social event graph. If the target potential relationship weight between two nodes is greater than a first threshold, determine that the corresponding two nodes are of the same type.

[0082] For the current node t i Perform a bidirectional depth-first traversal, if and Then t i and t j are considered to belong to the same class. ij Represents the relationship weight, and A represents the first threshold. Nodes with a bidirectional relationship and a relationship weight greater than the first threshold are considered to be of the same type.

[0083] The same label is assigned to nodes of the same class. If two nodes of the same class do not have a label, a label is automatically assigned. If one of the two nodes of the same class has a label, the label is propagated to the node without a label.

[0084] For example, if node t1 and node t2 are identified as belonging to the same class, node t1 has label L1 and node t2 does not have a label, then the label L1 of node t1 is propagated to node t2. At this time, both node t1 and node t2 have label L1. If both node t1 and node t2 do not have a label, the system automatically assigns a label L1. i Given node t1 and node t2.

[0085] S420: The depth-first sequence traversal of the current node is completed, and a bidirectional depth-first sequence traversal is performed again on the new node.

[0086] S430: For nodes in the target social event graph that only have unidirectional relationships, set a second threshold B. If the target potential relationship weight between two nodes is greater than the second threshold, determine that the corresponding two nodes are of the same type.

[0087] Node t1 and node t2 will be classified into the same category only when they have a bidirectional relationship. If node t1 only has a unidirectional relationship, it cannot be classified into the same category as other nodes.

[0088] For nodes with only one-way relationships, obtain the labels of neighboring nodes whose relationship weight is greater than B, and determine the label with the largest number of occurrences as the label of the node, so as to classify nodes with only one-way relationships within a certain range.

[0089] S440: Cluster the nodes according to the labels to obtain the target clustering result.

[0090] like Figure 4As shown in (a), in G'(V',E'), the first nodes t1 and t2 have a bidirectional relationship And w 12 >A,w 21 > A. Through depth-first traversal, t1 and t2 are aggregated into the same class e2.

[0091] According to the characteristics of the target social event graph, a deep clustering scheme based on bidirectional relationships is proposed, which can achieve unsupervised and efficient clustering of social text data.

[0092] Step S500 specifically includes:

[0093] According to the node relationship in the initial social event graph, the target clustering result is returned, and the label of the second node is propagated to the first node having a direct relationship therewith, to generate a clustering result of the intersection text data stream.

[0094] like Figure 4 As shown in (b), during the return process of the clustering results, the second node propagates its label to the first node with which it has a direct relationship. Each second node has a direct relationship with the corresponding first node, thereby obtaining the clustering result of the cross-text data stream.

[0095] The contents involved in the above embodiment are described below in conjunction with a preferred embodiment.

[0096] For example, trending topics on social media reflect real-world events. As an event unfolds, develops, and fades, it generates numerous interrelated topics on social media. For example, the 2021 Zhengzhou rainstorm, which initially trended as #Zhengzhou rainstorm floods subway stations#, evolved into #Searching for Zou Deqiang's 72 hours#. Because social network data contains freely expressed opinions, within a data stream captured over a period of time, text content labeled topic A often includes keywords from topic B originating from the same event. As related topics derived from the same event become widely discussed online, a network of latent relationships between topics is formed. By establishing these latent relationships, social text data related to the same event can be effectively clustered together, ensuring data integrity for subsequent public opinion analysis.

[0097] The technical solution of the embodiment of the present invention transforms unstructured social text data into a graph structure, constructs an initial social event graph, and mines the potential relationships between social data; flexibly calculates the importance of keywords in social topics through local importance calculation, thereby accurately calculating the node relationship weights between nodes in the social event graph; then, based on the relationship between nodes, a mapping condition is constructed to map the initial social event graph with more data to a smaller domain to generate a target social event graph; then, based on the characteristics of the social event graph, a deep clustering method based on bidirectional relationships is proposed to cluster the target social event graph to obtain the target clustering result; finally, the clustering result of the cross-text data stream is generated based on the node relationship feedback. This solves the technical problems such as slow clustering speed, high clustering overhead, and poor interpretability caused by the explosive growth of social data, and achieves efficient clustering of social text data with higher clustering performance and less time overhead, while meeting the requirements of subsequent public opinion analysis applications for data integrity and timeliness.

[0098] Example 2

[0099] like Figure 5 As shown in FIG, an unsupervised clustering system for social text data based on an event graph model includes:

[0100] An event graph generation module is configured to obtain a first node and a second node in a social text data stream, and mine a node relationship between the first node and the second node to generate an initial social event graph between the social text data;

[0101] a weight calculation module, configured to obtain a keyword of the second node and calculate the local importance of the keyword; obtain a common keyword of the first node and the second node, and calculate a node relationship weight corresponding to each of the node relationships based on the local importance of the keyword of the second node and the local importance of the common keyword;

[0102] A relationship mapping module, configured to map the initial social event graph to generate a target social event graph according to the node relationship weights;

[0103] A graph clustering module is used to perform a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result; and to return the target clustering result based on the node relationship to generate a clustering result of the social text data.

[0104] An unsupervised clustering system for social text data based on an event graph model provided by an embodiment of the present invention can execute an unsupervised clustering method for social text data based on an event graph model provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0105] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An unsupervised clustering method for social text data based on event graph model, characterized by: The method comprises the following steps: Obtain a first node and a second node in a social text data stream, mine a node relationship between the first node and the second node, and determine the node relationship between the first node and the second node; if the first node contains the second node, then the node relationship between the first node and the second node is a direct relationship; if the first node and the second node have a common keyword, then the node relationship between the first node and the second node is an indirect relationship; connect the associated first node and the second node according to the node relationship to generate an initial social event graph; Obtaining a keyword of the second node and calculating the local importance of the keyword; obtaining a common keyword of the first node and the second node, and calculating a node relationship weight corresponding to each of the node relationships according to the local importance of the keyword of the second node and the local importance of the common keyword; Mapping the initial social event graph to generate a target social event graph according to the node relationship weights; Performing a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result; The target clustering result is returned according to the node relationship to generate a clustering result of the social text data.

2. The method according to claim 1, wherein The obtaining of the keyword of the second node and calculating the local importance of the keyword includes: Obtain keywords from each second node based on text segmentation, traverse all second nodes, and calculate the local importance of the keywords of the second nodes: Where N represents all second nodes {t1, t2, ... t N }, T j Represents the second node t j The keyword set after word segmentation (1 <j<N)。 3. The method according to claim 2, wherein The calculating the node relationship weight corresponding to each of the node relationships according to the local importance of the keyword of the second node and the common keyword includes: If the node relationship between the first node and the second node is a direct relationship, the node relationship weight is a direct relationship weight, and the direct relationship weight is 1; If the node relationship between the first node and the second node is an indirect relationship, the node relationship weight is an indirect relationship weight; the indirect relationship weight: Among them, k c Represents the second node t j The keyword set T after word segmentation j and the first node b i The keyword set B after word segmentation j The common keywords in k a Represents the second node t j Keyword set T j All keywords in .

4. The method according to claim 1, wherein When the first node and the second node are in a direct relationship, the second node is the source topic of the first node; when the first node and the second node are in an indirect relationship, the second node is the destination topic of the first node; The step of mapping the initial social event graph to generate a target social event graph according to the node relationship weights includes: Determining a potential relationship between the source topic and the destination topic using the relationship between the first node and the source topic and the indirect relationship between the first node and the destination topic as mapping conditions; If there are multiple potential relationships between the source topic and the target topic, the potential relationship with the largest weight is selected as the target potential relationship; The initial social event graph is mapped into a target social event graph according to the mapping condition.

5. The method according to claim 1, wherein The performing a depth-first traversal on all nodes in the target social event graph to obtain a target clustering result includes: Performing a bidirectional depth-first sequence traversal on the current node in the target social event graph, and if the target potential relationship weight between two nodes is greater than a first threshold, determining that the corresponding two nodes are of the same type; The depth-first sequence traversal of the current node is completed, and the bidirectional depth-first sequence traversal is performed again on the new node.

6. The method according to claim 5, wherein The performing a depth-first traversal on all nodes in the target social event graph to obtain a target clustering result further includes: Assign the same label to nodes of the same class; if two nodes of the same class do not have a label, a label is automatically assigned; if one of the two nodes of the same class has a label, the label is propagated to the unlabeled node; Get the labels of nodes with only one-way relationships and their neighbor nodes, and determine the label with the most occurrences as the label of the node; Cluster the nodes according to the labels to obtain the target clustering result.

7. The method according to claim 6, wherein The step of returning the target clustering result according to the node relationship to generate the clustering result of the social text data includes: According to the node relationship in the initial social event graph, the target clustering result is returned, and the label of the second node is propagated to the first node having a direct relationship therewith, so as to generate a clustering result of the social text data.

8. An unsupervised clustering system for social text data based on an event graph model, characterized in that: include: An event graph generation module is configured to obtain a first node and a second node in a social text data stream, mine a node relationship between the first node and the second node, and determine the node relationship between the first node and the second node; if the first node contains the second node, the node relationship between the first node and the second node is a direct relationship; if the first node and the second node have a common keyword, the node relationship between the first node and the second node is an indirect relationship; Connecting the associated first node and the second node according to the node relationship to generate an initial social event graph; a weight calculation module, configured to obtain a keyword of the second node and calculate the local importance of the keyword of the second node; obtain a common keyword of the first node and the second node, and calculate a node relationship weight corresponding to each of the node relationships based on the local importance of the keyword and the local importance of the common keyword; A relationship mapping module, configured to map the initial social event graph to generate a target social event graph according to the node relationship weights; A graph clustering module is used to perform a depth-first sequence traversal on all nodes in the target social event graph to obtain a target clustering result; And the target clustering result is returned according to the node relationship to generate a clustering result of the social text data.

Images